Low-entropy browsing history for semi-personalized content

Abstract

This disclosure relates to low-entropy browsing history for quasi-personalized content. It provides systems and methods for retrieving quasi-personalized or anonymized content from aggregated browsing histories across multiple devices, such as millions or billions of devices. A sparse matrix can be constructed from the aggregated browsing history and reduced in dimensionality, thereby reducing entropy and providing anonymity for individual devices. Relevant content can be selected via quasi-personalized clustering to represent similar browsing histories without exposing individual device details to the content provider.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese invention patent application 202080006300.4, filed on March 31, 2020.

[0003] Cross-reference to related applications

[0004] This application claims priority to U.S. Patent Application No. 16 / 698,548, filed November 27, 2019, entitled “Low Entropy Browsing History for Content Quasi-Personalization,” and U.S. Patent Application No. 16 / 535,912, filed August 8, 2019, entitled “Low Entropy Browsing History for Content Quasi-Personalization,” and U.S. Patent Application No. 62 / 887,902, filed August 16, 2019, entitled “Low Entropy Browsing History for Content Quasi-Personalization,” the entire contents of each of which are incorporated herein by reference. Technical Field

[0005] This disclosure relates to low-entropy browsing history used for quasi-personalized content. Background Technology

[0006] In computer networking environments such as the Internet, content providers can offer content items to be inserted into information resources (e.g., web pages) processed and rendered by applications (e.g., web browsers) running on client devices.

[0007] Personalized content delivery is typically based on capturing identifying information about the user and / or device, such as the device's browsing or access history. However, this can lead to the collection of separately trackable data, exposing users to potential security risks and consuming excessive computing resources. Summary of the Invention

[0008] The systems and methods discussed in this paper provide quasi-personalized or anonymized content retrieval via aggregated browsing histories of millions or billions of devices. Sparse matrices can be constructed from aggregated browsing histories and reduced in dimensionality to decrease entropy and provide anonymity for individual devices. Relevant content can be selected via quasi-personalized clustering to represent similar browsing histories without exposing individual device details to content providers.

[0009] In one aspect, this disclosure relates to a method for anonymized content retrieval. The method includes generating a profile based on the browsing history of a browser application on a computing device. The method further includes encoding the profile into an n-dimensional vector by the browser application. The method further includes calculating a reduced-dimensional vector by the browser application based on the n-dimensional vector. The method further includes determining a first cluster corresponding to the reduced-dimensional vector by the browser application. The method further includes sending a request for content items to a content server by the browser application, the request including an identifier of the first cluster. The method further includes receiving, by the browser application, content items selected from the content server based on the identifier of the first cluster.

[0010] In some embodiments, the method includes generating a profile based on the browsing history by identifying the number of visits to each of a plurality of addresses within a predetermined time period from a log of the user's browsing history on the computing device. In some embodiments, the method includes generating a string having a value representing each of one or more visits to an address associated with a corresponding location in the string within the predetermined time period.

[0011] In some embodiments, the method includes performing singular value decomposition on an n-dimensional vector. In another embodiment, the method includes receiving a set of singular vectors from a second computing device. In yet another embodiment, the method includes transmitting an n-dimensional vector to a second computing device, which computes the set of singular vectors based on an aggregation of the n-dimensional vector from the computing device and n-dimensional vectors from at least one other computing device.

[0012] In some embodiments, the method includes receiving the boundary of each of a plurality of clusters from a second computing device. In another embodiment, the method includes selecting a first cluster of the plurality of clusters in response to a dimensionality reduction vector being within the boundary of a first cluster.

[0013] In some implementations, the method includes receiving weights of a neural network model from a second computing device, determined by an aggregation of an n-dimensional vector from that computing device and an n-dimensional vector from at least one other computing device, by a browser application; applying the neural network model to the reduced-dimensional vectors using a machine learning system of the browser application to generate a ranking of a predetermined set of clusters; and selecting a first cluster as the highest-ranking cluster in the predetermined set of clusters by the browser application.

[0014] In another aspect, this disclosure relates to a method for anonymized content retrieval. The method includes receiving profiles based on the browsing history of each of a plurality of client computing devices from a server computing device, each profile including an n-dimensional vector. The method further includes the server computing device aggregating the n-dimensional vectors of the plurality of profiles into a matrix. The method also includes the server computing device computing a singular value decomposition of the matrix to generate a set of singular values. Finally, the method includes the server computing device transmitting the set of singular values ​​to each of the plurality of client computing devices and at least one content provider device.

[0015] In some implementations, the method includes determining the boundaries of each cluster in the cluster set of the matrix by a server computing device. In another implementation, the method includes transmitting the boundaries of each cluster in the cluster set of the matrix by the server computing device to each of a plurality of client computing devices and at least one content provider device.

[0016] In some embodiments, the method includes identifying each cluster in the cluster set from the matrix by the server computing device via a neural network model. In another embodiment, the method includes transmitting weights of the neural network model to each of a plurality of client computing devices and at least one content provider device.

[0017] In another aspect, this disclosure relates to a system for anonymized content retrieval. The system includes: a computing device including a network interface for communicating with a content server; a memory storing the browsing history of the computing device; and a browser application. The browser application is configured to: generate a profile based on the browsing history of the computing device; encode the profile into an n-dimensional vector; calculate a reduced-dimensional vector based on the n-dimensional vector; determine a first cluster corresponding to the reduced-dimensional vector; transmit a request for content items to the content server via the network interface, the request including an identifier of the first cluster; and receive, via the network interface, content items selected from the content server based on the identifier of the first cluster.

[0018] In some embodiments, the browser application is also configured to generate a string with a value representing each of one or more accesses to an address associated with a corresponding location in the string within a predetermined time period. In some embodiments, the browser application is also configured to perform singular value decomposition of an n-dimensional vector. In another embodiment, the browser application is further configured to receive a set of singular vectors from the singular value decomposition from a second computing device via a network interface. In yet another embodiment, the browser application is further configured to transmit an n-dimensional vector to a second computing device via a network interface, the second computing device computing the set of singular vectors based on an aggregation of the n-dimensional vector from the computing device and n-dimensional vectors from at least one other computing device.

[0019] In some implementations, the browser application is further configured to receive, via a network interface, weights of a neural network model determined by the aggregation of an n-dimensional vector from the second computing device and an n-dimensional vector from at least one other computing device; apply the neural network model to the reduced-dimensional vectors to generate a ranking of a predetermined set of clusters; and select a first cluster as the highest-ranking cluster in the predetermined set of clusters.

[0020] At least one aspect relates to a method for encoding identifiers for content selection. A first application executing on a client device can identify a browsing history maintained on the client device. The browsing history can record information resources accessed by the client device via the first application. The first application can apply a classification model to its browsing history to identify a category to which the first application is classified. The first application can assign a category identifier corresponding to the identified category. The category identifier for the first application can be the same as the category identifier for a second application. The first application can receive information resources from a content publisher device, the information resources including main content and content slots that can be used to receive content from a content selection service. The first application can generate a request for content for the content slots of the information resource, the request for content including the category identifier for the first application. The first application can transmit the request for content to the content selection service. The content selection service can use the category identifiers for the first application and the second application to select content items to insert into the content slots of the information resource.

[0021] In some implementations, the first application may receive a set of parameters from an application manager service to update a classification model used to categorize the application into one of several categories. In some implementations, the first application may modify a classification model maintained on a client device based on the set of parameters received from the application manager service. In some implementations, the first application may collaboratively train the classification model maintained on the client devices using a distributed learning protocol with multiple application systems running on corresponding client devices.

[0022] In some implementations, the first application can generate a simplified set of feature vectors based on the browsing history identified from the client device using a dimensionality reduction process. These feature vectors are smaller in file size than the browsing history. In some implementations, applying the classification model can include applying the classification model to the simplified set of feature vectors generated from the browsing history.

[0023] In some implementations, the first application may determine whether to apply the classification model to the browsing history based on an identifier allocation strategy. In some implementations, applying the classification model may include: applying the classification model in response to determining that it will be applied according to the identifier allocation strategy.

[0024] In some implementations, the application classification model may include an application classification model to identify categories from a plurality of categories. Each of the plurality of categories may have at least a predetermined number of clients to be classified into that category. In some implementations, assigning a first application to a category identifier may also include assigning the first application to a category identifier among a plurality of category identifiers. Each category identifier may correspond to one of the plurality of categories.

[0025] In some implementations, generating a request for content may include generating a request for content in the absence of a unique tracking identifier corresponding to an account associated with the first application, the first application, or a client device provided by the content selection service. In some implementations, generating a request for content may include generating a request for content including a security cookie for transmission via a connection to the content selection service. The security cookie may have a category identifier for the first application. In some implementations, identifying browsing history may include identifying the browsing history within a predetermined time frame in which the classification model is applied.

[0026] At least one aspect relates to a system for encoding identifiers for content selection. The system may include a first application executable on a client device having one or more processors. The first application executing on the client device may identify a browsing history maintained on the client device. The browsing history may record information resources accessed by the client device via the first application. The first application may apply a classification model to the browsing history of the first application to identify a category to which the first application is classified. The first application may assign the first application to a category identifier corresponding to the identified category. The category identifier for the first application may be the same as the category identifier for a second application. The first application may receive information resources from a content publisher device, the information resources including main content and content slots that can be used to receive content from a content selection service. The first application may generate a request for content for the content slots of the information resource, the request for content including the category identifier for the first application. The first application may transmit the request for content to the content selection service. The content selection service may use the category identifiers for the first application and the second application to select content items to insert into the content slots of the information resource.

[0027] In some implementations, a first application may receive a set of parameters from an application manager service to update a classification model used to classify the application into one of a plurality of categories. In some implementations, the first application may modify a classification model maintained on a client device based on the set of parameters received from the application manager service. In some implementations, the first application may collaboratively train the classification model maintained on the client devices using a distributed learning protocol with multiple application systems running on corresponding client devices.

[0028] In some implementations, the first application may use a dimensionality reduction process to generate a simplified set of feature vectors based on the browsing history identified from the client device, the feature vectors being smaller in file size than the browsing history. In some implementations, the first application may apply a classification model to the simplified set of feature vectors generated from the browsing history.

[0029] In some implementations, the first application may determine whether to apply a classification model to the browsing history based on an identifier allocation strategy. In some implementations, the first application may apply the classification model in response to determining that a classification model should be applied according to the identifier allocation strategy.

[0030] In some implementations, the first application may apply a classification model to identify the categories from a plurality of categories. Each of the plurality of categories may have at least a predetermined number of clients to be classified into that category. In some implementations, the first application may be assigned to a category identifier among a plurality of category identifiers. Each category identifier may correspond to one of the plurality of categories.

[0031] In some implementations, the first application generates a request for content when no unique tracking identifier corresponds to the account associated with the first application, the first application itself, or the client device provided by the content selection service. In some implementations, the first application generates a request for content including a security cookie for transmission via a connection to the content selection service. The security cookie may have a category identifier for the first application. In some implementations, identifying browsing history may include identifying the browsing history within a predetermined time range in which the classification model is applied.

[0032] This disclosure also provides a computer program including instructions that, when executed by a computing device, cause the computing device to perform any of the methods disclosed herein. This disclosure also provides a computer-readable medium including instructions that, when executed by a computing device, cause the computing device to perform any of the methods disclosed herein.

[0033] At least one aspect relates to a method for encoding an identifier for content selection. The method may include identifying a plurality of information resources accessed via a first application executed on a client device. The method may include extracting features from each of the plurality of information resources corresponding to at least a portion of the content of that information resource. The method may include applying a classification model to the features extracted from the plurality of information resources to identify a set of categories to which the first application is classified. The method may include determining a threshold number of applications assigned to a category from the set of categories. The method may include assigning the first application to a category identifier corresponding to the category in response to determining that the category satisfies the threshold number. The category identifier for the first application may be the same as the category identifier for a second application. The method may include receiving information resources from a content publisher device for presentation via the first application, the information resources including main content and content slots available for receiving content from a content selection service. The method may include generating a request for content for the content slots of the information resource, the request for content including the category identifier for the first application. The method may include sending the request for content to the content selection service, which uses category identifiers for the first application and the second application to select content items to be inserted into the content slots of the information resource.

[0034] In some implementations, the method may include: in response to receiving the information resource, selecting a category from the category set according to an obfuscation strategy. The obfuscation strategy may specify a condition under which a corresponding category identifier is allowed to be included in a request for content associated with the content slot of the information resource.

[0035] In some implementations, the method may include: in response to receiving a second information resource, selecting a second category identifier from the set of categories identified by the classification model applied, according to an obfuscation strategy. The second category identifier may be different from the category assigned to the first application. In some implementations, the method may include generating a second request for content for a content slot of the second information resource, the second request for content including a second category identifier corresponding to the second category, rather than the category identifier corresponding to the category.

[0036] In some implementations, the method may include: in response to receiving a second information resource, determining, according to an obfuscation policy, that any category identifier is not included in a second request for content to be inserted into a content slot of the second information resource. In some implementations, the method may include: in response to determining that no category identifier is included, transmitting the second request for content to the content selection service, the second request for content lacking any category identifier.

[0037] In some implementations, determining the number of times a category satisfies the threshold may include: performing a threshold cryptography protocol in collaboration with a category authorization service by using an encrypted category identifier generated using the category identifier corresponding to the category. In some implementations, determining the number of times a category satisfies the threshold may include: checking the category identifier against a probabilistic data structure for category identifiers maintained by the category authorization service.

[0038] In some implementations, sending the request for content may include sending the request for content. The content selection service may use the category identifier to maintain aggregated browsing history for the first application and the second application. In some implementations, applying the classification model may include: applying the classification model to identify a set of categories that are within a proximity threshold of each other in a feature space defined by the classification model from a plurality of categories.

[0039] In some implementations, the method may include generating a simplified set of feature vectors based on the browsing history identified from the client device using a dimensionality reduction process, the feature vectors being smaller in file size than the browsing history. In some implementations, applying the classification model may include applying the classification model to the simplified set of feature vectors generated based on the browsing history. In some implementations, extracting the features further includes extracting the features from at least a portion of the content of the information resource, the portion including at least one of: text data, visual data, or audio data.

[0040] At least one aspect relates to a system for encoding identifiers for selecting content. The system may include a first application executable on a client device having one or more processors. The first application may identify a plurality of information resources accessed via the first application executing on the client device. The first application may extract features corresponding to at least a portion of the content of each of the plurality of information resources. The first application may apply a classification model to the features extracted from the plurality of information resources to identify a set of categories to which the first application is classified. The first application may determine a threshold number of applications assigned to a category from the set of categories. In response to determining that the category satisfies the threshold number, the first application may assign the first application to a category identifier corresponding to the category. The category identifier for the first application may be the same as the category identifier for a second application. The first application may receive information resources from a content publisher device for presentation via the first application, the information resources including main content and content slots available for receiving content from a content selection service. The first application may generate a request for content for the content slots of the information resources, the request for content including the category identifier for the first application. The first application may transmit the request for content to the content selection service. The content selection service can use category identifiers for the first application and the second application to select content items to be inserted into the content slots of the information resource.

[0041] In some implementations, the first application may, in response to receiving the information resource, select a category from the category set according to an obfuscation strategy. The obfuscation strategy may specify a condition under which a corresponding category identifier is allowed to be included in a request for content associated with the content slot of the information resource.

[0042] In some implementations, the first application may, in response to receiving the second information resource, select a second category identifier from the set of categories identified by the classification model, according to an obfuscation strategy. The second category identifier may be different from the category assigned to the first application. In some implementations, the first application may generate a second request for content for a content slot of the second information resource, the second request including the second category identifier corresponding to the second category, instead of the category identifier corresponding to the category.

[0043] In some implementations, the first application may, in response to receiving a second information resource, determine, according to an obfuscation strategy, that it excludes any category identifier from the second request for content to be inserted into the content slot of the second information resource. In some implementations, the first application may, in response to determining that it excludes any category identifier, transmit the second request for content to the content selection service, the second request for content lacking any category identifier.

[0044] In some implementations, the first application can determine the number of times a category satisfies the threshold by using an encrypted category identifier generated using the category identifier corresponding to the category, and by cooperating with a category authorization service to execute a threshold cryptographic protocol. In some implementations, the first application can determine the number of times a category satisfies the threshold by checking the category identifier against a probabilistic data structure used for category identifiers maintained by the category authorization service.

[0045] In some implementations, the first application may transmit the request for content. The content selection service may use the category identifier to maintain aggregated browsing history for the first and second applications. In some implementations, the first application may apply the classification model to identify a set of categories that are within a proximity threshold to each other in a feature space defined by the classification model from a plurality of categories.

[0046] In some implementations, the first application may generate a simplified set of feature vectors based on the browsing history identified from the client device using a dimensionality reduction process. These feature vectors are smaller in file size than the browsing history. In some implementations, the first application may apply the classification model to the simplified set of feature vectors generated based on the browsing history. In some implementations, the first application may extract the features from at least a portion of the content of the information resource, including at least one of the following: text data, visual data, or audio data.

[0047] An optional feature of one aspect can be combined with any other aspect. Attached Figure Description

[0048] Details of one or more embodiments are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this disclosure will become apparent from the specification, drawings, and claims, wherein:

[0049] Figure 1A This is a diagram of an example document vector based on some implementation methods;

[0050] Figure 1B This is a diagram illustrating the aggregation of profile vectors into a sparse matrix according to some implementation methods.

[0051] Figure 1C This is a diagram illustrating a process for anonymization to provide pseudo-personalized clustering, based on some implementation methods;

[0052] Figure 2 This is a block diagram of a system for anonymization to provide pseudo-personalized clustering, according to some implementation methods;

[0053] Figure 3 This is a flowchart of a method for anonymization to provide pseudo-personalized clustering according to some implementations;

[0054] Figure 4 It is a block diagram depicting a system for encoding identifiers for selecting content using a classification model, according to an illustrative implementation.

[0055] Figure 5 It is a block diagram depicting a client device and application manager service in a system for encoding identifiers for selecting content using a classification model, according to an illustrative implementation.

[0056] Figure 6 It is a block diagram depicting a client device, content provider, content publisher, and content selection service in a system for encoding identifiers for selecting content using a classification model, according to an illustrative implementation.

[0057] Figure 7 It is a block diagram depicting a client device and a content selection service in a system for encoding identifiers for content selection using a classification model, according to an illustrative implementation.

[0058] Figure 8 It is a flowchart depicting a method for encoding identifiers for selecting content using a classification model, according to an illustrative embodiment; and

[0059] Figure 9 This is a block diagram illustrating a general architecture of a computer system according to an illustrative embodiment, which can be used to implement the elements of the systems and methods described and illustrated herein.

[0060] Similar reference numerals and names in the various figures indicate similar elements. Detailed Implementation

[0061] Personalized content delivery is often based on capturing identifying information about users and / or devices, such as the device's browsing or access history. For example, devices can provide identifying information such as device identifiers, account names, cookies, or other similar information; and content providers can store this information to select personalized content. As a result, content providers can acquire vast amounts of data about individuals. This significantly impacts the privacy and security of both devices and users. Op-out and non-tracking policies allow users some degree of control over their privacy (if content providers follow these policies). However, these policies diminish the ability of content providers to deliver relevant content. Furthermore, requests generated under these policies may lack any user request at all, hindering the proper parsing of these requests.

[0062] The systems and methods discussed in this paper provide quasi-personalized or anonymized content retrieval through aggregated browsing history from a vast number of devices, such as millions or billions of devices. For example, the browsing history of each device can be encoded as a long data string or an n-dimensional vector. Figure 1A This is an illustration of an example profile vector 100 according to some implementations. A profile vector can include identifiers of a device's visits to a domain, website, or webpage within a given time period. In the example shown, the vector identifies the number of views or visits to each of a large number of domains 1-n throughout the day of a week. Although shown as a one-dimensional vector, in similar implementations, the vector can include an n-by-7 array (e.g., one row per day). In other implementations, additional data can be included (e.g., each day of a month, a time period within a day, etc.). Therefore, the vector can include a large n-dimensional vector or array. The vector can include additional data such as the time of generation, location, IP address, or any other such information.

[0063] As mentioned above, in many implementations, vectors can be very large. For example, there are approximately 1.8 billion active websites and over 330 million registered domains on the Internet. In some implementations, a vector may record visits or access to any of these domains or websites. In other implementations, a vector may record visits or access to only a subset of domains or websites. For example, fewer than one million websites account for approximately 50% of web traffic. Therefore, in many implementations, a vector may only record or represent visits or tours to a subset of domains or websites. However, even in many such implementations, vectors can still be very large, such as approximately 2 26 Or larger.

[0064] Vectors can be provided to a server device, which can then aggregate those vectors with those from other devices, such as... Figure 1BAs shown in the diagram, the profile vectors 100 from each of the multiple devices 102 can be aggregated to create a very large matrix. For example, there are billions of [devices] on the web every month (e.g., in 2 [devices]). 30 (Scale) of active users or devices. The combined profile vector of each of these devices is a matrix of 100, therefore it can be 2. 30 ×2 26 Dimensions, or in some implementations, even greater.

[0065] However, this matrix is ​​highly sparse. A typical user might browse hundreds of hundreds of millions of possible domains within a given time period (such as a week). Due to its highly sparse nature and the redundancy in the browsing history of any given device, the dimensionality of such a matrix can be reduced.

[0066] In some implementations, linear dimensionality reduction techniques such as Singular Vector Decomposition (SVD) can be used to compute the rank X matrix of the closest matrix (e.g., minimizing the squared error). Each profile vector 100 can be projected onto X dimensions, where X is less than the original dimension of the matrix. For each dimension, the projection result can be quantized to 2^Ni buckets, where i∈[1,X], and the number of buckets is proportional to the singular value of the corresponding dimension. K = ∑Ni bits can be used to represent the quantized projection results for all X dimensions. Those bits concatenated together can be clustering identifiers for the devices. In other implementations, clustering algorithms (e.g., nearest neighbor) can be used to cluster the devices together in the dimensionality-reduced space.

[0067] Because singular vectors are orthogonal to each other, and as a result of the quantization process, statistically similar numbers of devices (e.g., approximately equal in the case of a large total number of devices) may lie in or be identified as belonging to each cluster, assuming no nonlinear correlation between the profile vectors. Each cluster may be represented by an identifier (sometimes called a cluster identifier, browse group identifier) ​​or by similar terms.

[0068] In other implementations, other linear decomposition methods, such as principal component analysis, can be used. In other implementations, nonlinear dimensionality reduction algorithms can be used to reduce the dimensionality of the matrix. Various classification techniques can be used, including nearest neighbor search, latent class analysis, etc.

[0069] Figure 1C This is an illustration of the process according to some implementations. As shown, in step 120, profile vectors from a large number of devices can be aggregated into a large matrix. The matrix can be reduced in dimensionality at step 122. As described above, clustering can be identified at step 124.

[0070] In some implementations, the classification system can be trained as part of and / or based on cluster identifiers. For example, in some implementations, a neural network can be used to classify a device as part of a predetermined number of clusters. Such a network can use a reduced-dimensionality profile vector as input and a cluster identifier as output. In various implementations, the network can be trained via supervised or unsupervised learning.

[0071] In some implementations, the neural network model or its weights can be provided to the client device, or other parameters for classification can be provided. Singular vectors generated from dimensionality reduction can also be provided to the client device. Therefore, after receiving the vectors and / or model, the client can generate its own classifications using the local profile vectors without further communication with the server. The server can periodically (e.g., monthly or quarterly) regenerate the singular vectors and / or classification model parameters. In some implementations, data transmission can be very large (e.g., in some implementations, in 2...). 4 The order of magnitude of singular vectors, with each singular vector approximately 2... 26 The dimensions result in a data requirement of approximately 2GB. In some implementations, to reduce data transfer to other devices, the server can compare the parameters and vectors with previously generated parameters and vectors, and only provide a new set if there are significant differences (e.g., exceeding a threshold difference), or only provide a subset of parameters and / or vectors with significant changes. In various implementations, clients can use these parameters and vectors to update their classifications locally more frequently (e.g., daily, hourly, or with every content request).

[0072] Similarly, in some implementations, singular vectors and / or model parameters may be provided to the content provider. When a client device requests a content item, the request may include cluster identifiers. These cluster identifiers may be embedded in the request payload or header, such as in an HTTPS request or in an optional field of an HTTP header. In some implementations, the content provider may use a neural network model or a provider-specific neural network model to infer demographics and / or user interests for devices within each cluster (e.g., by determining an approximate profile vector corresponding to the cluster identifier based on reduced singular vectors, and then feeding that vector to a machine learning system to infer the features), without being able to identify the characteristics of any single device or user.

[0073] Accordingly, these implementations provide device anonymity through aggregation, i.e., grouping devices with similar browsing histories or patterns together. Clustering algorithms attempt to maintain similar cluster sizes, such that each cluster contains approximately the same number of users. Given a fixed number of active devices on the internet, the entropy of the cluster identifier controls the cluster size (e.g., the higher the entropy of the cluster identifier, the fewer devices in each cluster, which reduces privacy protection). By tuning the entropy of the cluster identifier (e.g., by providing fewer or more clusters), the system can achieve the desired level of anonymization and privacy protection while still maintaining the effectiveness of content personalization.

[0074] These implementations improve upon systems that do not use aggregation and pseudo-personalization while maintaining effectiveness and increasing privacy. In such unimproved systems, user or device identifiers can be used to record device browsing history, and demographic information and interests of the corresponding users can be inferred based on the accumulated history. This inferred information can be used to predict the effectiveness of personalized content selection, such as click-through rates, engagement, or other such metrics. However, with the systems and methods discussed herein, browsing history can be accumulated only for groups of devices associated with a given cluster identifier or group. The inferred group demographics and interests can still be used in conjunction with the inferred effectiveness metrics for content selection, making it impossible for content providers to distinguish the characteristics of devices or users within the group or cluster.

[0075] In some implementations, 2^K typical browsing history patterns are defined based on the aggregated browsing history of billions of internet users. Each typical browsing history pattern has a unique cluster identifier or browsing history identifier, which can be represented by a K-bit integer for some small K, such that each cluster identifier is shared by a large number of devices. When a user chooses to opt out of personalized content selection or join pseudo-personalization, their device's browser application can select the typical browsing history pattern that best matches the user's actual browsing history. The browser can provide the corresponding cluster identifier to the content provider for content personalization purposes.

[0076] This approach, which associates approximately an equal number of devices with each cluster identifier, selects typical browsing history patterns and encodes the cluster identifiers. By controlling the value of K and the entropy of other signals that each content provider can obtain from the browser through content requests (e.g., IP address, user agent identifier, etc.), the browser can significantly reduce the risk of user re-identification while achieving pseudo-personalization of content.

[0077] Figure 2This is a block diagram of a system for anonymization to provide pseudo-personalized clustering according to some implementations. Client device 200 (which may include desktop computers, laptop computers, tablet computers, wearable computers, smartphones, embedded computers, smart cars, or any other type and form of computing device) can communicate with one or more servers (such as classifier server 230 and / or content server 250) via network 225.

[0078] In many embodiments, client device 200 may include processor 202 and memory device 206. Memory device 206 may store machine instructions that, when executed by the processor, cause the processor to perform one or more of the operations described herein. Processor 202 may include a microprocessor, ASIC, FPGA, or a combination thereof. In many embodiments, the processor may be a multi-core processor or a processor array. Memory device 206 may include, but is not limited to, electronic, optical, magnetic, or any other storage device capable of providing program instructions to the processor. Memory devices may include floppy disks, CD-ROMs, DVDs, magnetic disks, memory chips, ROMs, RAMs, EEPROMs, EPROMs, flash memory, optical media, or any other suitable memory from which the processor may read instructions. Instructions may include code from any suitable computer programming language (e.g., but not limited to C, C++, C#, Java, JavaScript, Perl, HTML, XML, Python, and Visual Basic).

[0079] Client device 200 may include one or more network interfaces 204. Network interface 204 may include any type and form of interface, including: Ethernet including 10BaseT, 100BaseT, or 1000BaseT (“Gigabit”); any of various 802.11 wireless (such as 802.11a, 802.11b, 802.11g, 802.11n, or 802.11ac); cellular networks, including CDMA, LTE, 3G, or 4G cellular networks; Bluetooth or other short-range wireless connections; or any combination of these or other interfaces for communicating with a network. In many embodiments, client device 200 may include multiple network interfaces 204 of different types to allow connection to various networks 225. Accordingly, network 225 may include a local area network (LAN), a wide area network (WAN) (such as the Internet), a cellular network, a broadband network, a Bluetooth network, an 802.11 (WiFi) network, a satellite network, or any combination of these or other networks, and may include one or more additional devices (such as routers, switches, firewalls, hubs, network accelerators, caches, etc.).

[0080] The client device may include one or more user interface devices. The user interface device can be any electronic device (e.g., keyboard, mouse, pointing device, touchscreen display, microphone, etc.) used to convey data to the user by generating sensory information (e.g., visualization on a display, one or more sounds, haptic feedback, etc.) and / or converting sensory information received from the user into electronic signals. According to various implementations, one or more user interface devices may be inside the housing of the client device (e.g., a built-in display, touchscreen, microphone, etc.) or outside the housing of the client device (e.g., a monitor connected to the client device, a speaker connected to the client device, etc.).

[0081] Memory 206 may include application 208 executed by processor 202. Application 208 may include any type and form of application (such as a media application, web browser, productivity application, or any other such application), and may generally be referred to herein as a browser application. Application 208 may receive content from a content server and may display that content to a user on a client device via a user interface.

[0082] Memory 206 may store access log 210 (displayed as log 210a on client device 200), which may be part of or maintained by application 208 (e.g., as part of a profile, preference file, history file, or other such file). Access log 210 may be stored in any format available to application 208. Access logs may include identifiers of websites, domains, web pages, content, or other data accessed, retrieved, displayed, or otherwise obtained by application 208. Access log 210 may also store profile vector 100 as discussed above, which may be generated from the access history of the application and / or device. Profile vector 100 may include an array of n-dimensional strings or values ​​representing access to one or more domains, web pages, websites, or other such data during a predetermined time period (e.g., a day, a week, a time of day, etc.). As discussed above, the profile vector may be provided to classifier server 230 (step A). Profile vector 100 may be generated by application 208 or logging reducer 214, which may include an application, service, daemon, routine, plugin, or other executable logic for generating profile vectors from access logs. In many implementations, logging reducer 214 may include a portion of application 208.

[0083] The memory 206 may also store singular vectors 212. As described above, singular vectors 212 can be obtained from the classifier server 230 (step B), which can be computed from a matrix of profile vectors from multiple client devices 200, as discussed above. Singular vectors 212 can be stored in any suitable format (such as a flat file, data array, or other structure) and can be compressed in many implementations.

[0084] Memory 206 may also store parameters of neural network model 216. As discussed above, neural network model 216 may be generated by a classifier server, and its parameters or weights may be provided to client device 200 (step C). Classifier 218 of client device 200 (which may include an application, service, server, daemon, routine, or other executable logic for performing machine learning algorithms) may utilize the parameters of neural network model 216 to generate cluster identifiers 220 from the reduced-dimensional profile vector of the client device. In some embodiments, classifier 218 may include hardware circuitry (such as a tensor processing unit) or other such hardware. In other embodiments, classifier 218 may include software executed by the device's processor 202.

[0085] The memory 206 may also store cluster identifiers 220. Cluster identifiers 220 may include cookies or other strings associated with the cluster identifier, and / or may encode or include information identifying characteristics of the cluster (e.g., XML code or parameters, parameter value pairs, etc.). Cluster identifiers 220 may be predetermined or generated by server 230 and provided to client device 200. The classifier 218 of the client device may use a neural network model to process the client device's reduced-dimensional profile vector to generate a ranking or score for each cluster identifier 220, and may select the identifier with the highest ranking or score to be transmitted to the content server during content requests (step D).

[0086] One or more classifier servers 230 may include one or more server computing devices, and may include one or more physical computing devices, or one or more virtual computing devices (e.g., cloud, cluster, or server farm) performed by one or more physical computing devices. One or more classifier servers 230 may generally be referred to as a server, measurement server, aggregation server, or by other such terms.

[0087] One or more classifier servers 230 may include one or more processors 202, a network interface 204, and a memory device 206, as well as other devices not shown. One or more classifier servers 230 may store access logs and / or profile vectors 210a-210n obtained from multiple client devices 200 in memory. As discussed above, an aggregator 232 of the classifier server 230 (which may include applications, servers, services, routines, or other executable logic executed by the processor 202) may aggregate the profile vectors 210a-210n into a matrix or an n-dimensional array. The aggregator 232 may also compute a decomposition or dimensionality reduction of the matrix into a singular vector 212, which may be provided to the client devices 200 (in some embodiments, a content server 250).

[0088] One or more classifier servers 230 may also store classifier 218 in memory. Classifier 218 may be part of aggregator 232, or it may be a separate application, service, server, routine, or other executable logic executed by processor 202 and / or hardware processors (such as tensor processing units) for identifying clusters in a dimensionality-reduced matrix. In many implementations, classifier 218 may include a neural network or similar artificial intelligence for classifying the dimensionality-reduced vector into one of several clusters. Once trained, the parameters of the neural network can be provided to one or more client devices 200 to similarly generate cluster identifiers or rankings, as discussed above.

[0089] One or more content servers 250 may include any type and form of content providing server or service, including content delivery networks, web servers, media servers, social media services, or any other type and form of computing system. One or more content servers 250 may include one or more processors 202, a network interface 204, and a storage device 206. In many embodiments, one or more content servers 250 may store multiple content items 252, which may include content of any type and form, including text, audio, video, animation, images, executable scripts, web pages, or any other such data.

[0090] One or more content servers 250 may include a content selector 254, which may be part of a web server or file server, or may be a separate application, service, server, daemon, routine, or other executable logic for selecting content to deliver to client devices. Content may be selected based on inferred characteristics of the device. The content server may receive cluster identifiers and requests for content from the device, and may select content based on inferred characteristics of the device for that cluster. In some implementations, the content server may use singular vectors obtained from classifier server 230 to expand the cluster identifiers into corresponding profile vectors representing aggregated clusters. The profile vector of the cluster may not match the profile vector of any particular device, but may be an approximation or average of the vectors of all devices associated with the cluster.

[0091] Figure 3 This is a flowchart of a method for anonymization to provide pseudo-personalized clustering according to some implementations. At step 302, the client device 200 may provide access logs or profile vectors generated from the access logs to the classifier server 230. As discussed above, the profile vector may be based on the client device's browsing or access history and may include an n-dimensional vector or string whose values ​​represent each of one or more accesses to an address associated with a corresponding location in the string or array within a predetermined time period.

[0092] At step 304, classifier server 230 can aggregate the profile vectors or logs from client device 200 with those obtained from one or more additional client devices 200 into a matrix. The profile vectors can be obtained periodically by classifier server 230 or upon login to a service provided by classifier server or an associated server. Steps 302-304 can be repeated for multiple client devices, which may be a small subset of all devices from which the singular vectors in 312 can be used to perform the dimensionality reduction in 314. In some implementations, steps 302-312 can be replaced by joint learning on the client devices, and classifier server 230 may be optional or removed.

[0093] At step 306, the classifier server 230 can compute the dimensionality reduction or decomposition of the matrix. In some implementations, the classifier server can use a singular value decomposition algorithm and can generate multiple singular vectors and dimensionality-reduced matrices.

[0094] At step 308, the classifier server may optionally identify the cluster boundaries or clustering parameters of the dimensionality reduction matrix. The classifier server may use any suitable algorithm (e.g., principal component analysis) or machine learning system (e.g., neural network) to identify the clusters.

[0095] At step 310, in some embodiments, the classifier model can be tuned or trained based on the identified clustering and dimensionality-reduced profile vectors. In various embodiments, the training can be supervised or unsupervised.

[0096] At step 312, the singular vectors and the weights of the neural network model or other learning algorithm in some embodiments may be provided to one or more client devices 200, and in some embodiments, to one or more content servers 250.

[0097] At step 314, the client device 200 can compute a dimensionality reduction of the device's profile vector or access logs using the singular vectors received from the classifier server. At step 316, the dimensionality-reduced vectors can be classified via artificial intelligence or a neural network using model parameters received from the classifier server, and at step 318, the classification is determined. Determining the classification may include calculating a score or ranking for each of multiple clusters (e.g., identified by the classifier server and provided via the parameters) for the device's profile vector.

[0098] At step 320, the client device may send a request for a content item to one or more content servers 250. The request may include an identifier of a cluster corresponding to the device's profile vector. This request may be generated in response to the execution of a script on the webpage after playback of the media item or a portion thereof is completed, or in any other such case.

[0099] At step 322, the content server can determine cluster features based on classifier model parameters and singular vectors received from the classifier server. In some implementations, the content server can use the singular vectors to compute profile vectors representing the aggregated browsing history of all devices in the cluster, and then infer features of the cluster based on various aspects of the history (e.g., keywords / topics associated with web pages or domains or other content, other associated domains or web pages, etc.). At step 324, the content server can select content items using the cluster identifier in the request (or the inferred features associated with the cluster as discussed above). Content can be delivered to the client device at step 326, and at step 328, the client device can render or display the content items.

[0100] Therefore, the systems and methods discussed in this paper provide quasi-personalized or anonymized content retrieval via aggregated browsing histories of millions or billions of devices. Sparse matrices can be constructed from aggregated browsing histories and reduced in dimensionality, thereby reducing entropy and providing anonymity for individual devices. Relevant content can be selected via quasi-personalized clustering to represent similar browsing histories without exposing individual device details to content providers.

[0101] In a networked environment, an application (e.g., a web browser) running on a client device can receive an information resource (e.g., a webpage) having main content provided by a content publisher and one or more content slots (e.g., embedded frames) for supplementary content. The information resource may include code snippets or scripts (e.g., content selection tags) that specify retrieving content items from a content selection service to insert into the content slots. While parsing the script of the information resource, the application can generate a request for the content to be inserted into the content slots and can send the request to the content selection service. In response to receiving the request, the content selection service can select one of the content items supplied by the content provider to embed into the content slot of the information resource.

[0102] The selection of content items through a content selection service can rely on the use of a unique, deterministic tracking identifier for the user (or via an account), the client device operated by the user, or the application running on the client device. This identifier can be maintained on the client device and accessed by the content selection service via cookies. Cookies can be, for example, third-party cookies associated with a domain of the content selection service that is different from the domain of the content publisher of the information resource. When a content slot of an information resource specifies that content should be retrieved from the content selection service, a cookie containing the identifier can be passed from the client device to the content selection service. By using cookies, the content selection service can track which information resources the user accesses via an application running on the client device. Additionally, the content selection service can identify content items determined to be associated with a user operating the application on the client device based on the tracked information resources.

[0103] While using these unique tracking identifiers allows for the selection of content items tailored to specific users, several drawbacks exist, particularly regarding data security and integrity. On one hand, users may face data security risks due to the transfer of user data between client devices and the content selection service. For example, the content selection service manager could intentionally provide private user data to third parties without the user's consent. Furthermore, unauthorized malicious entities could intercept third-party cookies in transit and use the unique tracking identifiers to monitor the online activity of affected client devices and applications. On the other hand, accessing collections of information resources using this category of identifiers could increase the risk of data breaches on a portion of the content selection service. For instance, the accidental release of data or malicious attacks that steal collected data (some of which could be traced back to specific users) could lead to a loss of user data privacy.

[0104] One approach to addressing the issue of unique tracking identifiers may include disabling third-party cookies with unique tracking identifiers from the client device. Applications running on the client device can be configured to prevent the generation, maintenance, or transmission of unique tracking identifiers to a content selection service. However, restrictions on third-party cookies can introduce several other problems. By disabling third-party cookies, the content selection service may be unable to track information resources accessed by the client device via the application. Thus, when a request for content is received, the content selection service may be unable to use such information to determine the relevance of content items to the user operating the application. Consequently, the selected content items may have a lower likelihood of interaction with the user of the client device compared to content items selected using tracking identifiers. As a result, information resources on which content items are inserted for display may suffer from a degradation in the quality of human-computer interaction (HCI).

[0105] To address the technical challenge of prohibiting the use of unique identifiers to track individual client devices or applications when selecting content, each application can be categorized into one of several clusters based on locally maintained browsing history. Users of the application, and relatedly, those with similar browsing patterns, can be grouped into the same cluster. Users with similar browsing patterns and categorized into the same cluster can be associated with similar characteristics and interests, and are therefore more likely to have similar responses to the same content items. Each cluster can have a large number of associated users (e.g., more than 1000), so categorizing users into clusters may not be a feature specific to individual users.

[0106] When assigning an application to one of its clusters, the application can transform or encode its browsing history into a vector with a preset dimension. For example, one element in the feature vector could indicate whether a user visited a specific domain, a part of a website, a webpage of a specific category (e.g., a holiday), or even a specific URL during a specific time slot (e.g., a specific hour of day and a specific day of week). The application can then apply a clustering or classification algorithm (e.g., k-nearest neighbor, linear classification, support vector machines, and pattern identification) to the feature vector to identify the cluster to which the application is assigned and, consequently, the user. The clustering algorithm can be provided and updated by the application's application manager (e.g., a browser vendor).

[0107] After clusters are found using a clustering algorithm, the application can identify cluster identifiers (also known as category identifiers or browsing history identifiers) for those clusters. Cluster identifiers can be assigned to each cluster by the application manager and can be provided to the application and content selection service. Unlike unique deterministic tracking identifiers, cluster identifiers may not be specific to a single user, application, or client device. Because multiple users can be grouped into the same cluster, cluster identifiers can also be shared among multiple users, applications, or client devices with similar browsing patterns. Because cluster identifiers are shared among multiple users, they can have lower entropy compared to unique tracking identifiers assigned to individual users. For example, a deterministic unique tracking identifier for all internet users can have more than 30 bits of entropy, while a cluster identifier can be set to 18 to 22 bits of entropy. With lower entropy, cluster identifiers themselves can have a smaller length and size than unique tracking identifiers.

[0108] When receiving an information resource with a content slot, the application can include a clustering identifier, rather than a unique tracking identifier, in the request for content in the content slot and send the request to a content selection service. Additional protection measures can be taken to increase data security and integrity and reduce the likelihood of leakage of the clustering identifier and related information. To protect against interception of the clustering identifier during transmission over the network, the application can use an encryption protocol, such as Hypertext Transfer Protocol Security (HTTPS). Additionally, to protect the clustering identifier maintained on the client device, the application can restrict access to the clustering identifier by other client processes (e.g., JavaScript on the information resource). For example, the clustering identifier can be included in a security flag or an HTTP flag-only cookie maintained on the client device to prevent access to the clustering identifier. This contrasts with third-party cookies that do not have such access controls.

[0109] In response to a received request, the content selection service can select one of the content items using a cluster identifier. The content selection service can accumulate the browsing history of users categorized into a cluster by using previous requests for content containing that cluster identifier. By applying a profile model to the accumulated history of the clusters, the content selection service can infer the characteristics and interests of users within those clusters. Using the results of the profile model, the content selection service can find content items identified as related to the cluster to which the user associated with the request was categorized.

[0110] By using clustering identifiers, the browsing history of applications maintained on client devices can be prevented from being accessed by the content selection service to make that browsing history traceable to a specific user, application, or client device. Furthermore, the content selection service may not be able to track individual users, applications, or client devices across different domains to aggregate detailed browsing history. Instead, the content selection service can aggregate the browsing history of specific clusters of users associated with clustering identifiers received from the application. During aggregation, the content selection service can protect individual user data privacy by merging the browser histories of different users from the same cluster. The degree of data privacy can also be controlled by setting the number of users to be assigned to each cluster.

[0111] Furthermore, assuming that users within the same cluster share similar browsing patterns, the content selection service can extrapolate and determine the similar characteristics and interests of users within the same cluster based on browsing history aggregated for that cluster. The service can further select content items based on the expectation that users within the same cluster will respond similarly. In this way, the security, integrity, and privacy of data from each user's browsing history can be maintained. Simultaneously, the ability to determine the relevance in the selection of content items can maintain the quality of human-computer interaction (HCI) with the selected content items or the overall information resource.

[0112] Now for reference Figure 4This diagram depicts a block diagram illustrating one implementation of a computer networking environment or system 400 for encoding identifiers for content selection using a classification model. In general, system 400 may include at least one network 405 for communication between components of system 400. System 400 may include at least one application manager service 410 (also referred to herein as a browser vendor) to provide services to at least one application (e.g., a browser). System 400 may include at least one content provider 415 to provide content items. System 400 may include at least one content publisher 420 to provide information resources (e.g., web pages). System 400 may include at least one content selection service 425 to select content items. System 400 may include one or more client devices 430A-N (generally referred to herein as client devices 430). Each client device 430 may include at least one application 435A-N (generally referred to herein as application 435). Each component of system 400 (e.g., network 405, application manager service 410 and its components, content provider 415 and its components, content publisher 420 and its components, content selection service 425 and its components, and client device 430 and its components) can be combined with this document Figure 9 The components of the computing system 900 are described in detail for implementation.

[0113] More specifically, network 405 of system 400 can communicatively couple application manager service 410, content provider 415, content publisher 420, content selection service 425, and client device 430 to each other. Each of application manager service 410, content provider 415, content publisher 420, and content selection service 425 of system 400 may include multiple servers located in at least one data center or server farm, which multiple servers are communicatively coupled to each other via network 405. Application manager service 410 can communicate with content provider 415, content publisher 420, content selection service 425, and client device 430 via network 405. Content provider 415 can communicate with application manager service 410, content publisher 420, content selection service 425, and client device 430 via network 405. Content publisher 420 can communicate with application manager service 410, content publisher 420, content selection service 425, and client device 430 via network 405. The content selection service 425 can communicate with the application manager service 410, content provider 415, content publisher 420, and client device 430 via network 405. Each client device 430 can communicate with the application manager service 410, content provider 415, content publisher 420, and content selection service 425 via network 405.

[0114] Application Manager Service 410 may include a server or other computing device operated by an application vendor (sometimes referred to herein as a browser vendor) to provide resources and updates to application 435 running on client device 430. For example, Application Manager Service 410 may provide application 435 to be installed on client device 430. Application Manager Service 410 may also provide updates to application 435 installed on client device 430. These updates may affect at least one sub-component of application 435. Application Manager Service 410 may also provide plugins or add-ons to application 435 to enhance the functionality of application 435 running on client device 430. Application Manager Service 410 may communicate with Content Selection Service 425 to provide information about application 435 running on client device 430. The provision of application 435 and related files or data may be communicated by Application Manager Service 410 via network 405.

[0115] Content provider 415 may include a server or other computing device operated by the content provider entity to provide content items for display on information resources at client device 430. The content provided by content provider 415 may take any convenient form. For example, third-party content may include content related to other displayed content and may be, for example, a page from a website related to the displayed content. The content may include third-party content items or creative ideas (e.g., advertisements) for display on information resources, such as information resources that include main content provided by content publisher 420. These content items may also be displayed on search results web pages. For example, content provider 415 may provide content item 455 or may be the source of content item 455 to be displayed in a content slot (e.g., an inline frame element) of information resource 450 (e.g., a company's webpage, the main content of which is provided by the company), or on a search results landing page provided by a search engine. Content items associated with content provider 415 may be displayed on information resources other than webpages, such as content displayed as part of an application running on a smartphone or other client device 430.

[0116] Content publisher 420 may include servers or other computing devices operated by the content publishing entity to provide information resources, including main content for display, via network 405. For example, content publisher 420 may include a web operator providing main content for display on the information resource. The information resource may include content other than that provided by content publisher 420, and may include content slots configured to display content items from content provider 415. For example, content publisher 420 may operate a company's website and may provide content about the company for display on web pages of the website. The web pages may include content slots configured to display content items provided by content provider 415 or by content publisher 420 itself. In some embodiments, content publisher 420 includes search engine computing devices (e.g., servers) of a search engine operator operating a search engine website. The main content of a search engine webpage (e.g., results or landing page) may include search results and third-party content items (such as content items from content provider 415) displayed in content slots of the information resource. In some embodiments, content publisher 420 may include one or more servers for providing video content.

[0117] Content selection service 425 may include a server or other computing device operated by a content placement entity to select or identify content items to be inserted into content slots of an information resource via network 405. In some embodiments, content selection service 425 may include a content placement system (e.g., an online advertising server). Content selection service 425 may maintain a list of content items to select from for provision via network 405 to be inserted into content slots of an information resource. The list may be maintained on a database accessible to content selection service 425. The content items or identifiers of the content items (e.g., addresses) may be provided by content provider 415.

[0118] Each client device 430 may be a computing device that communicates via network 405 to display data. The displayed data may include content provided by content publisher 420 (e.g., information resources) and content provided by content provider 415 (e.g., content items for display in content slots of information resources), as identified by content selection service 425. Client device 430 may include a desktop computer, laptop computer, tablet computer, smartphone, personal digital assistant, mobile device, consumer computing device, server, client, digital video recorder, television set-top box, video game console, or any other computing device configured to communicate via network 405. Client device 430 may be a communication device through which an end user can submit a request to receive content. This request may be a request to a search engine, and may include a search query. In some embodiments, the request may include a request to access a webpage.

[0119] Application 435 executing on client device 430 may include, for example, an internet browser, a mobile application, or any other computer program capable of executing or otherwise invoking computer-executable instructions (such as computer-executable instructions contained in information resources and content items) provided to client device 430. The information resources included may correspond to scripts, logic, markup, or instructions (e.g., Hypertext Markup Language (HTML), Extensible Hypertext Markup Language (XHTML), Extensible Markup Language (XML), Cascading Style Sheets (CSS), and JavaScript, or any combination thereof). Content items may be inserted into content slots of information resources.

[0120] Now for reference Figure 5 This diagram depicts a block diagram of one implementation of a client device 430 and an application manager service 410 in a system 400. In general, the application manager service 410 may include at least one classification model 500 to classify applications 435 based on browsing patterns. The application manager service 410 may include at least one model trainer 505 to train and maintain at least one classification model 500. The application manager service 410 may include at least one model updater 510 to modify or adjust the classification model 500. The application manager service 410 may include at least one database 515 to store and maintain a set of category identifiers 520A-N (generally referred to herein as category identifier 520, and sometimes referred to herein as "network area identifier" or "network postal code"). The application manager server 410 may include at least one instance of a category certifier 550 (sometimes referred to herein as a category authentication service) to allow the inclusion of category identifier 520 in requests transmitted via network 405.

[0121] On each client device 430, application 435 may include at least one classification model 500 to categorize application 435 based on browsing patterns. Application 435 may include at least one model trainer 505 to train and maintain the classification model 500. Application 435 may include at least one content extractor 525 to select features from information resources accessed via application 435. Application 435 may include at least one model applicator 530 to categorize application 435 using the classification model 500. Application 435 may also include at least one instance of a category authorizer 550. The model trainer 505, content extractor 525, model applicator 530, and category authorizer 550 may be integral parts of application 435, add-ons or plugins to application 435, or additional applications that interface with application 435. Application 435 may include at least one browsing history 535 to maintain and store one or more entries 540A-N (generally referred to herein as entries 540). Application 435 may include at least one identifier cache 545 to maintain and store at least one category identifier 520 for application 435.

[0122] In further detail, the classification model 500 can classify, group, or otherwise categorize each application 435 (or each client device 430 running the application 435 or the account associated with the application 435) based on browsing history 535. The classification of applications 435 on client devices 430 can indicate or represent the classification of instances of applications 435 running on client devices 430. For example, there may be an instance of an application 435 (e.g., a web browser) running on one client device 430 and another instance of an application 435 (e.g., a web browser of the same type) running on another client device 430. These two instances can be classified into the same category or different categories. Furthermore, the classification of applications 435 can extend to classifications corresponding to or including users associated with the account operating the application 435 or the client device 430 operated by the user.

[0123] The classification model 500 can be a classification or clustering model or algorithm. The classification model 500 can include classification algorithms such as linear classifiers (e.g., linear regression, log regression, and Naive Bayes classifiers), support vector machines (SVM), quadratic classifiers, k-nearest neighbor algorithms, and artificial neural networks (ANNs), etc. The classification model 500 can include clustering models such as centroid-based clustering algorithms (e.g., k-means or expectation-maximization (EM) algorithms), density-based clustering algorithms (e.g., density-based spatial clustering applications with noise), distribution-based clustering algorithms (e.g., Gaussian mixture models), and artificial neural networks (ANNs), etc. The classification model 500 can also include hash functions such as position-sensitive hashing schemes (e.g., MinHash, SimHash, and Nilsimsa Hash).

[0124] According to any classification or clustering model and algorithm, classification model 500 may include an input set, a parameter set, and an output set. The input set may take or include entries 540 of the browsing history 535. In some embodiments, the input set may take or include a dimensionality-reduced representation of entries 540 of the browsing history 535. In some embodiments, the input set may include a complete representation of entries 540 of the browsing history 535. The parameter (or weight) set may concatenate or correlate the input set with the output set. In some embodiments, the parameter set may include multiple categories and values ​​for each category. For example, for each category, the parameter set may be the centroids in a k-means cluster. In another example, the parameter set may include a range of values ​​associated with each category. The number of categories may be equal to the number of category identifiers 520. The number of categories and the number of category identifiers 520 may be predetermined (e.g., fixed values) or dynamically determined. The output set may produce or include categories corresponding to one of the category identifiers 520. The output set may include one of the category identifiers 520 themselves. For example, the category identifier 520 may be a hash value calculated using a hash function. Each category identifier 520 can be or include a set of alphanumeric characters or numeric values ​​(e.g., integers or binary numbers).

[0125] A model trainer 505, running on the application manager service 410, can train a classification model 500. The model trainer 505 can use a training dataset to train the classification model 500. Training of the classification model 500 can be unsupervised. The training dataset can include sample browsing history 530 from a sample set of an application 435 running on client devices 430. In some implementations, the model trainer 505 can acquire and accumulate sample browsing history 530 from a content provider 415, a content publisher 420, a content selection service 425, or an application 435 running on client devices 430. Before training, the model trainer 505 can identify the number of categories to be used for the classification model 500. In some implementations, the number of categories can be predetermined or assigned by the administrator of the application manager service 410. In some implementations, the number of categories can be determined by the model trainer 505 based on the number of client devices 430 with application 435. For example, the number of categories can be set such that the set number of client devices 430 (e.g., 800 to 1000 devices) are assigned to each category.

[0126] When training classification model 500, model trainer 505 can change, adjust, or otherwise set the parameter values ​​(e.g., values ​​for each class) in classification model 500 by using a training dataset. In each iteration, model trainer 505 can determine whether classification model 500 has converged relative to a previous iteration based on changes in the set of parameter values. In some implementations, model trainer 505 can compare changes in a set of values ​​of the parameters of classification model 500 with a convergence threshold. If the change is less than the convergence threshold, model trainer 505 can determine that classification model 500 has converged. Otherwise, if the change is greater than the convergence threshold, model trainer 505 can determine that classification model 500 has not yet converged. When it is determined that classification model 500 has not converged, model trainer 505 can continue training classification model 500.

[0127] Otherwise, when it is determined that the classification model 500 has converged, the model trainer 505 may end, terminate, or stop training the classification model 500. The model trainer 505 may store the classification model 500 on the application manager service 410. Additionally, the model trainer 505 may transmit or send the classification model 500 to the application 435 running on the client device 430. In some embodiments, the model trainer 505 may transmit or send a set of parameters for the classification model 500. For each category in the classification model 500, the model trainer 505 may identify, assign, or otherwise associate the category with a corresponding category identifier 520. The category identifier 520 may be a set of alphanumeric characters to refer to the category. The classification model 500 may associate each category with a corresponding category identifier 520. The model trainer 505 may transmit and provide the set of category identifiers 520 to the application 435 running on each client device 430 and to the content selection service 425.

[0128] A model trainer 505 of an application 435 running on client device 430 can also train classification model 500. In some implementations, model trainer 505 may use a distributed learning protocol to train classification model 500. The distributed learning protocol may collaborate with application 435 running on other client devices 430 and application manager service 410 communicating via network 405. The distributed learning protocol may include, for example, joint learning using optimization algorithms (e.g., stochastic gradient descent (SGD) or averaging) to train classification model 500. As discussed above, the number of classes and the number of class identifiers 520 may be predetermined (e.g., fixed values) or dynamically determined. During each iteration, each model trainer 505 may change, adjust, or otherwise set the values ​​of parameters in classification model 500 (e.g., values ​​for each class) by using the training dataset. At the end of the iteration, each model trainer 505 (an instance of model trainer 505 on other client devices 430) may provide each other with the values ​​of parameters in classification model 500 (e.g., values ​​for each class). The model trainer 505 can use the received parameter values ​​to adjust, change, or set the parameters on the locally maintained classification model 500. The model trainer 505 can iterate repeatedly until it is determined that the classification model 500 has converged relative to the previous iteration based on changes in the set of parameter values ​​discussed above.

[0129] The model updater 510, executing on the application manager service 410, can provide or send updates to the classification model 500 maintained on each client device 430 running on the application 435. The model updater 510 can determine whether to update the classification model 500 based on a model update strategy. The model update strategy can specify a set of conditions under which the classification model 500 will be updated. In some implementations, the model update strategy can include a schedule that indicates when to update the classification model 500. The model update strategy can specify that the classification model 500 will be updated when the number of applications 435 not evenly distributed to each category (e.g., the difference in category size is within 5% to 100% of another category). The model update strategy can specify that the classification model 500 will be updated when the amount of accumulated additional browsing history 535 meets a threshold amount. If it is determined that the classification model 500 should not be updated, the model updater 510 can maintain the classification model 500.

[0130] On the other hand, if an update is determined to be necessary, the model updater 510 can invoke the model trainer 505 (on the application manager service 410 or on the application 435 running on the client device 430) to retrain the classification model 500. In some implementations, the model updater 510 can accumulate browsing history 530 of the training dataset from the content provider 415, content publisher 420, content selection service 425, or the application 435 running on the client device 430. The updater 510 can pass the accumulated browsing history 530 to the model trainer 505 to retrain the classification model 500. After determining that the classification model 500 has converged, the model trainer 505 can end, terminate, or stop training the classification model 500. The model updater 510 can transfer or send the newly trained classification model 500 (or the set of parameters used for the classification model 500) to each application 435 to update the classification model 500. The model updater 510 can transmit and provide a set of category identifiers 520 to the application 435 running on each client device 430 and the content selection service 425.

[0131] Model application 530, running on client device 435, can receive classification model 500 from application manager service 410 via network 405. Upon receipt, model application 530 can store and maintain classification model 500 on client device 430. In some embodiments, model applicator 530 can receive a set of parameters for classification model 500 from application manager service 410. Receiving the parameter set can be used to update classification model 500. Upon receipt, model applicator 530 can change, configure, or otherwise modify classification model 500 by using the received parameter set. Furthermore, model applicator 530 can receive a set of category identifiers 520 for each category in classification model 500 from application manager service 410. Receiving category identifiers 520 can be part of updating classification model 500.

[0132] By configuring the classification model 500, the model applicator 530 can identify the browsing history 535 maintained on the client device 435 via the application 435. The application 435 can maintain the browsing history 535 by creating entries 540 each time an information resource is accessed. The browsing history 535 can record information resources (e.g., web pages) and other online content accessed via the application 435. The browsing history 535 can include a set of entries 540. Each entry 540 can include an address for the accessed information resource (e.g., a Uniform Resource Locator (URL) including the hostname and pathname of the web page) and a timestamp indicating the time of access to the information resource. The set of entries 540 can be indexed by the timestamps or addresses of information resources within the browsing history 535. In some embodiments, the model applicator 530 can identify a portion of the browsing history 535 over an entire time span that is further processed by the model applicator 530. A portion of the browsing history 535 can include a subset of entries 540 with timestamps within the time span. This time span can be, for example, the number of entries 540 over one week from the current time.

[0133] In conjunction, the content extractor 525 of application 435, executing on client device 430, can identify, select, or otherwise extract one or more features from each information resource for a corresponding entry 540 in browsing history 535. These features can be extracted from at least a portion of the content on the information resource. These features may include, for example, text data, visual data, or audio data. In some embodiments, the content extractor 525 can identify the features from the content of the information resource when application 435 accesses the information resource via network 405. In some embodiments, the content extractor 525 can access each information resource via a corresponding entry in browsing history 535 to extract the features. This access may be separate from or after the presentation of the information resource via application 435.

[0134] When extracting from information resources, content extractor 525 may apply one or more attribute selection algorithms to the content of each information resource accessed by application 535. To extract text data, content extractor 525 may apply at least one natural selection processing algorithm, such as term extraction, named entity identification, relation extraction, automatic summarization, and term frequency-inverse document frequency (TF-IDF), etc. Text data identified using natural language processing algorithms may include a subset of the text content on the information resource. To extract visual data, content extractor 525 may apply at least one computer vision algorithm, such as object identification and optical character recognition (OCR). Visual data identified using computer vision algorithms may include a subset of graphics on the information resource (or identifiers corresponding to that subset). To extract audio data, content extractor 525 may apply at least one audio signal analysis algorithm and speech identification algorithm. The extracted audio data may include, for example, words identified from audio.

[0135] After identifying the browsing history 535, the model applicator 535 can form or generate a set of feature vectors using the entries 540 of the browsing history 535. In some embodiments, the model applicator 535 can generate the set of feature vectors using features extracted from information resources accessed by the application 435. The set of feature vectors can be used as input to the classification model 500. The set of feature vectors can include dimensions, or can be defined by dimensions. Dimensions can include time ranges (e.g., hours of a day or days of a week) and address attributes (e.g., domain name, part of a website, topic category, or the address itself). Dimensions can also include text data, image data, and audio data corresponding to the extracted features. Each feature vector can be associated with at least one time range based on a timestamp associated with the corresponding entry 540. Each feature vector can be associated with at least one of the address attributes of the information resource based on the address of the information resource identified in the corresponding entry 540.

[0136] In some implementations, the model applicator 535 can generate a set of feature vectors by projecting the entries 540 of the browsing history 535 onto the dimensions of feature vectors defined by time range and address attributes. For each entry 540 of the browsing history 535, the model applicator 535 can identify whether an existing feature vector exists based on the address and timestamp in the entry 540. To identify this, the model applicator 535 can determine whether the entry 540 matches any of the existing feature vectors. When an existing feature vector exists, the model applicator 535 can add the entry 540 to the feature vector. Conversely, when no existing feature vector exists, the model applicator 535 can create a new feature vector for the entry 540.

[0137] In some implementations, the model applicator 535 may use a dimensionality reduction process to form or generate a simplified set of feature vectors. The dimensionality reduction process may include linear dimensionality reduction techniques (e.g., Principal Component Analysis (PCA), Singular Value Decomposition (SVD), Nonnegative Matrix Factorization (NMF)), nonlinear dimensionality reduction (e.g., Generalized Discriminant Analysis (GDA), Locally Linear Embedding, and Sammon Mapping)) or others (e.g., Johnson-Lindenstrauss Lemma and Multifactor Dimensionality Reduction), etc.). In some implementations, the model applicator 530 may apply the dimensionality reduction process when projecting the entries 540 of the browsing history 535 onto the dimensions of the feature vectors. In some implementations, the model applicator 535 may apply the dimensionality reduction process to the generated set of feature vectors. By applying the dimensionality reduction process, the model applicator 535 can reduce the dimensionality in the original set of feature vectors to generate a simplified set of feature vectors. The simplified set of feature vectors may have fewer dimensions and data points than the initial set of simplified feature vectors. For example, the simplified set of feature vectors may omit time ranges or address attributes without any associated entries 540. In some implementations, the model applicator 535 may omit the dimensionality reduction process and continue processing the feature vectors without dimensionality reduction.

[0138] Model applicator 530 can apply classification model 500 to browsing history 535 (or any subset or representation of entries 540 of browsing history 535, such as feature vectors) to identify one or more categories in which application 435 is categorized. For application, model applicator 530 can feed browsing history 535 as the input set to classification model 500. In some embodiments, model applicator 530 can feed feature vectors or a simplified set of feature vectors as input to classification model 500. Once fed, model applicator 530 can use classification model 500 to compare the input with parameters used to define the categories and generate or produce an output set based on said comparison. The output set may include one or more categories, as indicated in the browsing history 535 maintained by application 435, classifying browsing patterns into those categories.

[0139] Based on the output of classification model 500, model applicator 530 can identify one or more categories. In some implementations, model applicator 530 can identify a single category (sometimes referred to herein as the closest or nearest class) from the output. A single category may correspond to a portion of the feature space defined by classification model 500 that is closest to the input feature vector. The feature space defined by classification model 500 may have the same parameters and values ​​as the input.

[0140] In some implementations, model applicator 530 can identify a set of categories (sometimes referred to herein as the closest or nearest categories) from the output. The identified set can be a subset of categories defined by classification model 500. In the feature space defined by classification model 300, the member categories of the set can be within a proximity threshold of each other. The proximity threshold can be defined as the distance in a specific space in which one or more categories are to be selected. In some implementations, model applicator 530 can identify a set of categories within a proximity threshold of the feature vectors input to classification model 500. In some implementations, model applicator 530 can rank the set of categories by their distance from the input feature vectors in the feature space.

[0141] Each identified category may correspond to one of multiple categories as defined by classification model 500. The categories identified by application 435 running on client device 430 may be common, shared, or identical to the categories identified by at least some other applications 435 running on other client devices 430. As discussed above, each category defined by classification model 500 may have multiple client devices to be categorized into that category.

[0142] By identifying one or more categories for application 435, category authorizer 550 can determine whether each category meets (e.g., is greater than or equal to) a threshold number of applications 435 assigned to that category. This threshold number can delineate or define the number of applications 435 assigned to that category, in which the category can be used to generate requests for content. The threshold number can be set to match or reach a target entropy. Multiple applications 435 may exist, each assigned to the same category by a single instance of model applicator 530. However, a category can only be used to generate requests for content until the number of such applications 435 exceeds the threshold number. The determination of applications 435 on client device 435 can be coordinated or harmonized with a category authentication service (e.g., category authorizer 550 executing on application manager service 410). The functionality of category authorizer 550 can be divided between applications 435 and application manager service 410 (or some other server device). This determination can be based on a threshold cryptographic protocol or a check against a probabilistic data structure.

[0143] In some implementations, the category certifier 550 of application 435 and the category certifier 550 of application manager service 410 can perform a threshold cryptography protocol during determination. For each identified category, the category certifier 550 on application 435 can identify a category identifier 520 corresponding to that category. Upon identification, the category certifier 550 can generate an encrypted copy of the category identifier 520. In some implementations, the category certifier 550 can generate the encrypted copy using at least a portion of a private encryption key based on an asymmetric cryptographic algorithm. Asymmetric cryptographic algorithms can include, for example, digital signatures, Diffie-Hellman key exchange, elliptic curve cryptography, or the Rivest-Shamir-Adleman (RSA) algorithm, etc. The category certifier 550 of application 435 can generate an authentication request that includes an encrypted copy of the category identifier 520. Once generated, the request can be transmitted to the category certifier 550 on application manager service 410 via network 405.

[0144] Upon receiving the authentication request from client device 430, the class authorizer 550, executing on application manager service 410, can parse the authentication request to identify an encrypted copy of class identifier 520. Using this identification, class authorizer 550 can attempt to decrypt the original class identifier 520 from the encrypted copy. The decryption can be performed using an asymmetric cryptographic algorithm. Under a threshold cryptography protocol, class authorizer 550 may fail to decrypt the encrypted copy until the number of received authentication requests with the same encrypted copy of class identifier 520 meets a threshold number. For example, failure to decrypt might be because class authorizer 550 may not have received a sufficient portion of the private key from requests from different applications 435. Once a sufficient portion of the private key is received, class authorizer 550 is able to successfully decrypt the encrypted copy to recover the original class identifier 520.

[0145] The category authorizer 550 can generate output from the decryption of the encrypted copy. If the number of requests for encrypted copies of the same category identifier 520 does not meet (e.g., less than) the threshold, the category authorizer 550 may not be able to recover the original category identifier 550 from the decryption. Therefore, the output generated from the decryption attempt may not match the original category identifier 520. On the other hand, if the number of requests for encrypted copies of the same category identifier 520 meets (e.g., greater than or equal to) the threshold, the category authorizer 550 can recover the original category identifier 550 from the decryption. Therefore, the output generated from the decryption attempt can match the original category identifier 520. Using the obtained output, the category authorizer 550 can generate a response that includes that output. Once generated, the category authorizer 550 on the application manager service 510 can transmit the response with the obtained output to the client device 430 via network 405.

[0146] The category authorizer 550 of application 435 can then receive the response from the category authorizer 550 running on application manager service 410. Upon receipt, the category authorizer 550 can parse the response to identify the output obtained from the decryption attempt. The category authorizer 550 can compare the obtained output with the category identifier 520 corresponding to the category included in the authentication request. When it is determined that the obtained output matches, the category authorizer 550 can determine that the category meets the threshold number of application 435. The model applicator 530 can continue to use the category and the corresponding category identifier 520. Conversely, when the obtained output is determined to be a mismatch, the category authorizer 550 can determine that the category does not meet the threshold number of application 435. The model applicator 530 can discard the category and the corresponding category identifier 520 for further use. Alternatively, the category authorizer 550 can utilize another category repetition threshold cryptography protocol to find the category to be used in the request for content.

[0147] In some implementations, for each identified category, the category authorizer 550 of application 435 can check the category identifier 520 corresponding to that category against at least one data structure to determine its suitability. The data structure may be maintained by the category authorizer 550 running on application manager service 410 (e.g., database 515). The data structure may indicate whether any application 435 should be assigned to that category. The data structure may also indicate whether the number of applications 435 assigned to that category meets a threshold number. In some implementations, the data structure may be a probabilistic data structure. Probabilistic data structures may include, for example, count Bloom filters, quotient filters, Cuckoo filters, count-minimum sketches, etc.

[0148] To perform the check, the category authorizer 550 of application 435 can generate an authentication request that includes a category identifier 520 corresponding to the category. Upon generation, the category authorizer 550 can send the authentication request to the category authorizer 550 running on the application manager service 410. The category authorizer 550 on the application manager service 410 can then receive the authentication request. The category authorizer 550 can parse the authentication request to identify the category identifier 520. The category authorizer 550 can use the application data structure to determine whether the number of applications 435 assigned to the category meets a threshold. When applying the data structure, the category authorizer 550 can feed the category identifier 520 included in the request into the data structure and identify the output from the data structure. Additionally, each time an authentication request is received, the category authorizer 550 can update the category data structure maintained by the application manager service 410.

[0149] If the data structure indicates that the quantity meets (e.g., is greater than or equal to) a threshold quantity, the category authorizer 550 on application manager service 410 can generate a success response. This response can indicate that the quantity of application 435 meets the threshold quantity. The category authorizer 550 on application manager service 410 can send the success response to the category authorizer 550 on application 435. Based on the received success response, the category authorizer 550 of application 435 can identify the response as indicating that the threshold quantity is met. Additionally, model applicator 530 can continue to use the category and the corresponding category identifier 520.

[0150] On the other hand, if the data structure indicates that the quantity does not meet (e.g., is less than) a threshold quantity, the category authorizer 550 on the application manager service 410 can generate a failure response. This response can indicate that the quantity of application 435 does not meet the threshold quantity. The category authorizer 550 on the application manager service 410 can send the failure response to the category authorizer 550 on application 435. Based on the received failure response, the category authorizer 550 of application 435 can identify the response as indicating that the threshold quantity is not met. The model applicator 530 can discard the category and the corresponding category identifier 520 for further use. Alternatively, the category authorizer 550 can repeat the check using another category to find the category to use in the content request.

[0151] Once it is determined that at least one category meets the threshold number of applications 435 to be assigned to that category, the model applicator 530 can assign applications 435 to the corresponding category identifier 520 for that category. In some embodiments, the model applicator 530 can identify the category identifier 520 corresponding to the closest category determined to meet the threshold number. By default, the category identifier 520 assigned to applications 435 by the model applicator 530 can correspond to the closest category. In some embodiments, the model applicator 530 can use a classification model 500 to identify the category identifier 520 associated with the identified category. With this identification, the model applicator 530 can then assign the category identifier 520 to applications 435. Assigning the category identifier 520 to applications 435 can indicate that the browsing history 535 of applications 435 is similar to other browsing history 530 on other applications 435 having the same category identifier 520. The category identifier 520 assigned to the application 435 running on the client device 430 may be common, shared, or the same as the category identifier 520 of at least some other applications 435 running on other client devices 430.

[0152] Through the allocation, the model applicator 530 can store and maintain the category identifier 520 in the identifier cache 545. For example, as depicted, the model applicator 530 of the first application 435A may have identified the first application 435A as having a browsing pattern similar to other applications 435B-N having a category corresponding to the first category identifier 520A. The identifier cache 545 can control access to the category identifier 520 by scripts executed by application 435. In some embodiments, the model applicator 530 can store the category identifier 520 in a security cookie maintained in the identifier cache 545. The security cookie may include a cookie with a security flag or a set of flags for HTTP only. The security cookie can prevent scripts on information resources from accessing the category identifier 520 maintained in the identifier cache 545. Furthermore, the security cookie can allow authorized entities to access the category identifier 520 stored in the identifier cache 545. The security cookie can identify the content selection service 425 or the application manager service 410 as allowing access to the category identifier 520 in the identifier cache 545.

[0153] Model applicator 535 can determine whether to apply classification model 500 based on an identifier allocation strategy. The identifier allocation strategy can specify a set of conditions under which classification model 500 is applied for reassigning category identifiers 520. In some implementations, the identifier allocation strategy may include a schedule for indicating when classification model 500 is applied. The identifier allocation strategy can specify that classification model 500 is updated when application manager service 410 provides a new update to application 435. The identifier allocation strategy can specify that classification model 500 is applied when the number of accumulated additional browsing history 535 meets a threshold number. If it is determined that classification model 500 is not to be reapplied, model applicator 530 can maintain category identifiers 520. On the other hand, if it is determined that classification model 500 is to be reapplied, model applicator 530 can repeat the above-described functions. For example, model applicator 530 can identify browsing history 535, use a dimensionality reduction process to generate a simplified set of feature vectors, apply classification model 500 to the simplified set of feature vectors to identify the category, and assign a classifier identifier 520 associated with the identified category to application 435.

[0154] Now for reference Figure 6 This diagram depicts a block diagram illustrating one implementation of a client device 430, content provider 415, content publisher 420, and content selection service 425 within system 400. In general, an application 435 on each client device 430 (e.g., as depicted for a first client device 430A) may include at least one resource processor 615 to handle the reading and parsing of at least one information resource 600 and other data communicating with the content provider 415, content publisher 420, or content selection service 425. Application 435 may also include at least one identifier selector 620 to determine which category identifier 520 to insert into a request for content sent via network 405.

[0155] More specifically, the resource processor 615, executing on client device 430, can receive information resource 600 from content publisher 420. Receipt of information resource 600 can be in response to a request for information resource 600 sent by application 435 to content publisher 420, and can be used for presentation on client device 430. The received information resource 600 (e.g., a webpage) may include at least one main content 605 (e.g., the body, text, and images of the webpage) and at least one content slot 610 (e.g., an inline frame of the webpage). The main content 605 may correspond to a portion of the information resource 600 provided by content publisher 420. The content slot 610 can be used to receive content from content provider 415 or content selection service 425. Content to be inserted into content slot 610 may have a hostname different from the hostname of information resource 600. Once received, resource processor 615 can resolve information resource 600 including main content 605 and content slot 610.

[0156] For content slot 610 of information resource 600, resource processor 615 can generate requests 625A-N (generally referred to herein as requests for content 625) to be sent to content selection service 425. The generation of requests for content 625 can be based on a script (e.g., an ad tag or content selection tag) used to insert content into content slot 610. The script for content slot 610 can be embedded or included in content slot 610 itself or in another part of information resource 600. When generating requests for content 625, resource processor 615 can include addresses 630A-N (generally referred to herein as address 630) in the requests for content 625. Address 630 can reference the content selection service 425 specified by the script for content slot 610, such as a URL for content selection service 425. Address 630 can indicate the destination address to which the requests for content 625 should be routed. Furthermore, resource processor 615 can include a source address for referencing client device 430 in the requests for content 625. Resource processor 615 may also include an address corresponding to content publisher 420 that provides information resource 600 to application 435.

[0157] Furthermore, the identifier selector 620 can identify or otherwise select which category identifier 520 to insert into the request for content 625 based on at least one obfuscation strategy (sometimes referred to herein as a distribution strategy). The obfuscation strategy can specify one or more conditions that allow or restrict the inclusion of the category identifier 520 in the request for content 625. In some embodiments, the conditions of the obfuscation strategy can be specific to one or more information resources 600, the main content 605 of information resource 610, or the information slot 610 of information resource 600, or any combination thereof. In some embodiments, the conditions can depend on entries 540 of the browsing history 535. For example, the conditions of the obfuscation strategy can specify that another category identifier 520 or no category identifier 520 is included in the request for content 625 for the first access of information resource 600 via application 435. In another example, the conditions of the obfuscation strategy can specify that another category identifier 520 is used for information resource 600 that is rarely accessed (or has been accessed less than a threshold number as indicated by the browsing history 535). In another example, the obfuscation strategy could specify that class identifier 520 will not be used unless the received information resource 600 is in accordance with the Hypertext Transfer Protocol Security (HTTPS) protocol. In this way, the obfuscation strategy can further protect data privacy by allowing access to information resource 600 via application 435 on network 405.

[0158] When making selections based on obfuscation strategies, identifier selector 620 can identify one or more conditions for comparing obfuscation strategies. In some embodiments, identifier selector 620 can identify information resource 600 including various parts of content and address (e.g., protocol, domain name, and pathname). In some embodiments, identifier selector 620 can identify various main content 605 on information resource 600. In some embodiments, identifier selector 620 can identify content slot 610 for which a request 625 for content is to be generated. In some embodiments, identifier selector 620 can identify entries 540 of the browsing history 535 of application 435. Using these identifiers, identifier selector 620 can compare the conditions specified by the obfuscation strategy. If it is determined that the conditions do not match, identifier selector 620 can maintain category identifier 520 to be included in the request 625 for content.

[0159] On the other hand, if the conditions are determined to match, identifier selector 620 can determine whether the obfuscation policy specifies allowing or restricting category identifier 520. When the obfuscation policy specifies allowing category identifier 520, identifier selector 620 can maintain category identifier 520 to be included in the request for content 625. Conversely, when the obfuscation policy specifies restricting category identifier 520, identifier selector 620 can find another category identifier 520 or have no category identifier 520. In some embodiments, the obfuscation policy can specify that another category identifier 520 will be used under such conditions. Therefore, identifier selector 620 can use another category identifier 520 corresponding to the set of categories identified by model applicator 530. In some embodiments, the obfuscation policy can specify that category identifier 620 is not included under such conditions. Therefore, identifier selector 620 can prevent, remove, or otherwise restrict any category identifier 620 from being included in the request for content 625.

[0160] By selecting a category identifier 520, resource processor 615 can include the selected category identifier 520 for application 435 in the request for content 625. In some embodiments, resource processor 615 can access identifier cache 545 to retrieve category identifier 520. Once retrieved, resource processor 615 can include category identifier 520 to include the request for content 625. In some embodiments, the request for content 625 is initially generated by application 435 when parsing the script for content slot 610, and may initially include a unique tracking identifier. Resource processor 615 can remove or otherwise exclude any unique tracking identifier corresponding to application 435 (or the client device 430 running application 435, or the account associated with application 435) from the request for content 625. The unique tracking identifier may include, for example, a cookie user identifier corresponding to the account associated with application 435, or a cookie device identifier corresponding to the client device 430 running application 435. The unique tracking identifier may have already been provided by content provider 415 or another content placement service. Once removed, resource processor 615 can include category identifier 520 in the content request 625. In some implementations, resource processor 615 can replace the unique tracking identifier included in the content request 625 with category identifier 520. In some implementations, resource processor 615 can remove any identifier or any category identifier 520 that includes the tracking identifier in response to the determination of an obfuscation policy.

[0161] In some implementations, resource processor 615 may package or include category identifier 520 in a designated portion of a request for content 620. In some implementations, resource processor 615 may include category identifier 520 in at least one header field of a request for content 625. In some implementations, resource processor 615 may include category identifier 520 in the body of a request for content 625. In some implementations, resource processor 615 may include category identifier 520 in a cookie. In some implementations, a cookie may be generated in conjunction with a request for content 625. In some implementations, the cookie may be retrieved from application 435 (e.g., identifier cache 545). The cookie may have a set of security flags or HTTP-only flags to prevent unauthorized entities from intercepting and accessing category identifier 520. Unauthorized entities may include those other than content selection service 425 or application manager service 410. By setting security flags or HTTP-only flags, cookies may also restrict access to category identifier 520 over network 405 on a secure communication channel (e.g., Hypertext Transfer Protocol Security (HTTPS)). Resource processor 615 may include a cookie containing category identifier 520 in the content request 625. The cookie may also include an address corresponding to the content publisher 420 that provided information resource 600 to application 435. Upon generation, resource processor 615 may transmit the content request 625 to content selection service 425 via network 405. In some embodiments, application 435 may establish a secure communication channel (e.g., via HTTP) between client device 430 and content selection service 425 to transmit the content request 625. The establishment of this channel may allow content selection service 425 to access the category identifier 520 included in the cookie of the content request 625.

[0162] The category identifiers 520A-N assigned by the respective model applicators 530 to the different applications 435A-N running on client devices 430A-N may not be specific to a single application 435A-N and may not uniquely identify an application 435A-N. For example, as depicted, a first category identifier 520A may have been assigned to a first application 435A running on a first client device 430A and a second application 435B running on a second client device 430B. In contrast, a second category identifier 520B may have been assigned to an nth application 435N running on an nth client device 430N. This may be the opposite of a unique tracking identifier such as a user or client identifier that specifically identifies an application 435A-N or client device 430A-N provided by a content provider 415 or other content placement service. Furthermore, since the category identifier 520A-N may not uniquely identify each application 435 running on client device 430, the category identifier 520A-N may have lower entropy than such a unique tracking identifier. For example, the entropy of a unique tracking identifier can be greater than 63 bits, while the entropy of category identifiers 520A-N can be between 18 and 52 bits. Thus, the size of category identifiers 520A-N can be smaller than the size of these unique tracking identifiers, thereby reducing the size of content requests 625 sent over network 405.

[0163] Now for reference Figure 7 This diagram depicts a block diagram of one embodiment of a client device 430 and a content selection service 425 in system 400. The content selection service 425 may include at least one history aggregator 700 to store and maintain browsing history. The content selection service 425 may include at least one category characterizer 705 to determine selection parameters for each category. The content selection service 425 may include at least one content placer 710 to identify content items 725A-N (generally referred to herein as content items 725) for application 435 associated with request 620 by using the identified category. The content selection service 425 may include at least one history database 715 to maintain and store browsing history 720A-N (generally referred to herein as entries 720) of category identifiers 520.

[0164] More specifically, the history aggregator 700, which operates on the content selection service 425, can collect, aggregate, or otherwise maintain a history database 715 by using cookies 630 included in requests for content 620 received from client device 430. The history database 715 can include a set of entries 720 indexed by category identifiers 520 defined by the application manager service 410. Each entry 720 can include an address for the accessed information resource 600 and a timestamp indicating when the information resource 600 was accessed. Instead of aggregating the browsing history of client device 430 using unique tracking identifiers (e.g., cookie identifiers), the history aggregator 700 can aggregate browsing history via category identifiers 520. Unlike the browsing history 530 maintained on each individual application 435, the history database 715 may not individually identify the application 435 from which the entries 720 are generated (or the user associated with application 435). Each time a request for content 625 is received, the history aggregator 700 can identify the address corresponding to the information resource 600 on which content will be returned. The history aggregator 700 can also identify the category identifier 520 included in the content request 625. Using these identifiers, the history aggregator 700 can aggregate data including addresses and timestamps. Figure 4 Entry 720 (not shown in the text) is added to the set of entries 720 for use in the category identifier 520 included in the request for content 625.

[0165] The category representative 705, executing on the content selection service 425, can determine one or more features for a category based on entries 720 for each category identifier 520. These features may include, for example, common characteristics, profiles, behaviors, or interests of the category corresponding to the category identifier 520. In some embodiments, the category representative 705 can use a category profile model to determine features for the category based on a set of entries 720 for category identifier 520 in a history database 715. The category profile model can be any model, such as linear regression, logistic regression, artificial neural networks (ANN), support vector machines (SVM), and Naive Bayes classifiers. The category profile model can be trained using a sample dataset that associates browsing history tagged by category identifier 520 with certain features. The category profile model can transform entries 720 for each category identifier 520 in the history database 715 into features for the corresponding category. In some embodiments, the category representative 705 can store and maintain features for each category identifier 520. Content placer 710, executing on content selection service 425, can select or identify content items 725 from a set of content items 725 in response to a request for content 625 using category identifier 520. The use of category identifier 520 may contrast with the use of a unique identifier for a specific user associated with the request for content, as the selection of content items 725 may not be based on a unique identifier for a specific user. Each content item 725 may include an object or element to be embedded, inserted, or otherwise added to a content slot 610 of information resource 600. Each content item 725 may be provided by one or more content providers 415. Upon receipt, content placer 710 may parse the request for content 625 to identify category identifier 520. Upon identification, content placer 710 may identify characteristics of the category corresponding to category identifier 520. Content placer 710 may identify or select content items 725 associated with characteristics of the category. In some embodiments, content placer 710 may use a content placement process to select content items 725. The content placement process can use models such as linear regression, logistic regression, artificial neural networks (ANN), support vector machines (SVM), and Naive Bayes classifiers. For each content item 725, the content placement process can calculate, determine, or generate a predicted probability of user interaction within the category corresponding to the category identifier 520 included in the content request 625. The content placer 710 can identify the content item 725 with the highest probability of interaction with a user in that category. Once selected, the content placer 710 can send the content item 725 to the client device 430 that initiated the content request 625.In some implementations, the content placer 710 may send an address for the content item 725 to the client device 430 for the application 435 to retrieve the selected content item 725 from the content provider 415.

[0166] Combination Figure 7 Return to reference Figure 6 Resource processor 615 can receive content item 725 identified by content selection service 425. Resource processor 615 can embed, insert, or add content item 725 into content slot 610 of information resource 600. In some embodiments, resource processor 615 can receive the address of content item 725. The address of content item 725 can reference content provider 415. Resource processor 615 can send another request to content provider 415 to retrieve content item 725 and insert content item 725 into content slot 610 of information resource 600.

[0167] In this way, the content selection service 425 can select content items 725 based on the expectation that users associated with the same category identifier 520 will have similar responses. Furthermore, the security, integrity, and privacy of data on each user's browsing history 530 can be preserved. Simultaneously, the quality of human-computer interaction (HCI) with the overall information resource 600 can be maintained through the ability to determine the relevance of the selection of content items 725 to various categories.

[0168] Now for reference Figure 8 The document depicts a flowchart of one implementation of a method 800 for encoding identifiers used to select content using a classification model. The functionality described herein with respect to method 800 can be derived from, for example... Figure 4 The system 400 shown or Figure 9The computing device described herein performs or otherwise executes the operation. In a brief overview, an application running on a client device may identify the accessed information resource (805). The application may reduce dimensionality (810). The application may apply a classification model (815). The application may identify categories (820). The application may assign category identifiers (825). The application may determine whether a category identifier is allowed (830). If not, the application may assign another category identifier (835). The application may receive information resources (840). The application may generate a request for content (845). The application may determine whether to obfuscate the category identifier (850). If so, the application may use another category identifier (855). The application may include a category identifier (860). The application may transmit a request for content (865). The application may receive a selected content item (870). The application may determine whether to reassign (875). If not, the application may maintain the category identifier (880).

[0169] More specifically, an application (e.g., application 435) on a client device (e.g., client device 430) can identify the accessed information resource (805). In some embodiments, the application can identify the accessed information resource from browsing history (e.g., browsing history 530). Browsing history can include a set of entries (e.g., entry 540). Each entry can include the address of the accessed information resource and, in some embodiments, a timestamp to identify the time of access to the information resource. For each accessed information resource, the application can extract features from the content on the information resource. The application can generate a set of feature vectors from the set of entries in the browsing history. The feature vectors can be a projection of the browsing history onto a set of dimensions. Dimensions can include time ranges and address attributes, etc. The application can reduce the dimensions (810). By using a dimensionality reduction process, the application can generate a simplified set of feature vectors from an initial set of feature vectors. In some embodiments, step (810) can be performed across application 435 and another server. In some embodiments, step (810) can be omitted. In some embodiments, step (810) can be replaced by or combined with training of a classification model. For example, a classification model can be trained using a distributed learning protocol, such as joint learning using optimization algorithms (e.g., stochastic gradient descent (SGD) or averaging). During each iteration, each application can set or adjust the values ​​of the classification model by using the training dataset and can provide these values ​​to each other according to the distributed learning protocol.

[0170] The application may apply a classification model (e.g., classification model 500) to features extracted from stored information resources (815). The classification model may include an input set, a parameter set, and an output set. The classification model may be, for example, a classification algorithm, a clustering model, or a locality-sensitive hash function, etc. The input set may include features extracted from accessed information resources, entries in browsing history, or a representation of browsing history. The parameter set may associate the inputs with the outputs. The output set may include categories into which users interacting with the application are classified based on the user's browsing history on the application. The application may feed a set of dimensionality-reduced feature vectors as input to the classification model. The classification model may apply the parameters to the inputs. The application may identify categories (820). Through the application of the classification model, the classification model may generate outputs for identifying users interacting with the application and classifying them into one or more categories. The classification of the application may consequently correspond to or include categories of users associated with an account operating the application or a client device operated by that user. The application may assign category identifiers (e.g., category identifier 520) (825). The classification model may associate each category with one of the category identifiers. Once a category identifier is identified, the application can identify the category identifier corresponding to that category and assign the application to that category identifier.

[0171] The application can determine whether a category identifier is allowed (830). This determination can be coordinated or harmonized with an authentication service (e.g., a category authorizer 550 on application manager service 410), depending on the threshold cryptography scheme. The application can send an encrypted copy of the category identifier corresponding to the category. Under the threshold cryptography scheme, the authentication service may be unable to decrypt the category identifier unless the number of requests for encrypted copies of the same identifier exceeds a threshold. The inability to decrypt may be, for example, because the authentication service may not receive a sufficient portion of the decryption key (e.g., a private key) from the request. Once a sufficient portion is received, the authentication service is able to successfully decrypt the encrypted copy. The authentication service can return the output of the decryption attempt. The application can compare the output with the original category identifier. If a match is found, the application can determine that the category is allowed. Otherwise, the application can determine that the category is not allowed. If the category identifier is not allowed, the application can assign another category identifier (835). The application can find another category identifier from the identified set of categories and can repeat the functionality in (830).

[0172] The application can receive information resources (e.g., information resource 600) (840). The information resource may include main content (e.g., main content 605) and content slots (e.g., content slot 610). The main content may be provided by a content publisher (e.g., content publisher 420). The content slots can be used to insert content from a content provider (e.g., content provider 415) or a content selection service (e.g., content selection service 425). Upon receipt, the application can parse the information resource. The application can generate a request for the content (e.g., request for content 625) (845). Generating a request for content can be combined with the parsing of the information resource.

[0173] The application can determine whether to obfuscate the category identifier (850). This determination can be based on an obfuscation policy. The obfuscation policy can specify one or more conditions that restrict the inclusion of the category identifier in a request for content. For example, the conditions can include a security protocol (e.g., HTTPS) used to receive the information resource. In this example, the obfuscation policy can specify that the category identifier is not included in the request for content when the information resource is not encrypted under HTTPS. The application can identify conditions related to the information resource for comparison with the conditions specified by the obfuscation policy. When the conditions do not match, the application can determine not to obfuscate the category identifier and maintain the current category identifier. Otherwise, when the conditions do match, the application can determine to obfuscate the category identifier. If it is determined that the category identifier is obfuscated, the application can use another category identifier (855).

[0174] The application may include a category identifier in its request for content (860). The request for content may include a category identifier corresponding to a category identified using a classification model. The application may also remove any unique tracking identifiers associated with the user of the application, the application itself, or the client device running on the application. Unique tracking identifiers may have been provided as part of a third-party cookie from a content provider or another content placement platform. The category identifier may be included as part of a security cookie included in the request for content. The application may transmit the request for content to a content selection service or other content provider (865). This transmission may be made via a secure communication channel established between the client device and the content selection service. The request for content may be received by the content selection service. The content selection service may use the category identifier included in the request to identify a content item from a set of content items (e.g., content item 725). Through this selection, the content selection service may send the content item to the application. The application may receive the selected content item (870). The application may insert the content item into a content slot defined on the information resource.

[0175] The application can determine whether to reassign a category identifier (875). Reassignment can be based on an identifier assignment strategy. This strategy can specify a set of conditions under which the classification model will be reapplied to the browsing history to find a new category identifier. For example, the reassignment strategy can specify that the classification model should be reapplied when the number of additional entries in the browsing history since the previous assignment exceeds a threshold number. If it is determined that reassignment is to be performed, the application can repeat functions (805)-(835) and move forward. On the other hand, if no reassignment is determined, the application can maintain the category identifier (880).

[0176] Therefore, the system and method described in this paper allow for the selection of user-relevant content items without individually tracking user activity. In this way, the security, integrity, and privacy of data from each user's browsing history can be preserved. Simultaneously, the ability to determine the relevance of content item selection to various categories ensures the quality of human-computer interaction (HCI) with the overall information resource.

[0177] Figure 9 The overall architecture of an exemplary computer system 900, according to some embodiments, is shown. This system can be used to implement any computer system discussed herein, including application manager service 410 and its components, content provider 415 and its components, content publisher 420 and its components, content selection service 425 and its components, and client device 430 and its components. The computer system 900 can be used to provide information for display via network 930. The computer system 900 includes one or more processors 920 communicatively coupled to memory 925, one or more communication interfaces 905 communicatively coupled to at least one network 930 (e.g., network 405), and one or more output devices 910 (e.g., one or more display units) and one or more input devices 915.

[0178] Processor 920 may include a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. Memory may include, but is not limited to, electronic, optical, magnetic, or any other storage or transmission device capable of providing program instructions to the processor. Memory 925 may include any computer-readable storage medium and may store computer instructions such as processor-executable instructions for implementing the various functionalities described herein with respect to the corresponding system, and any data generated therefrom or received via a communication interface or input device (if present). Memory 925 may include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ASIC, FPGA, read-only memory (ROM), random access memory (RAM), electrically erasable ROM (EEPROM), erasable programmable ROM (EPROM), flash memory, optical media, or any other suitable memory from which the processor may read instructions. The instructions may include code from any suitable computer programming language.

[0179] Figure 9 The processor 920 shown can be used to execute instructions stored in memory 925, and in doing so, can also read from or write to memory various information processed and / or generated according to the execution of the instructions. The processor 920 coupled to memory 925 (collectively referred to herein as a processing unit) can be included in application manager service 410. For example, application manager service 410 may include memory 925 as database 515. The processor 920 coupled to memory 925 (collectively referred to herein as a processing unit) can be included in content provider 415. For example, content provider 415 may include memory 925 for storing content items 725. The processor 920 coupled to memory 925 (collectively referred to herein as a processing unit) can be included in content publisher 420. For example, content publisher 420 may include memory 925 for storing information resources 600. The processor 920 coupled to memory 925 (collectively referred to herein as a processing unit) can be included in content selection service 425. For example, content selection service 425 may include memory 925 as historical database 720. A processor 920 (collectively referred to herein as a processing unit) coupled to memory 925 may be included in client device 430. For example, client device 430 may include memory 925 as browsing history 535 and identifier cache 545.

[0180] The processor 920 of the computer system 900 can also be communicatively coupled to or configured to control the communication interface 905 to transmit or receive various information according to the execution of instructions. For example, the communication interface 905 can be coupled to a wired or wireless network, a bus, or other communication device, and thus allow the computer system 900 to send information to or receive information from other devices (e.g., other computer systems). Although in Figures 4 to 7 or Figure 9 While not explicitly shown in the system, one or more communication interfaces facilitate the flow of information between components of system 900. In some implementations, the communication interfaces can be configured (e.g., via various hardware or software components) to provide a website as an access portal to at least some aspects of computer system 900. Examples of communication interfaces 905 include user interfaces (e.g., application 435, information resource 600, main content 605, content slot 610, and content item 725) through which users can communicate with other users of system 400.

[0181] For example, it can provide Figure 9 The output device 910 of the computer system 900 shown allows for viewing or otherwise perceiving various information in conjunction with the execution of instructions. For example, an input device 915 may be provided to allow a user to manually adjust, select, input data, or interact with the processor in any of a variety of ways during instruction execution. Additional information relating to general computer system architectures that can be used in the various systems discussed herein is further provided herein.

[0182] Network 930 may include computer networks such as the Internet, local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs) or other LANs, intranets, satellite networks, other computer networks such as voice or data mobile phone communication networks, and combinations thereof. Network 930 may be any form of computer network that relays information between components of system 400, such as application manager service 410, content provider 415, content publisher 420, content selection service 425, and client device 430. For example, network 930 may include the Internet and / or other types of data networks, such as local area networks (LANs), wide area networks (WANs), cellular networks, satellite networks, or other types of data networks. Network 930 may also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) configured to receive and / or transmit data within network 930. Network 930 may further include any number of hardwired and / or wireless connections. For example, client device 430 can communicate wirelessly with a transceiver (e.g., via WiFi, cellular, radio, etc.) that is hardwired (e.g., via fiber optic cable, CAT5 cable, etc.) to other computing devices in network 930.

[0183] The embodiments of the subject matter and operations described in this specification may be implemented in digital electronic circuits, or in computer software embodied in tangible media, firmware, or hardware including the structures disclosed in this specification and their equivalents, or in a combination thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more components of computer program instructions encoded on a computer storage medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Program instructions may be encoded on artificially generated propagated signals, such as machine-generated electrical, optical, or electromagnetic signals, generated to encode information for transmission to a suitable receiver device for execution by the data processing apparatus. The computer storage medium may be or be included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof. Furthermore, although the computer storage medium is not a propagated signal, it may include a source or destination of computer program instructions encoded as artificially generated propagated signals. The computer storage medium may also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0184] The features disclosed herein can be implemented on a smart TV module (or connected TV module, hybrid TV module, etc.) that may include a processing module configured to integrate an internet connection with more traditional television program sources (e.g., via cable, satellite, wireless, or other signal reception). The smart TV module may be physically integrated into a television set or may include separate devices such as set-top boxes, Blu-ray or other digital media players, game consoles, hotel TV systems, and other complementary devices. The smart TV module can be configured to allow viewers to search for and find videos, movies, photos, and other content on the web, on local cable TV channels, on satellite TV channels, or stored on a local hard drive. A set-top box (STB) or set-top unit (STU) may include an information appliance that may contain a tuner and connect to the television set and external signal sources, tune the signals to content, and then display that content on the television screen or other display devices. The smart TV module can be configured to provide a main screen or top-level screen that includes icons for multiple different applications (such as web browsers) and multiple streaming services, connected cable or satellite media sources, other web channels, etc. The smart TV module can be further configured to provide users with an electronic program guide. Companion applications for the smart TV module can operate on mobile computing devices to provide users with additional information about available programming, allowing users to control the smart TV module, etc. In some embodiments, the features can be implemented on laptop computers or other personal computers, smartphones, other mobile phones, PDAs, tablets, or other computing devices. In some embodiments, the features disclosed herein can be implemented on wearable devices or components (e.g., smartwatches), which may include a processing module configured to integrate internet connectivity (with another computing device or network 930).

[0185] The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0186] The terms "data processing apparatus," "data processing system," "user equipment," or "computing device" encompass all types of apparatus, devices, and machines used for processing data, including, for example, the aforementioned programmable processors, computers, systems-on-a-chip or systems on multiple chips, or combinations thereof. The apparatus may include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program discussed, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, cross-platform execution environments, virtual machines, or combinations thereof. The apparatus and execution environment can implement various different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0187] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, objects, or other units suitable for use in a computing environment. A computer program may, but is not required to, correspond to a file in a file system. A program may be stored as part of a file used to store other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to the program under discussion, or multiple coordinating files (e.g., files used to store one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.

[0188] The processes and logic flows described herein can be executed by one or more programmable processors for executing one or more computer programs to perform actions by manipulating input data and generating outputs. The processes and logic flows can also be executed by dedicated logic circuitry, and the apparatus can also be implemented as such dedicated logic circuitry, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0189] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors of any kind of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are the processor for performing actions according to instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to mass storage devices to receive or transfer data therefrom, or both. However, it is not necessary for a computer to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive). Devices suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as: semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The processor and memory can be supplemented by or incorporated into dedicated logic circuitry.

[0190] To provide user interaction, embodiments of the subject matter described in this specification can be implemented on a computer with a display device and a keyboard and pointing device, such as a CRT (cathode ray tube), plasma, or LCD (liquid crystal display) monitor for displaying information to the user, and a pointing device such as a mouse and trackball through which the user can provide input to the computer. Other types of devices can also be used to provide user interaction. For example, feedback provided to the user can include any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0191] The embodiments of the subject matter described in this specification can be implemented in computing systems that include backend components (e.g., as a data server), middleware components (e.g., an application server), or frontend components (e.g., a client computer with a graphical user interface or web browser through which a user can interact with embodiments of the subject matter described in this specification), or combinations thereof, including one or more such backend, middleware, or frontend components. The components of the system can be interconnected via any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), the internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0192] Computing systems such as System 900 or System 400 may include clients and servers. For example, each of the application manager service 410, content provider 415, content publisher 420, and content selection service 425 of System 400 may include one or more servers in one or more data centers or server farms. Clients (e.g., client devices 430) and servers are typically geographically distant from each other and typically interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. In some implementations, the server transmits data (e.g., HTML pages) to the client device (e.g., for the purpose of displaying data to a user interacting with the client device and receiving user input therefrom). Data generated at the client device (e.g., the result of user interaction) may be received from the client device at the server.

[0193] The embodiments of the subject matter described in this specification can be implemented in computing systems that include backend components (e.g., as a data server), middleware components (e.g., an application server), or frontend components (e.g., a client computer with a graphical user interface or web browser through which a user can interact with embodiments of the subject matter described in this specification), or combinations thereof, including one or more such backend, middleware, or frontend components. The components of the system can be interconnected via any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (“LANs”) and wide area networks (“WANs”), the internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0194] The systems described herein that collect personal information about users, or where personal information may be used, can provide users with the opportunity to control whether a program or feature can collect user information (e.g., information about the user's social networks, social behaviors or activities, occupation, user preferences, or the user's current geographic location), or to control whether and / or how user-relevant content is received from the content server. Furthermore, some data may be processed in one or more ways before storage or use, such that personally identifiable information is removed when parameters are generated. For example, a user's identity may be anonymized, making it impossible to determine any personally identifiable information for that user, or the user's geographic location may be generalized (such as city, zip code, or state level) if location information is available, making it impossible to determine the user's specific location. Therefore, users can control how information about them is collected and / or used by the content server.

[0195] Although this specification contains details of many specific embodiments, these should not be construed as limiting any invention or the scope of the claims, but rather as descriptions of features specific to particular embodiments of the systems and methods described herein. Certain features described in the context of separate embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in certain combinations and even initially claimed in this way, in some cases one or more features from the claimed combination may be removed, and the claimed combination may be for sub-combinations or variations thereof.

[0196] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring that such operations be performed in the specific order shown or sequentially, or that all illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0197] Therefore, specific embodiments of this subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and the desired result can still be achieved. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking or parallel processing can be utilized.

Claims

1. A method for anonymization to provide pseudo-personalized clustering, comprising: One or more computing devices aggregate vectors received from client devices and vectors obtained from other client devices into a matrix; The matrix is ​​reduced in dimension by the one or more computing devices to obtain a dimension-reduced matrix representing the aggregated vectors obtained from the client device and the other client devices; The clustering of the dimensionality reduction matrix is ​​determined by the one or more computing devices; The classifier model is adjusted by the one or more computing devices based on the identified clusters and singular vectors of the dimensionality reduction matrix; The one or more computing devices transmit at least some of the singular vectors and the weights of the classifier model to the client device; The request for content is received from the client device by the one or more computing devices, the request including a clustering identifier generated by the client device using at least some of the singular vectors in the singular vectors, the weights of the classifier model, and features of the resource accessed by a first application executed at the client device; and In response to the request, the client device is transmitted with content selecting a parameter using one of the clusters corresponding to the cluster identifier received in the request.

2. The method according to claim 1, further comprising: The parameters of each specific cluster in the cluster are determined by the one or more computing devices based on the weights of the classifier model and the singular vectors of the dimensionality reduction matrix.

3. The method according to claim 2, wherein, Determining the parameters for each specific cluster includes: For each specific cluster: Calculate a profile vector representing the aggregated browsing history of client devices represented by this specific cluster; and The parameters of the cluster are inferred based on the keywords or topics corresponding to the aggregated browsing history.

4. The method according to claim 1, further comprising: The client device identifies multiple resources accessed through the first application executed on the client device; and The client device extracts one or more features from each of the plurality of resources that correspond to at least a portion of the content provided by that resource.

5. The method according to claim 4, further comprising: The cluster identifier is generated by the client device using at least some of the singular vectors, the weights of the classifier model, and one or more features extracted from each of the plurality of resources; The client device generates the request for the content, which includes the clustering identifier; and The request, including the clustering identifier, is transmitted from the client device to the one or more computing devices.

6. The method according to claim 5, further comprising: The client device determines that at least a threshold number of applications have been assigned the cluster identifier, wherein including the cluster identifier in the request is conditional on at least the threshold number of applications being assigned the cluster identifier.

7. The method according to claim 6, wherein, Determining that at least the threshold number of applications are assigned the cluster identifier includes using cryptographic identifiers generated with the cluster identifiers to collaboratively execute a threshold cryptographic protocol with an authorization service.

8. A system comprising: One or more computing devices; as well as One or more memory devices coupled to the one or more computing devices, wherein instructions executed by the one or more computing devices cause the one or more computing devices to perform operations, the operations including: The vectors received from the client device are aggregated with the vectors obtained from other client devices into a matrix; Calculate the dimensionality reduction of the matrix to obtain a dimensionality-reduced matrix representing the aggregated vectors obtained from the client device and the other client devices; Determine the clustering of the dimensionality reduction matrix; The classifier model is adjusted based on the identified clusters and singular vectors from the reduced-dimensional matrix. Transmit at least some of the singular vectors and the weights of the classifier model to the client device; A request for content is received from the client device, the request including a clustering identifier generated by the client device using at least some of the singular vectors, the weights of the classifier model, and features of a resource accessed by a first application executed on the client device; and In response to the request, the client device is transmitted with content selecting a parameter using one of the clusters corresponding to the cluster identifier received in the request.

9. The system according to claim 8, wherein, The instructions cause the one or more computing devices to perform an operation, the operation further including: The parameters of each specific cluster in the clustering are determined based on the weights of the classifier model and the singular vectors of the dimensionality reduction matrix.

10. The system according to claim 9, wherein, Determining the parameters for each specific cluster includes: For each specific cluster: Calculate a profile vector representing the aggregated browsing history of client devices represented by this specific cluster; and The parameters of the cluster are inferred based on the keywords or topics corresponding to the aggregated browsing history.

11. The system according to claim 8, further comprising: A client device configured to execute instructions, the instructions causing the client device to perform operations, the operations including: Identify multiple resources accessed through the first application running on the client device; and Extract one or more features from each of the plurality of resources that correspond to at least a portion of the content provided by that resource.

12. The system according to claim 11, wherein, The instruction causes the client device to perform an operation, the operation further including: The cluster identifier is generated using at least some of the singular vectors, the weights of the classifier model, and one or more features extracted from each of the plurality of resources; Generate the request for the content, including the clustering identifier; and The request, including the clustering identifier, is transmitted to the one or more computing devices.

13. The system according to claim 12, wherein, The instruction causes the client device to perform an operation, the operation further including: A minimum threshold number of applications are determined to have been assigned the cluster identifier, wherein the inclusion of the cluster identifier in the request is conditional upon at least the threshold number of applications being assigned the cluster identifier.

14. The system according to claim 13, wherein, Determining that at least the threshold number of applications are assigned the cluster identifier includes using cryptographic identifiers generated with the cluster identifiers to collaboratively execute a threshold cryptographic protocol with an authorization service.

15. A non-transitory computer-readable medium storing one or more instructions, which, when executed by one or more data processing means, cause the one or more data processing means to perform operations, the operations including: The vectors received from the client device are aggregated with the vectors obtained from other client devices into a matrix; Calculate the dimensionality reduction of the matrix to obtain a dimensionality-reduced matrix representing the aggregated vectors obtained from the client device and the other client devices; Determine the clustering of the dimensionality reduction matrix; The classifier model is adjusted based on the identified clusters and singular vectors from the reduced-dimensional matrix. Transmit at least some of the singular vectors and the weights of the classifier model to the client device; The client device receives a request for content, the request including a clustering identifier generated by the client device using at least some of the singular vectors in the singular vectors, the weights of the classifier model, and features of the resource accessed by a first application executed at the client device. and In response to the request, the client device is transmitted with content selecting a parameter using one of the clusters corresponding to the cluster identifier received in the request.

16. One or more non-transitory computer-readable media according to claim 15, wherein, The instruction causes the one or more data processing devices to perform an operation, the operation further including: The parameters of each specific cluster in the clustering are determined based on the weights of the classifier model and the singular vectors of the dimensionality reduction matrix.

17. One or more non-transitory computer-readable media according to claim 16, wherein, Determining the parameters for each specific cluster includes: For each specific cluster: Calculate a profile vector representing the aggregated browsing history of client devices represented by this specific cluster; and The parameters of the cluster are inferred based on the keywords or topics corresponding to the aggregated browsing history.

18. One or more non-transitory computer-readable media according to claim 15, wherein, The instruction causes the client device to perform an operation, the operation including: Identify multiple resources accessed through the first application running on the client device; and Extract one or more features from each of the plurality of resources that correspond to at least a portion of the content provided by that resource.

19. One or more non-transitory computer-readable media according to claim 18, wherein, The instruction causes the client device to perform an operation, the operation further including: The cluster identifier is generated using at least some of the singular vectors, the weights of the classifier model, and one or more features extracted from each of the plurality of resources; Generate the request for the content, including the clustering identifier; and The request, including the clustering identifier, is transmitted to the one or more data processing devices.

20. One or more non-transitory computer-readable media according to claim 19, wherein, The instruction causes the client device to perform an operation, the operation further including: A minimum threshold number of applications are determined to have been assigned the cluster identifier, wherein the inclusion of the cluster identifier in the request is conditional upon at least the threshold number of applications being assigned the cluster identifier.

Images