Deep learning for real time targeting using vector embeddings

Abstract

A system for real-time contextual message targeting using deep learning embeddings enables targeted message delivery without relying on third-party identifiers or storing personally identifiable information. The system comprises a deep-learning model that processes user interaction data through a temporal graph builder and relational graph transformer to generate high-dimensional vector embeddings representing user interests. These embeddings are stored in a vector database with privacy-preserving user identifiers. A similarity search engine performs nearest neighbor searches to match user embeddings with campaign embeddings for message relevance determination. The system operates within real-time bidding environments with sub-100 millisecond latency, storing vector embeddings in first-party cookies or CDN layers within first-party domains.

Description

Attorney Docket No. RAM P-00316-WODEEP LEARNING FOR REALTIME TARGETING USING VECTOR EMBEDDINGS CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to US Provisional Patent App. No. 63 / 744,450, filed on January 13, 2025. Such application is incorporated by reference in its entirety.FIELD OF THE INVENTION

[0002] The present invention relates generally to digital messaging systems, and more particularly to systems and methods for real-time contextual message targeting using deep learning embeddings without reliance on persistent user identifiers.BACKGROUND OF THE INVENTION

[0003] Personally Identifiable Information (PI I) refers to data that can be used to identify, contact, or locate a specific individual or user. This includes direct identifiers like names, addresses, and social security numbers, as well as indirect identifiers that can be combined with other information to identify a person. The use of Pll for message targeting in an online environment has become disfavored due to privacy concerns.

[0004] In addition, traditional digital messaging systems rely heavily on tracking identifiers and personal information to target messages to users. These systems typically use device IDs, third- party cookies, first-party cookies, connected TV IDs, IP addresses, email addresses, and phone numbers to resolve to person-based identifiers. However, increasing privacy regulations and consumer concerns have created challenges for such tracking-based approaches.

[0005] In digital messaging systems, contextual messaging refers to delivering personalized content, notifications, or communications to users based on specific contexts like their behavior, location, time, preferences, or other relevant factors. The goal is to provide more relevant and timely messages that resonate with the recipient's current situation or needs. Non-limiting examples could include sending a mobile notification about nearby restaurants around lunchtime; showing different website content based on whether someone is a new or returning visitor; customizing email content based on a customer's previous purchases; adjusting app notifications based on a user's typical usage patterns; and modifying targeted messages based on the weather in the recipient's location. The key principle is that the message's content,Attorney Docket No. RAM P-00316-WOtiming, and delivery channel are all informed by contextual data to make the communication more meaningful and effective.

[0006] In the field of machine learning, an embedding is a deep-learning technique that transforms discrete variables (such as words or categories) into dense vectors of continuous numbers in a lower-dimensional space. The vectors are used to capture semantic relationships and meaningful patterns in data. Thus embeddings allow neural networks to work with categorical data by converting it into numerical form, reducing dimensionality while preserving important relationships between items. Embeddings are learned from data during training.

[0007] In the field of data searching, "nearest neighbor" refers to finding the closest point(s) to a given point in a dataset, based on some distance measure. It is a fundamental concept used in machine learning, pattern recognition, and data mining. The most basic version is the k-nearest neighbors (k-NN) algorithm, where k represents how many nearby points for which a search is conducted. For example, if k=l, then the single closest point is returned, whereas if k=3, then the three closest points are returned. The "distance" between points can be measured in various ways. Non-limiting examples include Euclidean distance (straight-line distance);Manhattan distance (distance along axes); cosine similarity (particularly for high-dimensional data); and Hamming distance (for binary data). Common applications of nearest neighbor searching include classification (predicting a category based on the categories of nearby points); recommendation systems (finding similar items or users); image recognition (finding similar images); and anomaly detection (identifying outliers based on distance to neighbors).

[0008] Current solutions to the problem of real-time message targeting face several technical challenges, including the need to store and process large amounts of personal data, complex cross-domain identifier mapping infrastructure, and multiple database queries for targeting. These approaches also raise privacy concerns and face regulatory scrutiny.SUMMARY OF THE INVENTION

[0009] The present invention provides systems and methods for real-time contextual message targeting using deep learning embeddings. The invention enables relevant message placement without relying on third-party identifiers or personal information, while maintaining compatibility with real-time bidding protocols. This is achieved in certain embodiments by training a deep-learning model that understands identity, that is, the temporal graph structure represents how people interact with the world from their various touchpoints and devices. A supply-side inventory of vectors is built up, which may be placed in a first-party cookie or on theAttorney Docket No. RAM P-00316-WOcontent delivery network (CDN) layer in a first-party domain. Those wishing to send targeted messages then can target those messages using the vectorized identity / context embedding stored as vectors on-device. This can also be used for joining audience datasets for the purposes of data collaboration, or for activating first-party based identifiers to premium publisher platforms for the purposes of traditional identity-based targeting.

[0010] Historically, the use of person-based identifiers in targeted messaging was only used as a means to an end, for those sending messages to effectively reach their audience and subsequently measure the effectiveness of that reach. The most effective way to target individuals is to understand what they are interested in at that point in time and display the most relevant message at that point in time. This requires much more than identity. Identity helps to build that signal in methods used today by acting as a bridge between those sending messages and those receiving them. Each step along that bridge has the potential to introduce error (e.g., resolving to an identifier that corresponds to a different person, or resolving to the correct person but only after such person has made a purchase of the goods or services that are the subject of the message). This invention, in certain embodiments as described herein, can be used to shorten that bridge, and as a result offers a stronger and more relevant signal to connect those sending messages to those receiving them through online channels.

[0011] Improvements in relevant message placements without the use of third-party identifiers allows for increased security and privacy of the compute environment by preventing proliferation and movement of sensitive data. Additionally, this invention is taking advantage of the trend in chip design to offer more efficient chip architectures for deep-learning inference operations, thereby allowing the compute workloads needed for relevant message placements to be distributed to what is referred to as edge computing, that is, a processing model that brings data processing and storage closer to the source of data, rather than relying solely on centralized cloud computing

[0012] More specifically, this invention in certain embodiments provides several concrete technical improvements to the computing environment. First, it allows for reduced storage requirements. Traditional message targeting systems require storing extensive user profiles and tracking data across multiple databases, whereas the embedding-based approach of the present invention reduces storage requirements by representing user interests in compact, fixed-size vectors. The invention eliminates the need to store and manage persistent cross-site identifiers and tracking cookies, resulting in a reduction in the number of rows to match against by a factorAttorney Docket No. RAM P-00316-WOof 106to 107on average, based on real-world messaging campaigns. This is due to campaignlevel matching versus individual identifier matching per campaign. The invention further results in quantifiable reduction in database size and memory requirements per user.

[0013] In addition, the invention provides for network traffic reduction. Traditional systems require multiple database lookups and cross-system ID synchronization. In certain embodiments, the present invention reduces network traffic by transmitting only compact embeddings (e.g., 256-dimensional vectors) instead of full user profiles. It also eliminates crosssite identifier synchronization calls between systems using different identifier systems.

[0014] In addition, the invention in certain embodiments provides for process optimization. It replaces multiple sequential database queries with efficient vector-similarity computations, and leverages modern GPU / TPU (graphics processing unit / tensor processing unit) architectures for parallel processing of embedding operations. The invention in various embodiments further enables edge computing deployment through lightweight inference models, resulting in a demonstrable reduction in CPU cycles per message request.

[0015] In addition, the invention in certain embodiments results in system architecture improvements. The invention eliminates the need for complex cross-domain identifier mapping infrastructure, reduces system complexity by removing the need for cookie synchronization and identifier translation layers, and enables more efficient horizontal scaling of message-serving infrastructure.

[0016] These and other features, objects and advantages of the present invention will become better understood from a consideration of the following detailed description of the preferred embodiments and appended claims in conjunction with the drawings as described following:BRIEF DESCRIPTION OF DRAWINGS

[0017] Fig. 1 is an overall architecture for a deep-learning model according to an embodiment of the present invention.

[0018] Fig. 2 is an embedding model using CDN infrastructure according to an embodiment of the present invention.

[0019] Fig. 3 is an embedding model without using CDN infrastructure according to an embodiment of the present invention.

[0020] Fig. 4 is a diagram showing an overall architecture of the system according to an embodiment of the present invention.Attorney Docket No. RAM P-00316-WO

[0021] Fig. 5 is a diagram showing an overall architecture of the system in a real-time bidding environment according to an embodiment of the present invention.

[0022] Fig. 6 is a schematic for a computing component of a computing cluster for the implementation of an embodiment of the present invention.DETAILED DESCRIPTION OF THE INVENTION

[0023] Before the present invention is described in further detail, it should be understood that the invention is not limited to the particular embodiments described, and that the terms used in describing the particular embodiments are for the purpose of describing those particular embodiments only, and are not intended to be limiting, since the scope of the present invention will be limited only by the claims.

[0024] Central to an embodiment of the present invention is a proprietary interest embedding model, specifically designed and trained to capture the nuanced interests of users in the digital messaging ecosystem, which will be used to construct the vector database. The interest embedding model utilizes a deep neural network architecture, combining elements of transformers for processing sequential data and graph neural networks to capture complex relationships between interests.

[0025] The model input layer is designed to handle diverse data types including user actions, content metadata, and contextual information. Multiple self-attention layers allow the model to weigh the importance of different input features dynamically. The output layer produces a high-dimensional embedding (e.g., 512 dimensions) that represents the user's interests in a rich, nuanced manner.

[0026] The model is trained on a vast corpus of anonymized data, including historical user interactions across various digital platforms; content consumption patterns from diverse publishers; message interaction data from multiple campaigns and industries; and conversion and engagement metrics from past messaging efforts. The proprietary dataset ensures that the model captures interests and behaviors specific to the digital messaging ecosystem.

[0027] The training process employs a multi-task learning approach, simultaneously optimizing for interest prediction (i.e., accurately forecasting user engagement with different content types); message relevance (i.e., maximizing the correlation between user interests and successful message interactions; and temporal dynamics (i.e., capturing how interests evolve over time. Advanced techniques like curriculum learning and adversarial training are used to enhance the model's robustness and generalization capabilities.Attorney Docket No. RAM P-00316-WO

[0028] Another feature of the embodiment is continuous learning. The model undergoes regular updates using federated learning techniques, allowing it to adapt to emerging trends and shifting user behaviors without compromising individual user privacy. An A / B testing framework is integrated to continuously evaluate and improve the model's performance in real- world scenarios.

[0029] Differential privacy techniques are applied during training to ensure that the model doesn't memorize individual user data, thereby providing privacy. Fairness constraints are incorporated to mitigate potential biases in interest representation across different user groups.

[0030] Upon receiving the data, a backend system performs a number of steps. The first step is data transformation. The incoming analytics / traffic data is processed and transformed into a standardized format suitable for our interest modeling.

[0031] The second step is interest embedding generation. Utilizing the proprietary interest embedding model, the system converts the standardized data into high-dimensional interest embeddings. These embeddings capture the nuanced aspects of user interests based on their interactions and behaviors, leveraging the deep insights gained from the extensive training data.

[0032]

[0033] Finally, the interest embeddings are stored in a high-performance vector database. This database is optimized for rapid nearest neighbor searches, crucial for real-time message targeting.

[0034] Hardware today may not be able to locally run a large enough model to sufficiently capture this information, but the trend on hardware development and efficiency of deep learning inference is heading in the right direction. But even in the absence of inference- optimized chip hardware on users devices, the site context, user actions, and authentication events can still be passed server side as they are today. This model can additionally and optionally contain user authentication information that captures the same identity signals used today. The embedding, which represents the users interests is constantly changing and does not contain sensitive PI I, is sent in the message request to the provider.

[0035] When a customer requires specific message targeting, that customer can utilize the system in two primary ways. The first is a user-based lookup. If the customer has a specific user first-party identifier, it can perform a direct lookup to retrieve that user's current interest embedding. A second approach is an interest-based search. For broader targeting, customers can provide a target interest embedding (derived from their message content or target audienceAttorney Docket No. RAM P-00316-WOprofile) and perform a nearest K lookup. This returns the K user identifiers whose interest embeddings are most similar to the target, enabling highly relevant message targeting.

[0036] In this system according to an embodiment of the invention, the provider also maintains a vector database of all active campaigns for customers. The system applies a different deep learning model that can understand the product or service the party sending the messages is campaigning for and generate an embedding that can be matched to the consumer's interest embedding. So long as the same vector format is used (i.e., the data is tokenized consistently), different models may be employed, and matching can still be meaningfully performed.

[0037] In order to fulfill the message request with a winning and relevant message placement, this embedding must be matched to the universe of demand which is stored in the vector database of messaging campaigns that the provider is facilitating. Rather than asking the question, "is this ID present in an active campaign?" (which is how supply side platforms or SSPs and demand side platforms or DSPs work today), the question this model answers is "which active creative will most effectively lead to a conversion?" This is achieved through efficient nearest neighbor searches in the embedding space, comparing the user's interest embedding with the embeddings of available messaging campaigns.

[0038] Provided that the model considers real-time signals such as which impressions have already been made, and which were conversions and which were not, then the predictions made by the model's inference will improve over time.

[0039] The system continuously updates user embeddings based on new interaction data, ensuring that the interest representations remain current and accurate.

[0040] Because the feedback signals are encoded in the model, frequency capping is considered by sending those signals back into the model and updating the user's embedding, i.e. the updated embedding would no longer produce a high enough relevant score when performing the nearest neighbor search in the message inventory database. This can be accomplished without using actual tracking identifiers.

[0041] Additionally, the use of reinforcement signals in the message response is important as well. Any single publisher site would not adequately capture the full view of the user's interests at a point in time. Updates to the user's embedding may propagate across devices and sites in a probabilistic manner. The selection of the winning placement can use a perturbative technique to select near neighbors that may not necessarily be the nearest, and then use reinforcement signals of message interaction, conversions, or lack thereof to update the user's interestAttorney Docket No. RAM P-00316-WOembedding. The winning bid is then returned in the message response, which contains the relevant information for displaying the message on the user's device.

[0042] The use of identity is still supported in this embodiment of the present invention, and in particular first-party consented data would become a valuable signal that can be used to update a consumer's interest embedding. This is achieved through collaboration between first-party data controllers and facilitated by an identity provider.

[0043] Improvements in relevant message placements without the use of third-party identifiers allows for increased security and privacy of the compute environment by preventing proliferation and movement of sensitive data. Additionally, in certain embodiments this invention is taking advantage of the trend in chip design to offer more efficient chip architectures for deep learning inference operations, thereby allowing the compute workloads needed for relevant message placements to be distributed to what is referred to as edge computing.

[0044] In order to be effective in a real-time bidding environment, processing must take place within a particular timeframe. In an embodiment as described herein, the complete round trip of (1) the inference of the interests context and generation of an embedding that encapsulates the real-time context of the consumers browsing; (2) the vector search; (3) the selection of the winning bidder; and (4) the response back to the browser must all happen within this timeframe. In one embodiment, this timeframe is 100ms or less. This places limits on the size of the interference model running on edge. The vector database must support an indexing method to enable quick retrieval of nearest neighbors.

[0045] A particular implementation of the deep-learning model 10 is shown in Fig. 1. The deeplearning model comprises several interconnected components operating in a coordinated fashion. One component is raw event stream processing 12, through which the system processes raw event streams including clicks, searches, and conversions. These events are enriched with contextual information using large language models (LLMs) at context enrichment 14. A temporal graph builder 16 constructs heterogeneous graphs from the enriched event data. The graph structure captures relationships between users, content, time, and context, creating edge feature vectors that encode temporal and relational information. In one embodiment, the graph utilizes node types including "user," "item," "click," "campaign," "query," "device," and "IP address" with corresponding edge relationships.Attorney Docket No. RAM P-00316-WO

[0046] A specialized transformer architecture 18 processes the temporal graph structure using temporal and relation-aware attention mechanisms. In a preferred embodiment, the system employs a two-layer relational graph transformer with eight attention heads and 128- dimensional hidden layers, incorporating time-decay bias on edges to weight recent interactions more heavily.

[0047] The system generates vector representations of stable identity and current intent through a readout / fusion 20 mechanism that combines the user's latest sub-graph with recent edge context. The resulting embeddings capture both persistent user characteristics and dynamic contextual signals.

[0048] Finally, a prediction head 22 generates next-event probabilities, enabling the system to predict user behavior and optimize for specific campaign objectives including cross-entropy and ranking-based loss functions.

[0049] The invention establishes vector-enabled supply-side inventory through dynamic, realtime embedding creation and updates. Unlike conventional identifier-based systems, this approach stores vector embeddings directly in first-party cookies or at the CDN layer within first-party domains. In one embodiment, shown in Fig. 2, users access websites 36 through CDN reverse proxies 34. Event logs 40 capture user interactions on user devices 30 through Internet 32, which are processed by an identity + context model 44 to generate user context vectors 42 stored as first-party data on user devices 30. The system caches page content and associated page-context vectors at cached + context vector 46, enabling real-time matching without relying upon or requiring third-party identity providers.

[0050] An alternative approach, shown in Fig. 3, is similar but uses networks where JavaScript tags 50 collect user interaction data. Event logs 40 feed the identity + context model 44, generating user embeddings 42 stored in first-party cookies on user devices 30. This architecture maintains user privacy while enabling sophisticated targeting capabilities.

[0051] The invention provides two primary methods for targeting users with vectorized identity and context embeddings. A first method is audience matching. In this method, the system matches audience data to premium publisher audiences using vector similarity operations in high-dimensional embedding space. A second method is intent-based matching. Machine learning models interpret user context embeddings to understand intent and match users directly to campaign or product-level data without requiring pre-existing audience segments.Attorney Docket No. RAM P-00316-WO

[0052] The system addresses the computational challenge of joining potentially large datasets represented as collections of points in high-dimensional embedding space. Multiple algorithmic approaches are supported. These include baseline similarity, metric-learning models, and locality-sensitive hashing (LSH) + community detection.

[0053] The joining operation supports latency requirements on the order of hours and produces mappings between first-party identifiers from different datasets. This enables high- accuracy data collaboration without dependence on third-party identity graphs.

[0054] The system's privacy-supporting architecture employs collaboration nodes with privacy layers and embedding join algorithms. Enterprise identifiers are processed through privacypreserving mechanisms before generating cached / deidentified mappings for activation to premium publishers.

[0055] The system implements real-time matching between user context embeddings and campaign vectors through sophisticated similarity search mechanisms. When user context embeddings are compared against available campaign vectors, the distance between vectors represents the relevance of products or services to user intent. Closer matches indicate higher relevance and targeting appropriateness. In cases where the closest campaign match exhibits significant vector distance from user embeddings, the system identifies unmet market demand. This provides real-time signals for product development opportunities and market gaps.

[0056] As shown in Fig. 4, the system integrates with real-time bidding platforms through interests-based message inventory vector database 60. Fig. 4 shows functions of a publisher in the left column, of the provider of the service in the middle column, and of the party wishing to send targeted messages in the right column. Nearest-neighbor vector search algorithms 74 use vector database 60 to match user interests to available product description / creatives inventory 72, with bidding optimization at bidder 70 considering multiple variables including budget constraints and reinforcement learning signals through interests vector search 68. This is processed through matching process 76, using first-party identifiers + context 80. A display message 66 is fed into real-time interests model 64 in response to, for example, user interacts 62. Embedding merge 78 links real-time interests model 64 with first-party identifiers + context 80. The invention continuously updates user embeddings based on real-time interaction data to optimize campaign performance.

[0057] When users view messages (impressions), the system updates user context embeddings in real-time within first-party domains. This immediate feedback loop allows the model toAttorney Docket No. RAM P-00316-WOunderstand user response patterns and adjust future targeting accordingly. Purchase and conversion events provide strong signals for embedding updates. The system automatically reduces targeting for purchased products while identifying opportunities for complementary or sequential purchases. This prevents messaging waste while maximizing customer lifetime value. The system optimizes for multiple campaign objectives simultaneously, including impression reach maximization and return on spend improvement. The deep learning architecture naturally incorporates these diverse signals to improve overall campaign performance.

[0058] As shown in Fig. 5, an alternative version of the system integrates with real-time bidding platforms by returning a pre-agreed upon DeallD as a result of the vector search. The DeallD represents a private marketplace deal between messenger and publisher. Supply side platforms (SSPs) and demand side platforms (DSPs) already support bidding on DeallDs as part of the real time bidding protocol, making this approach compatible with execution platforms as part of what is known as a curation flow in digital messaging. In this example, processing begins at content, user actions, clicks 62, when a user embedding is generated or updated. Real-time interests model 64 then sends a request with an embedding to SSP 90. User-initiated activation 94 registers DeallDs with SSP 90 and DSP 92. The embedding from real time interests model 64 is passed along to interests vector search 68, which executes a nearest neighbor search against vector database 60. The corresponding DeallD is returned to SSP 90, which then sends that data along with the real-time bidding information to DSP 92. The winning bid then is determined in the usual fashion, and the corresponding message is displayed to the user at display message 66. The user embedding is generated or updated again at real-time interests model 64 after the message is displayed to the user.

[0059] The invention significantly improves computational efficiency through distributed edge computing approaches. By performing first-pass inference at CDN layers or device levels, the system distributes computational load across massive edge infrastructure. This approach makes operations economically feasible that would be prohibitively expensive in centralized data centers. Local processing capabilities on user devices enable privacy-preserving computation while reducing bandwidth requirements and improving response times.

[0060] The system produces stable first-party identifier mappings through embedding joins, avoiding repetitive costly vector computations. Output mappings enable traditional SQL join operations on subsequent data processing, dramatically reducing computational overhead for recurring operations. Computed embedding similarities and identifier mappings are cached toAttorney Docket No. RAM P-00316-WOprevent redundant calculations. This optimization is particularly beneficial for recurring audience matching operations across multiple campaigns.

[0061] The system's deep-learning model utilizes specific architectural parameters optimized for identity resolution tasks. These include a two-layer graph transformation layer architecture with temporal decay mechanisms, eight parallel attention mechanisms for multi-faceted relationship modeling, 128-dimensional representations balancing expressiveness with computational efficiency, and 28 distinct customer journey types providing comprehensive behavioral coverage.

[0062] The system maintains strict latency requirements for production deployment. For embedding updates, the system provides real-time updates within milliseconds of user interactions. For vector search, the system provides sub-second similarity search across large campaign databases. For audience joins, the system provides batch processing completing within hours for large dataset operations. The system's model inference approach is edge- optimized, providing immediate predictions

[0063] The system maintains user privacy through several architectural choices. First, all user embeddings are stored within first-party domains. Raw user data is converted to anonymous vector representations that contain no personally identifying information. Edge computing reduces central data aggregation. And vector operations may be performed without exposing underlying personal identifiers.

[0064] The system thus enables sophisticated message targeting and audience insights while maintaining user privacy and computational efficiency through innovative applications of deep learning, edge computing, and vector similarity operations.

[0065] The methods described herein may in various embodiments be implemented by any combination of hardware and software. For example, in one embodiment, the methods may be implemented by a computer system (e.g., a computer system as in Fig. 6) or a collection of computer systems, each of which includes one or more hardware processors executing program instructions stored on a computer-readable physical storage medium coupled to the hardware processors. The program instructions may implement the functionality described herein (e.g., the functionality of various hardware servers and other components that implement the network-based cloud and non-cloud computing resources described herein). The various methods as illustrated in the figures and described herein represent example implementations.Attorney Docket No. RAM P-00316-WOThe order of any method may be changed, and various elements may be added, modified, or omitted.

[0066] Fig. 6 is a block diagram illustrating an example computer hardware system, according to various embodiments. Computer system 140 may implement a hardware portion of a cloud computing system as forming parts of the various implementations of the present invention. Computer system 140 may be any of various types of hardware devices, including, but not limited to, a commodity server, personal computer system, desktop computer, laptop or notebook computer, mainframe computer system, handheld computer, workstation, network computer, a consumer device, application server, physical storage device, telephone, mobile telephone, or in general any type of computing node, compute node, compute device, and / or hardware computing device.

[0067] Computer system 140 includes one or more hardware processors 140a, 141b...l41n (any of which may include multiple processing cores, which may be single or multi-threaded) coupled to a physical system memory 142 via an input / output (I / O) interface 144. Computer system 140 further may include a network interface 146 coupled to I / O interface 144. In various embodiments, computer system 140 may be a single processor system including one hardware processor 141a, or a multiprocessor system including multiple hardware processors 141a, 141b...l41n as illustrated in Fig. 6.

[0068] Processors 141a, etc. may be any suitable processors capable of executing computing instructions. For example, in various embodiments, processors 141a, etc. may be general- purpose or embedded processors implementing any of a variety of instruction set architectures. In multiprocessor systems, each of processors 141a, etc. may commonly, but not necessarily, implement the same instruction set. The computer system 140 also includes one or more hardware network communication devices (e.g., network interface 146) for communicating with other systems and / or components over a communications network, such as a local area network, wide area network, or the Internet. For example, a client application executing on system 140 may use network interface 146 to communicate with a server application executing on a single hardware server or on a cluster of hardware servers that implement one or more of the components of the systems described herein in a cloud computing environment as implemented in various sub-systems. In another example, an instance of a server application executing on computer system 140 may use network interface 146 to communicate with other instances of an application that may be implemented on other computer systems.Attorney Docket No. RAM P-00316-WO

[0069] In the illustrated embodiment, computer system 140 also includes one or more physical persistent storage devices 148 and / or one or more I / O devices 150. In various embodiments, persistent storage devices 148 may correspond to disk drives, tape drives, solid-state memory or drives, other mass storage devices, or any other persistent storage devices. Computer system 140 (or a distributed application or operating system operating thereon) may store instructions and / or data in persistent storage devices 148, as desired, and may retrieve the stored instructions and / or data as needed. For example, in some embodiments, computer system 140 may implement one or more nodes of a control plane or control system, and persistent storage 148 may include the solid-state drives (SSDs) attached to that server node. Multiple computer systems 140 may share the same persistent storage devices 148 or may share a pool of persistent storage devices, with the devices in the pool representing the same or different storage technologies, including such technologies as described above.

[0070] Computer system 140 includes one or more physical system memories 142 that may store code / instructions 143 and data 145 accessible by processor(s) 141a, etc. The system memories 142 may include multiple levels of memory and memory caches in a system designed to swap information in memories based on access speed, for example. The interleaving and swapping may extend to persistent storage devices 148 in a virtual memory implementation, where memory space is mapped onto the persistent storage devices 148. The technologies used to implement the system memories 142 may include, by way of example, static randomaccess memory (RAM), dynamic RAM, read-only memory (ROM), non-volatile memory, solid- state memory, or flash-type memory.

[0071] As with persistent storage devices 148, multiple computer systems 140 may share the same system memory systems 142 or may share a pool of system memories 142. System memory or memory systems 142 may contain program instructions 143 that are executable by processor(s) 141a, etc. to implement the routines described herein.

[0072] In various embodiments, program instructions 143 may be encoded in binary, Assembly language, any interpreted language such as Java, compiled languages such as C / C++, or in any combination thereof; the particular languages given here are only examples. In some embodiments, program instructions 143 may implement multiple separate clients, server nodes, and / or other components.

[0073] In some implementations, program instructions 143 may include instructions executable to implement an operating system (not shown), which may be any of various operating systems,Attorney Docket No. RAM P-00316-WOsuch as UNIX, LINUX, Solaris™, MacOS™, or Microsoft Windows™. Any or all of program instructions 143 may be provided as a computer program product, or software, that may include a non-transitory computer-readable storage medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to various implementations. A non-transitory computer-readable storage medium may include any mechanism for storing information in a form (e.g., software or processing application) readable by a machine (e.g., a physical computer).

[0074] Generally speaking, a non-transitory computer-accessible medium may include computer-readable storage media or memory media such as magnetic or optical media, e.g., disk or DVD / CD-ROM, coupled to or in communication with computer system 140 via I / O interface 144. A non-transitory computer-readable storage medium may also include any volatile or non-volatile media such as RAM or ROM that may be included in some embodiments of computer system 140 as system memory 142 or another type of memory. In other implementations, program instructions may be communicated using optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.) conveyed via a communication medium such as a network and / or a wired or wireless link, such as may be implemented via network interface 606. Network interface 146 may be used to interface with other devices 142, which may include other computer systems or any type of external electronic device.

[0075] In some embodiments, system memory 142 may include data store 145, as described herein. In general, system memory 142 and persistent storage 148 may be accessible on other devices 142 through a network and may store data blocks, replicas of data blocks, metadata associated with data blocks, and / or their state, database configuration information, and / or any other information usable in implementing the routines described herein.

[0076] In one embodiment, I / O interface 144 may coordinate I / O traffic between processors 141a, etc., system memory 142, and any peripheral devices in the system, including through network interface 146 or other peripheral interfaces. In some embodiments, I / O interface 144 may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory 142) into a format suitable for use by another component (e.g., processors 141a, etc.).

[0077] In some embodiments, I / O interface 144 may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral ComponentAttorney Docket No. RAM P-00316-WOInterconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, as examples. Also, in some embodiments, some or all of the functionality of I / O interface 144, such as an interface to system memory 142, may be incorporated directly into processor(s) 141a, etc.

[0078] Network interface 146 may allow data to be exchanged between computer system 140 and other devices attached to a network, such as other computer systems (which may implement one or more storage system server nodes, primary nodes, read-only node nodes, and / or clients of the database systems described herein), for example. In addition, I / O interface 144 may allow communication between computer system 140 and various I / O devices 150 and / or remote storage 148. Input / output devices 150 may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for entering or retrieving data by one or more computer systems 140. These may connect directly to a particular computer system 140 or generally connect to multiple computer systems 140 in a cloud computing environment, grid computing environment, or other system involving multiple computer systems 140.

[0079] Multiple input / output devices 150 may be present in communication with computer system 140 or may be distributed on various nodes of a distributed system that includes computer system 140. In some embodiments, similar input / output devices may be separate from computer system 140 and may interact with one or more nodes of a distributed system that includes computer system 140 through a wired or wireless connection, such as over network interface 146. Network interface 146 may commonly support one or more wireless networking protocols (e.g., Wi-Fi / I EEE 802.11, or another wireless networking standard).

[0080] Network interface 146 may support communication via any suitable wired or wireless general data networks, such as other types of Ethernet networks, for example. Additionally, network interface 146 may support communication via telecommunications / telephony networks such as analog voice networks or digital fiber communications networks, via storage area networks such as Fibre Channel SANs, or via any other suitable type of network and / or protocol. In various embodiments, computer system 140 may include more, fewer, or different components than those illustrated in Fig. 2 (e.g., displays, video cards, audio cards, peripheral devices, or an Ethernet interface).

[0081] Any of the distributed system embodiments described herein, or any of their components, may be implemented as one or more network-based services in the cloud computing environment. For example, a read-write node and / or read-only nodes within theAttorney Docket No. RAM P-00316-WOdatabase tier of a hardware database system may present database services and / or other types of physical data storage services that employ the distributed storage systems described herein to clients as network-based services.

[0082] In some embodiments, a network-based service may be implemented by a software and / or hardware system designed to support interoperable machine-to-machine interaction over a network. A web service may have an interface described in a machine-processable format. Other systems may interact with the network-based service in a manner prescribed by the description of the network-based service's interface. For example, the network-based service may define various operations that other systems may invoke, and may define a particular application programming interface (API) to which other systems may be expected to conform when requesting the various operations.

[0083] In various embodiments, a network-based service may be requested or invoked through the use of a message that includes parameters and / or data associated with the network-based services request. Such a message may be formatted according to a particular markup language such as Extensible Markup Language (XML), and / or may be encapsulated using a protocol. To perform a network-based services request, a network-based services client may assemble a message including the request and convey the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the web service, using an Internet-based application layer transfer protocol such as Hypertext Transfer Protocol (HTTP).

[0084] Unless otherwise stated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0085] Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of the exemplary methods and materials are described herein. It will be apparent to those skilled in the art that many more modifications are possible without departing from the inventive concepts herein.

[0086] All terms used herein should be interpreted in the broadest possible manner consistent with the context.

[0087] When a grouping is used herein, all individual members of the group and all combinations and sub-combinations possible of the group are intended to be individually included.Attorney Docket No. RAM P-00316-WO

[0088] When a range is stated herein, the range is intended to include all sub-ranges within the range, as well as all individual points within the range.

[0089] When "about," "approximately," or like terms are used herein, they are intended to include amounts, measurements, or the like that do not depart significantly from the expressly stated amount, measurement, or the like, such that the stated purpose of the apparatus or process is not lost.

[0090] All references cited herein are hereby incorporated by reference to the extent that there is no inconsistency with the disclosure of this specification.

[0091] The present invention has been described with reference to certain preferred and alternative embodiments that are intended to be exemplary only and not limiting to the full scope of the present invention, as set forth in the appended claims.

Claims

Attorney Docket No. RAM P-00316-WOClaims1. A system for real-time contextual message targeting using vector embeddings, comprising: a deep learning model configured to process user interaction data and generate highdimensional vector embeddings representing user interests without storing personally identifiable information;a vector database storing said vector embeddings associated with privacy-preserving user identifiers; anda similarity search engine configured to perform nearest neighbor searches in the vector database to match vector embeddings with a set of campaign embeddings;wherein the system enables targeted message delivery without relying on third-party identifiers.

2. The system of claim 1, wherein the deep learning model comprises:a temporal graph builder that constructs heterogeneous graphs from user event data;a relational graph transformer with temporal and relation-aware attention mechanisms; and a readout mechanism that combines user sub-graphs with recent edge context to generate the vector embeddings.

3. The system of claim 2, wherein the relational graph transformer employs a two-layer architecture with eight attention heads and 128-dimensional hidden layers..

4. The system of claim 1, wherein the vector embeddings are stored in first-party cookies or at a content delivery network (CDN) layer within first-party domains.

5. The system of claim 1, further comprising a campaign vector database storing embeddings representing messaging campaigns, wherein the similarity search engine matches user embeddings to campaign embeddings to determine message relevance.Attorney Docket No. RAM P-00316-WO6. The system of claim 1, wherein the system operates within real-time bidding environments with processing latency of 100 milliseconds or less for complete round-trip operations including embedding generation, vector search, and response delivery.

7. The system of claim 1, wherein the deep learning model is trained using multi-task learning to simultaneously optimize for interest prediction, message relevance, and temporal dynamics.

8. The system of claim 1, wherein the vector embeddings have 128 dimensions and are updated in real-time based on user interaction feedback.

9. A method for real-time contextual message targeting using vector embeddings, comprising: receiving user interaction data from digital touchpoints;processing the user interaction data through a deep learning model to generate highdimensional vector embeddings representing user interests;storing the vector embeddings in a vector database without persistent personally identifiable information;receiving a message targeting request;performing a nearest neighbor search in the vector database to identify users whose embeddings match a set of campaign criteria; anddelivering targeted messages based on vector similarity scores.

10. The method of claim 9, further comprising the steps of:constructing temporal graphs from the user interaction data that capture relationships between users, content, time, and context; andprocessing the temporal graphs through a relational graph transformer to generate the vector embeddings.Attorney Docket No. RAM P-00316-WO11. The method of claim 9, wherein the processing step includes enriching raw event streams with contextual information using large language models before embedding generation.

12. The method of claim 9, further comprising continuously updating the vector embeddings based on real-time user interaction feedback to maintain current interest representations.

13. The method of claim 9, wherein the nearest neighbor search uses distance measures selected from the group consisting of Euclidean distance, cosine similarity, and Manhattan distance.

14. The method of claim 9, further comprising implementing frequency capping by updating user embeddings based on message interaction signals such that updated embeddings produce lower relevance scores for previously shown content.

15. The method of claim 9, wherein the deep learning model utilizes node types including user, item, click, campaign, query, device, and IP address with corresponding edge relationships in the temporal graphs.

16. A computer-implemented system for privacy-preserving message targeting, comprising:one or more processors;memory storing instructions that, when executed by the processors, cause the system to:train a deep learning model on anonymized user interaction data to generate vector embeddings representing user interests;store the vector embeddings in a high-performance vector database optimized for nearest neighbor searches;receive targeting requests and perform vector similarity searches to identify relevant users; anddeliver personalized messages based on embedding proximity without exposing underlying user identifiers.Attorney Docket No. RAM P-00316-WO17. The system of claim 16, wherein the instructions further cause the system to:apply differential privacy techniques during model training to prevent memorization of individual user data; andincorporate fairness constraints to mitigate potential biases in interest representation across different user groups.

18. The system of claim 16, wherein the vector database supports indexing methods for sub-second retrieval of nearest neighbors across large campaign databases.

19. The system of claim 16, wherein the system performs embedding joins to create stable first- party identifier mappings between different datasets using algorithmic approaches including baseline similarity, metric-learning models, and locality-sensitive hashing with community detection.

20. The system of claim 16, wherein the deep learning model employs federated learning techniques for continuous updates while maintaining user privacy, and the system distributes computational workloads to edge computing infrastructure for improved latency and privacy.

21. A method for real-time message targeting in a programmatic advertising system, comprising:receiving user interaction data from a publisher server;processing the user interaction data through a real-time interests model to generate user context embeddings;storing the user context embeddings in an interests-based message inventory vector database;receiving product description and creative data from a messenger service;Attorney Docket No. RAM P-00316-WOprocessing the product description and creative data through a messenger-to-interests model to generate campaign embeddings;performing nearest-neighbor vector search algorithms using the vector database to match user interests to available product description and creatives inventory;executing bidding optimization at a bidder considering multiple variables including budget constraints and reinforcement learning signals through interests vector search;processing the matching through a matching process using first-party identifiers and context data; andmerging embeddings through an embedding merge linking the real-time interests model with first-party identifiers and context.

22. The method of claim 21, further comprising the step of continuously updating user embeddings based on real-time interaction data.

23. The method of claim 22, wherein the nearest-neighbor vector search algorithms compare user context embeddings against available campaign vectors to determine relevance of products or services to user intent based on vector distance measurements.

24. The method of claim 22, further comprising:returning a pre-agreed upon DeallD as a result of the vector search, wherein the DeallD represents a private marketplace deal between a messenger and publisher;registering DeallDs with a supply-side platform (SSP) and a demand-side platform (DSP) through user-initiated activation;sending the DeallD along with real-time bidding information from the SSP to the DSP; andAttorney Docket No. RAM P-00316-WOdetermining a winning bid through standard real-time bidding protocols using the DeallD.

25. The method of claim 24, wherein the method further comprises the steps of:generating or updating a user embedding;sending a request with the user embedding from the real-time interests model to the SSP;passing the embedding to interests vector search which executes a nearest neighbor search against the vector database;returning the corresponding DeallD to the SSP;displaying the corresponding message to the user; andgenerating or updating the user embedding again after the message is displayed to the user.

Images