Particle signals for offline-to-online modeling

Abstract

An example aspect of embodiments of the present disclosure provides an example computer-implemented method. The example method includes receiving source activity data. The example method includes performing a query on target activity related to the source activity data. In the example method, performing the query includes determining predicted target activity related to the source activity data using a first machine learning model of a machine learning model framework. In the example method, performing the query includes generating a predicted temporal distribution of target activity using a second machine learning model of the machine learning model framework. The example method includes outputting a query result responsive to the query based at least in part on the predicted target activity and the predicted temporal distribution of target activity.

Description

Technical Field

[0001] This disclosure generally relates to generating associations using machine learning models. More specifically, example aspects of this disclosure relate to generating associations between online and offline activities using a machine learning model framework. Background Technology

[0002] Some service providers may offer cross-platform services to users or customers. In some examples, cross-platform services may include services provided on a service pathway (e.g., an online system) that has limited communication with another service pathway (e.g., another system, a physical service exit, etc.). To provide improved cross-platform services, it may be desirable to associate data records of activities in one pathway with activities in another pathway. Summary of the Invention

[0003] Aspects and advantages of embodiments of this disclosure will be set forth in part in the description which follows, or may be learned from the description or by practice of the embodiments.

[0004] In one example aspect, this disclosure provides an example computer-implemented method. The example method includes receiving tagged records by a computing system including one or more processors. In the example method, the tagged records include linked source activities and linked target activities. The example method includes updating one or more parameters of a first machine learning model using the tagged records, the first machine learning model being configured to output data describing target activities associated with source activities. The example method includes updating one or more parameters of a second machine learning model using the tagged records, the second machine learning model being configured to output a distribution of target activities over time.

[0005] In one example aspect, this disclosure provides another example computer-implemented method. The example method includes receiving source activity data by a computing system including one or more processors. The example method includes the computing system executing a query for a target activity related to the source activity data. In the example method, executing the query includes the computing system using a first machine learning model within a machine learning model framework to determine a predicted target activity related to the source activity data. In the example method, executing the query includes the computing system using a second machine learning model within the machine learning model framework to generate a predicted temporal distribution of the target activity. The example method includes the computing system generating query results in response to the query, at least in part based on the predicted target activity and the predicted temporal distribution of the target activity.

[0006] In one example aspect, this disclosure provides an example system. The example system includes one or more processors and one or more memory devices storing computer-readable instructions. In the example system, the instructions, when executed, cause the one or more processors to perform operations. In the example system, the operations include receiving source activity data. In the example system, the operations include performing a query on a target activity associated with the source activity data. In the example system, performing the query includes determining a predicted target activity associated with the source activity data using a first machine learning model within a machine learning model framework. In the example system, performing the query includes generating a predicted temporal distribution of the target activity using a second machine learning model within the machine learning model framework. In the example system, the operations include generating a query result by a computing system in response to the query, at least in part based on the predicted target activity and the predicted temporal distribution of the target activity.

[0007] In one example aspect, this disclosure provides an example computer-readable medium storing computer-readable instructions that, when executed, cause one or more processors to perform operations. In the example computer-readable medium, the operations include receiving source activity data. In the example computer-readable medium, the operations include performing a query on a target activity related to the source activity data. In the example computer-readable medium, performing the query includes determining a predicted target activity related to the source activity data using a first machine learning model within a machine learning model framework. In the example computer-readable medium, performing the query includes generating a predicted temporal distribution of the target activity using a second machine learning model within the machine learning model framework. In the example computer-readable medium, the operations include generating a query result by a computing system in response to the query, at least in part based on the predicted target activity and the predicted temporal distribution of the target activity.

[0008] These and other features, aspects, and advantages of the various embodiments of this disclosure will be better understood with reference to the following description and the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and, together with the description, serve to explain the relevant principles. Attached Figure Description

[0009] A detailed discussion of embodiments for those skilled in the art is set forth in the specification, with reference to the accompanying drawings, in which:

[0010] Figure 1 An example system for processing queries on a dataset is described according to example aspects of this disclosure;

[0011] Figure 2 An exemplary embodiment of a query processing subsystem for processing queries on a dataset, according to an example aspect of this disclosure, is described;

[0012] Figure 3An example processing system for processing queries on a dataset is described according to example aspects of this disclosure;

[0013] Figure 4 A flowchart describing an example method for processing queries on a dataset according to an example aspect of this disclosure;

[0014] Figure 5 A flowchart describing another example method for processing queries on a dataset according to an example aspect of this disclosure; and

[0015] Figure 6 A flowchart is provided describing an example method for training a system for processing queries on a dataset, based on example aspects of this disclosure.

[0016] The reference numbers repeated on multiple graphs are intended to identify the same features in different implementations. Detailed Implementation

[0017] Overview

[0018] Generally, this disclosure relates to techniques for associating activity data from a source platform with a target activity through a means other than the source platform. In some embodiments, the source activity data may describe interactions with content on a source computing system (e.g., a system hosting content from a service provider). In some embodiments, the target activity may be related to the subject matter of the source activity data, but in some scenarios, the source platform may not have visibility into the target activity used to determine the relationship. For example, the source activity may include interactions with an online service, while the target activity may include accessing a physical location associated with the online service—in some cases, the online platform may not communicate with the physical location indicating information about any access.

[0019] Advantageously, the systems and methods according to the example aspects of this disclosure enable, for example, service providers to perform queries on source activity data to obtain data describing a target activity associated with (e.g., originating from, responding to, etc.) the source activity. For example, in some embodiments, a service provider associated with an online platform in the above-described scenario can perform queries on activity data on the online platform to obtain data describing access to a physical location. The obtained query results may include a set of conversion label information (e.g., indicating that the source activity has been "converted" to the target activity, etc.). In some examples, the set of conversion label information may correspond to a distribution over time. For example, the distribution over time may indicate a "hysteresis" between the source activity and any subsequent conversion to the target activity. In some examples, each hysteresis value (e.g., sampled from the output distribution) corresponds to the probability that the target activity has that hysteresis value. In this way, for example, the systems and methods according to the example aspects of this disclosure can allow queries for relevant target activities to be processed on the set of source activity data (e.g., when such query results might otherwise be unavailable).

[0020] In some embodiments, the systems and methods according to the example aspects of this disclosure may use a machine learning model framework to process queries on source activity data to achieve the aforementioned advantages. In some embodiments, the systems and methods according to the example aspects of this disclosure utilize a dual-model structure in the machine learning model framework. For example, the machine learning model framework may include a transformation model and a lag model. Each model may be trained on a set of labeled records (such as source activity data and lag times of subsequent transformations, labeled or otherwise linked to correspond to each other). In this way, for example, the machine learning model framework may use labeled (e.g., tagged) data to learn to use the transformation model to obtain target activity label information on a set of unlabeled source activity data, use the lag model to obtain the lag times of any subsequent target activities, and thereby obtain the results of queries on relevant target activities on the set of source activity data (e.g., otherwise, when such query results might not be available).

[0021] The example systems and methods based on exemplary aspects of this disclosure can provide various technical effects and benefits. For example, in some embodiments, the example systems and methods can enable the processing of queries for relevant events in data-constrained contexts (where there would otherwise be insufficient data to return usable query results). Query processing can be enabled, for example, by using a machine learning model framework of this disclosure to index one or more source activity events corresponding to one or more target activity events based on one or more parameters of the machine learning model framework.

[0022] In some embodiments, by employing a machine learning model framework to obtain transformed and lagged results on the source activity dataset (e.g., even when direct target activity mapping is typically unavailable), the example systems and methods can allow query results to be obtained with less time, effort, and / or cost (e.g., computational costs). For example, by learning a set of parameters for a model framework for machine learning on a smaller, known, indexed dataset, and using the learned set of parameters to determine the correlation between unindexed source activity inputs and target activity outputs, the example systems and methods can scale the capabilities of database processing systems to determine relationships between activity data (e.g., between unindexed online and offline activities).

[0023] In some embodiments, example systems and methods according to exemplary aspects of this disclosure can provide improved storage, management, retrieval, and cross-referencing of data structures in memory (e.g., in a database). For example, an example database may contain real-world data structures describing various unlabeled source activity instances. An example database (or another database) may also contain data structures describing labeled activity data instances. Based on the labeled data instances, an example computational system according to this disclosure can learn an intermediate set of data structures (e.g., a set of learned parameters for a machine learning model framework) to map unlabeled source activity instances to subsequent target activities. Although the intermediate set of data structures is not necessarily interpretable to human observers (e.g., can be understood as identifiable representations of underlying real data), the intermediate set of data structures is operable to enable a computational system executing a machine learning model to learn to correlate the set of unlabeled source activity instances with target activity label information and / or the temporal distribution of the target activities. In this way, for example, unindexed and / or unclassified source activity data that otherwise has no structured relationship with the target activity data can be advantageously queried by the systems and methods according to this disclosure. Furthermore, in some embodiments, for example, an intermediate set of data structures can be used to provide associations between untagged source activity instance data structures in the database and one or more transformation tags, enabling improved storage and / or retrieval of those data structures (e.g., indexed storage based on one or more tags, retrieval based on one or more tags, etc.).

[0024] In some examples, the dual-model framework of exemplary embodiments of this disclosure can particularly provide improved storage, management, retrieval, and cross-referencing of data structures in memory (e.g., in a database). For example, by utilizing two different models for transformation labeling and lag determination, the machine learning model framework can achieve modularity that allows for more options for reconfiguration, maintenance, troubleshooting, and updating of the query processing system of this disclosure. Furthermore, the query processing system of this disclosure utilizing the dual-model framework can be configured on-the-fly by replacing either or both of the transformation model or the lag model with an alternative model trained for one or more specific tasks. For example, lags may be optimally determined in some seasons differently than they are in other seasons. Thus, the dual-model framework provides a highly configurable query processing system for processing queries on unlabeled source activity data in a database.

[0025] In some embodiments, for example, an intermediate set of data structures can be used to provide processing and execution of queries on an unlabeled set of source activity instances. For example, a query may include a query to obtain a predicted temporal distribution of a target activity associated with an input set of source activities. Unlabeled source activity instances may not include any values ​​or labels recording links to the target activity. However, it is advantageous that the intermediate set of data structures can map input source activity data to an output data structure containing a query object: the temporal distribution of the target activity associated with the input set of source activities. In this way, for example, example systems and methods according to aspects of this disclosure can provide execution and processing of queries on an input dataset, even when such queries might otherwise be unavailable (e.g., due to data scarcity or communication-constrained implementations).

[0026] In some embodiments, example systems and methods according to example aspects of this disclosure can provide for determining correlations between a set of unlabeled source activity data. For example, correlations can be determined along dimensions in the source activity data that are unlabeled or have incomplete labels. For example, temporal correlations can be determined on source activity data even when the source activity data may lack complete (or any) labels of temporal relationships. For example, in some embodiments, the source activity data may include timestamps associated with the source activity (e.g., date, time, date and time, etc.), but the source activity data may lack timestamps of any subsequent target activities. Advantageously, example systems and methods according to example aspects of this disclosure can provide for determining correlations between source activity data based on a temporal dimension of any subsequent target data (e.g., for which labeling information may be missing). For example, example systems and methods according to example aspects of this disclosure can provide for determining temporal relationships within a time window (e.g., data describing all source activity instances identified as having been transformed within a given time window, such as counts, etc.).

[0027] As illustrated herein, the example systems and methods of this disclosure provide improvements to data storage, indexing, query processing, and result retrieval, which in turn improve the ability of computational systems to correlate data structures (e.g., by indexing previously unlabeled real data for querying), increase computational efficiency (e.g., by returning fewer empty query results due to unlabeled data), and reduce computational costs (e.g., by predicting label information for unlabeled data instead of requiring additional data collection), in some instances.

[0028] Example systems and methods

[0029] Exemplary embodiments of this disclosure will now be discussed in more detail with reference to the accompanying drawings. Figure 1 This description includes an example system 100 for processing example queries according to example aspects of this disclosure. Example system 100 includes a computing system 102. Computing system 102 can be any type of system containing one or more computing devices. Computing devices can be, for example, personal computing devices (e.g., laptops or desktops), mobile computing devices (e.g., smartphones or tablets), game consoles or controllers, wearable computing devices, embedded computing devices, server computing devices, nodes of distributed computing devices, virtual instances hosted on a shared server, or any other type of computing device. In some embodiments, computing system 102 includes multiple computing devices interconnected via a network or distributed in an interoperable manner. For example, computing system 102 may include a server for serving content over a network (e.g., network 180). For example, computing system 102 may include a web server for hosting web content and for collecting data about the web content (e.g., for receiving, monitoring, generating, or otherwise processing data about the web content, such as using, downloading, and / or interacting with the web content).

[0030] The computing system 102 may include a processor 112 and a memory 114. One or more processors 112 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be one or more processors operatively connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 executed by the processor 112 to cause the computing system 102 to perform operations.

[0031] In some implementations, the client computing system 102 may store or otherwise implement one or more machine learning models that execute a machine learning model framework. In some embodiments, the query processing subsystem 120 includes a dual-model machine learning model framework. The machine learning model framework may include a machine learning transformation model 122 (e.g., with learnable weights 124) and a machine learning lag model 126 (e.g., with learnable weights 128). The one or more machine learning models may be or may additionally include various machine learning models, such as neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear and / or linear models. Neural networks may include feedforward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some exemplary machine learning models may include multi-head self-attention models (e.g., transformer models).

[0032] like Figure 1 As shown, an embodiment of example system 100 can be configured to process a query 130 on a source activity 140 (e.g., containing source activity event 142) regarding a target activity 150 (e.g., containing target activity event 152). In response to processing query 130, computing system 102 can output query result 160. In some embodiments, query result 160 may include transformed data 162 (e.g., output from transformation model 122 according to weight 124) and lag distribution 164 (e.g., output from lag model 126 according to weight 128). Query 130 can be processed using tagged records 170 containing linked source activity events 142' and linked target activity events 152'.

[0033] In some embodiments, query 130 is input to computing system 102. For example, in some embodiments, query processing subsystem 120 may be configured to process various queries 130 as input to computing system 102. In some embodiments, query 130 may be implicit in the structure and / or configuration of query processing subsystem 120. For example, query processing subsystem 120 may be configured to generate a response to a query describing the predicted temporal distribution of a target activity. For example, query processing subsystem 120 may be configured to generate a response to a query describing the timing of a subsequent target activity 150 and relating to source activity 140.

[0034] In some embodiments, source activity 140 includes unlabeled source activity events 142. Unlabeled source activity events 142 may include instances of virtually any kind or type of data that can describe various phenomena. Typically, an instance refers to a set of one or more data values ​​that are combined to describe a particular subject or topic. For example, an instance may be a feature vector. Instances may be associated with image data (e.g., feature vectors of an image, hash images, etc.). Instances may be associated with measurement or other data collection events (e.g., at a specific time, or by a specific subject, or using a specific device, or from a specific angle, etc.). In some examples, instances of unlabeled source activity events may indicate communication from a source entity (e.g., a message in a content item).

[0035] An instance can be associated with a network session, such as a set of interactions with a web server. In some embodiments, an instance can be associated with a user's interaction with web content (e.g., anonymous or identified). In some embodiments, unlabeled source activity events 142 may not contain tagging information for the instance. In some embodiments, unlabeled source activity events 142 may contain some tagging information, but may lack other tagging information. For example, unlabeled source activity events 142 may lack tagging determinism or otherwise responding to queries expected to be processed on the set of unlabeled source activity events 142.

[0036] For example, in some embodiments, source activity event 142 may include data describing the source platform and / or content items on the platform that facilitate the event. Source activity event 142 may include data describing the user and their role in the event, such as how long the user interacted with the platform and / or content items, what actions the user took on the platform (e.g., what content they viewed, such as using a content ID, adding any products to their cart, and / or purchasing any products), shipping location information, known or estimated user demographics, etc. In some embodiments, source activity event 142 may include timing data. For example, timing data may include the date of the event (e.g., day, month, year, etc.), the duration of the event, local calendar characteristics (e.g., holidays, seasons, traditional and / or expected behavioral patterns for various days / dates, etc.). In some examples, timing data may be input into all models of query processing subsystem 120 (e.g., both transformation model 122 and lag model 126). In some examples, one or more elements of timing data are input only into lag model 126.

[0037] In some embodiments, source activity event 142 may be anonymized or otherwise obfuscated within source activity 140. For example, source activity 140 may be reported to computing system 102 in an aggregated manner (e.g., as a set of aggregated events).

[0038] In some embodiments, the target activity event 152 may include instances of data of virtually any kind or type that can describe a variety of phenomena. Generally, an instance refers to a set of one or more data values ​​that are combined to describe a particular subject or topic. For example, an instance may be a feature vector. Instances may be associated with image data (e.g., feature vectors of images, hash images, etc.). Instances may be associated with measurement or other data collection events (e.g., at a particular time, by a particular subject, using a particular device, or from a particular angle, etc.). Instances may be associated with a network session, such as a set of interactions with a web server. In some embodiments, instances may be associated with user interactions with web content (e.g., anonymous or identified). In some examples, instances of the target activity event may indicate the reception and / or response to communication from a source entity (e.g., messages in content items). Thus, in some embodiments, determining the target activity associated with source activity data may include estimating the read / view status of communication from the source entity. Similarly, in some embodiments, determining the target activity associated with source activity data may include determining the recipient of communication from the source entity (e.g., "A store visitor on day X is a likely recipient of online communication Y," etc.). In this way, for example, source activities and target activities may include communication between the source platform and the user.

[0039] In some embodiments, the target activity event 152 is not configured for transmission 154 to the computing system 102. For example, the target activity event 152 may be an event not captured by data available to the computing system 102. The target activity event 152 may include access to or other interactions with a physical location associated with a service provider. The target activity event 152 may include user interactions with systems (e.g., other than the source platform) that are not configured to transmit information through one or more boundaries 156 (e.g., hardware boundaries, such as network boundaries, firewalls, etc., and / or policy boundaries, such as data sharing policies, data storage policies, etc.).

[0040] Query result 160 may include, for example, transformed data 162 and lag distribution 164, or data based on transformed data 162 and lag distribution 164. Transformed data 162 may include a set of target activities (e.g., a count of positive transformation labels). In some examples, transformed data 162 includes data describing one or more features extracted (e.g., floated) from input source activity 140 that are associated with a positive transformation (e.g., a higher probability of a positive transformation). In this way, for example, one or more queries 130 may be executed on source activity 140 to obtain data associated with a positive transformation.

[0041] The lag distribution 164 may include the distribution of the target activity event 152 over time. The time of the target activity event 152 may, in some cases, be "lagged" by the time of the source activity event 142 (from which the source activity event 142 originates). The lag distribution 164 may include a list of estimated lag times of the target activity indicated in the transformed data 162. In some embodiments, the distribution 164 may include, for example, a histogram. An example histogram of the target activity event 152 may include multiple bins corresponding to timing data (e.g., event date, event time, etc.) and the probability associated with the occurrence of the target activity event 152 within that bin. In some embodiments, the bins may be associated with an estimated count of the bins.

[0042] In some embodiments, the lag distribution 164 can be sampled from the intermediate outputs of the query processing system 120. For example, in some embodiments, the transformation model 122 outputs predicted target activities (e.g., multiple transformations from source activities to target activities), the lag model 126 outputs the distribution of the probability of the target activities over multiple days, and both outputs can be used to sample to obtain the lag distribution 164.

[0043] For example, in some embodiments, the lag distribution 164 associates the transformed data 162 with multiple time periods. For example, the lag distribution 164 may include a data structure containing entries for the multiple time periods and entries of the transformed data 162 that the query processing subsystem 120 has already associated with the corresponding time periods of the multiple time periods.

[0044] In some embodiments, the query processing subsystem 120 may be configured to process query 130 using tagged records 170. Tagged records 170 may include, for example, linked source activity events 142' and linked target activity events 152' (e.g., events linked to each other respectively). Source activity events 142' may include source activity events as discussed above with respect to source activity events 142. Furthermore, source activity events 142' may contain data (e.g., using tags, using one or more structural features of a data structure, etc.) that links them to target activity events 152'.

[0045] In some embodiments, the tagged record 170 may include a subset of source activity 140 and target activity 150. For example, source activity 140 may include source activity event 142 and linked source activity event 142'. For example, a source platform may provide services to multiple users, providing only a subset of users with any indication that a user is participating in the corresponding target activity. For example, in some embodiments, a set of logged-in users (e.g., users associated with accounts, such as service provider accounts, source platforms, content on source platforms, etc.) may provide more information about unlogged-in users, and logged-in users may provide information about the source activity and the corresponding target activity (e.g., both associated with the user's user ID). In some embodiments, the user ID may be associated with a third-party system (e.g., a system different from the source platform system, a party other than the publisher of the source content, etc.).

[0046] In some embodiments, the query processing subsystem 120 may include a machine learning model framework trained using labeled records 170. For example, the labeled records 170 may be used to learn (e.g., set, update, adjust, tune, etc.) the weights 124 and 128 of the transformation model 122 and the lag model 126, respectively.

[0047] For example, Figure 2 An example embodiment of a query processing subsystem 120 describing the data flow of inference (solid line) and model update (dashed line) is described. Data storage 202 may contain source activities 140 and labeled records 170. During training, query processing subsystem 120 may execute queries (e.g., query 130) on the labeled records 170 to obtain query results 160. For example, transformation model 122 may use weights 124 (e.g., transformed data 162) to generate target activity data. Lag model 126 may use weights 128 to generate lagged data (e.g., lagged distribution 164). In some embodiments, the outputs of transformation model 122 and lagged model 124 are sampled by sampler 220 (e.g., random sampling, uniform sampling, etc.) to provide query results 160 (e.g., containing transformed data 162 and lagged distribution 164).

[0048] The query result 160 can be used to update one or more of weights 124 and 128. For example, evaluator 222 can evaluate output 160, such as determining whether the output query result 160 is aligned with the labeled record 170. For example, evaluator 222 can determine an objective (e.g., loss, cost, score, etc.) based on query result 160. For example, evaluator 222 can compare query result 160 with the labeled record 170, for example, by comparing one or more predictive lag distributions with one or more measurement lag times (e.g., by determining one or more measurement time delays between the linked source activity event 142' and the linked target activity event 152', and comparing them with the predictive time delay of the labeled record 170) and one or more measurement counts of the target activity. In this way, for example, some embodiments may use the linked target activity event 152' to form "ground truth" reference data for evaluating predictive data describing the target activity event.

[0049] In some embodiments, the evaluator determines a conversion model evaluation 223 and a lag model evaluation 224. In some embodiments, the conversion model evaluation 223 and the lag model evaluation 224 are the same evaluation (e.g., included or incorporated into the target, loss, cost, score, etc. of the conversion model 122 and the lag model 126). In some embodiments, the conversion model evaluation 223 and the lag model evaluation 224 include different evaluations of each of the conversion model 122 and the lag model 126. For example, in some embodiments, the conversion model evaluation 223 may be based on the count and / or rate of predicted target activity output by the conversion model 122 (e.g., directly from the output of the conversion model 122 and / or sampled via sampler 220, as shown by various options in the dashed lines in the figure). In some embodiments, the rate of predicted target activity may be determined relative to the total amount of source activity (e.g., the ratio of target activity to the total amount of source activity, such as "conversion rate," etc.). For example, in some embodiments, the hysteresis model evaluation 224 may be based on the distribution of probabilities output by the hysteresis model 126 (e.g., directly from the output of the hysteresis model 126 and / or sampled via sampler 220, as shown by the various options in the dashed lines in the figure).

[0050] Based at least in part on evaluator 222, model updater 226 can update one or more parameters of weights 124 and / or weights 128. For example, model updater 280 may include or perform virtually any model update technique, such as gradient-based methods, evolutionary methods, etc. In some embodiments, model updater 226 updates weights 124 and weights 128 together. In some embodiments, model updater 226 implements transformation model update 227 and lag model update 228. In some embodiments, transformation model update 227 and lag model update 228 are implemented independently (e.g., optionally together, but optionally with different evaluations, different configurations of model update techniques, such as using different hyperparameters, etc.).

[0051] In some embodiments, the evaluator 222 and the model updater 226 are included within the computing system 102. In some embodiments, the evaluator 222 and / or the model updater 226 are external to the computing system 102 (e.g., and connected to the computing system 102 via a network or other inter-system communication protocol).

[0052] In this way, for example, a machine learning model framework can be trained based on example aspects of this disclosure to obtain weights 124 and 128, which can provide processing for queries against unlabeled source activity 140. For example, the weights are not necessarily interpretable to human observers (e.g., interpretable as a cognitive representation of underlying real-world data), but can form a set of data structures whose function is to correlate the temporal distribution of unlabeled source activity events 142 with that of target activity events 152 in response to queries against them. In this way, for example, the data structures can be used to enable processing of improved queries relative to additional unlabeled and unindexed source activity events 142. In this way, for example, example implementations of the query processing systems and methods of this disclosure can provide the ability to extend computational systems to perform queries that they might otherwise be unable to do.

[0053] Figure 3 A block diagram of an example computing system 300 according to an example embodiment of the present disclosure is described. The example system 300 includes a client computing system 302, a server computing system 330, and a training computing system 350 that are communicatively coupled via a network 380.

[0054] Client computing system 302 can be any type of system containing one or more computing devices. Computing devices can be, for example, personal computing devices (e.g., laptops or desktops), mobile computing devices (e.g., smartphones or tablets), game consoles or controllers, wearable computing devices, embedded computing devices, server computing devices, nodes of distributed computing devices, virtual instances hosted on a shared server, or any other type of computing device. In some embodiments, client computing system 302 includes multiple computing devices interconnected via a network or distributed in an interoperable manner. For example, client computing system 302 may include a server for serving content over a network (e.g., network 380). For example, client computing system 302 may include a web server for hosting web content and for collecting data about the web content (e.g., for receiving, monitoring, generating, or otherwise processing data about the web content, such as using, downloading, and / or interacting with the web content).

[0055] In some embodiments, client computing system 302 may be associated with a service provider and / or source platform, for example, associated with source activity 140. For example, client computing system 302 may be a source platform hosting content on behalf of a service provider. For example, client computing system 302 may host online content for a service provider (e.g., with an offline presence, such as a physical location, for facilitating target activity 150) also associated with target activity 150. For example, client computing system 302 may host first-party content. Source activity 140 may include interactions with first-party content and may be measured as part of a first-party measurement collection. For example, a first-party entity may be associated with the publication of first-party content.

[0056] The client computing system 302 includes one or more processors 312 and memory 314. The one or more processors 312 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be one or more processors operatively connected. The memory 314 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 314 can store data 316 and instructions 318 that are executed by the processor 312 to cause the client computing system 302 to perform operations.

[0057] In some implementations, the client computing system 302 may store or include one or more machine learning models 320. For example, machine learning model 320 may be or may additionally include various machine learning models, such as neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear and / or linear models. Neural networks may include feedforward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. Some machine learning models may utilize attention mechanisms, such as self-attention. For example, some exemplary machine learning models may include multi-head self-attention models (e.g., transformer models). Example machine learning model 320 includes a machine learning model framework that includes a transformation model 122 and a lag model 126, as referenced above. Figure 1 and 2 Discussed.

[0058] In some implementations, one or more machine learning models 320 may be received from a server computing system 330 via a network 380, stored in a client computing system memory 314, and then used or otherwise executed by one or more processors 312. In some implementations, the client computing system 302 may execute multiple parallel instances of a single machine learning model 320.

[0059] Alternatively, one or more machine learning models 340 (which may be the same as or different from machine learning model 320) may be included in or otherwise stored and executed in server computing system 330, which communicates with client computing system 302 according to a client-server relationship. For example, machine learning model 340 may be implemented by server computing system 340 as part of a web service (e.g., a service for processing queries on source activity 140 according to any of the aspects of this disclosure). Example machine learning model 340 includes a machine learning model framework comprising transformation model 122 and lag model 126, as referenced above. Figure 1 and 2 This is under discussion. Therefore, one or more machine learning models 320 may be stored and implemented at the client computing system 302, and / or one or more machine learning models 340 may be stored and implemented at the server computing system 330.

[0060] In some embodiments, the server computing system 330 may configure a dual-model framework for the query processing subsystem 120. For example, in some embodiments, the server computing system 330 may store one or more transformation models 122 and / or one or more lag models 126 (e.g., a set of one or more weights 124, weights 128, etc.) to replace one or more other transformation models 122 and / or lag models 126, or to replace them with one or more other transformation models 122 and / or lag models 126 for the configuration and / or maintenance of the query processing subsystem 120. For example, in some embodiments, some transformation models or lag models may be broadly applicable, while some transformation models or lag models may provide improved performance in a specific domain (e.g., a specific type of subject activity, a specific type of target activity, etc.), and the server computing system 330 may replace and / or supplement the dual-model framework of the query processing subsystem 120 to improve and configure the processing of query 130.

[0061] The client computing system 302 may also include one or more input components 322 that receive input (e.g., user input, input from other systems, etc.). For example, the input component 322 may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) that is sensitive to the touch of an input object (e.g., a finger or stylus). Other example input components include a microphone, a keyboard (e.g., a physical and / or graphical interface), a network port (e.g., wireless, wired, etc.), a communication bus, etc.

[0062] Server computing system 330 includes one or more processors 332 and memory 334. The one or more processors 332 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be one or more processors operatively connected. Memory 334 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 334 can store data 336 and instructions 338 that are executed by processor 332 to cause server computing system 330 to perform operations.

[0063] In some implementations, the server computing system 330 includes or is implemented by one or more server computing devices. In instances where the server computing system 330 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0064] As described above, the server computing system 330 may store or otherwise include one or more machine learning models 340. Exemplary machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some machine learning models may utilize attention mechanisms, such as self-attention. For example, some exemplary machine learning models may include multi-head self-attention models (e.g., transformer models).

[0065] In some embodiments, the client computing system 302 may have access to information that is not available to the server computing system 330 and / or training computing system 350. In some embodiments, the client computing system 302 may be configured to host first-party content. First-party content may include, for example, content associated with the owner, operator, and / or beneficiary (e.g., contractual beneficiary, lessee of computing time on the client computing system 302). In some embodiments, the client computing system 302 may collect data (e.g., telemetry, analytics, usage statistics, logs, etc.) regarding the downloading, viewing, and use of first-party content (e.g., source activity 140). In some embodiments, the client computing system 302 may collect data regarding the downloading, viewing, and use of first-party content and / or other services or aspects associated with the client computing system or its beneficiaries, and / or data linked to first-party content and / or other services or aspects. In some embodiments, the server computing system 330 may not have full or unlimited access to the first-party content on the client computing system 302, or unlimited access to data regarding the viewing and use of the content.

[0066] In some embodiments, neither the client computing system 302 nor the server computing system 330 has full access to the target activity 150. For example, in some scenarios, the target activity 150 is not or cannot be fully recorded. In some scenarios, the client computing system 302 and / or the server computing system 330 do not have complete access to the target activity 150 (or, for example, the data describing it). However, in some embodiments, the server computing system 330 may, for example, have access to tagged records 170, which may optionally include a tagged subset of the target activity 150 and a tagged subset of the source activity 140. For example, the server computing system 330 may be associated with multiple logged-in users who report their participation in both the source activity 140 and the target activity 150.

[0067] Therefore, in some embodiments, according to exemplary aspects of this disclosure, one or more machine learning models 340 may be advantageously trained to associate source activity 140 with target activity 150. For example, one or more machine learning models 340 may be trained according to exemplary aspects of this disclosure to learn to generate a relationship between source activity 140 and target activity 150 to obtain a set of weights (e.g., weights 124 and 128) for obtaining the results of a query processed on source activity 140.

[0068] Client computing system 302 and / or server computing system 330 can train models 320 and / or 340 through interaction with training computing system 350, which is communicatively coupled via network 380. Training computing system 350 may be separate from server computing system 330, or it may be part of server computing system 330.

[0069] The training computing system 350 includes one or more processors 352 and memory 354. The one or more processors 352 can be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and can be one or more processors operatively connected. The memory 354 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 354 can store data 356 and instructions 358 executed by the processor 352 to cause the training computing system 350 to perform operations. In some implementations, the training computing system 350 includes one or more server computing devices or is implemented using one or more server computing devices.

[0070] Training computation system 350 may include model trainer 360, which uses various training or learning techniques (such as, for example, backpropagation of error) to train machine learning models 320 and / or 340 stored at client computation system 302 and / or server computation system 330. For example, a loss function can be used to update one or more parameters of the model by backpropagation of the model (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques can be used to iteratively update parameters in one or more training iterations. In some implementations, performing backpropagation of error may include performing truncated backpropagation over time. Model trainer 360 may perform one or more techniques (e.g., weight decay, dropout, etc.) to improve the capabilities of the trained model.

[0071] Specifically, the model trainer 360 can train machine learning models 320 and / or 340 based on a set of training data 362. The training data 362 may include, for example, aspects according to this disclosure (e.g., as described above, such as references...). Figure 1 and Figure 2 The record marked with ).

[0072] In some implementations, training data 362 may include data provided by or otherwise obtained by the client computing system 302. Therefore, in such an implementation, model 320 provided to the client computing system 302 and / or model 340 provided to the server computing system 330 may be trained by the training computing system 350 on data received from the client computing system 302. In some embodiments, training data 362 includes data inaccessible to the server computing system 330 and / or the training computing system 350 unless provided by the client computing system 302.

[0073] The model trainer 360 includes computer logic for providing the required functionality. The model trainer 360 can be implemented as hardware, firmware, and / or software that controls a general-purpose processor. For example, in some implementations, the model trainer 360 includes a program file stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, the model trainer 360 includes one or more sets of computer-executable instructions stored in a tangible computer-readable storage medium, such as RAM, a hard disk, or optical or magnetic media.

[0074] Network 380 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., the Internet), or a combination thereof, and can include any number of wired or wireless links. Generally, communication on Network 380 can be carried over any type of wired and / or wireless connection using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0075] The machine learning models described in this specification (such as models 120, 320, 340, etc.) can be used for a variety of tasks, applications, and / or use cases. In some implementations, the input to the machine learning models of this disclosure can be image data. The machine learning models can process image data to generate outputs. As an example, the machine learning models can process image data to produce image recognition outputs (e.g., recognition of image data, latent embedding of image data, encoded representation of image data, hashing of image data, etc.). As another example, the machine learning models can process image data to generate image segmentation outputs. As another example, the machine learning models can process image data to generate image classification outputs. As another example, the machine learning models can process image data to generate image data modification outputs (e.g., changes to image data, etc.). As another example, the machine learning models can process image data to generate encoded image data outputs (e.g., encoded and / or compressed representations of image data, etc.). As another example, the machine learning models can process image data to generate magnified image data outputs. As another example, the machine learning models can process image data to generate predictive outputs.

[0076] In some implementations, the input to the machine learning model of this disclosure can be text or natural language data. The machine learning model can process the text or natural language data to generate output. As an example, the machine learning model can process natural language data to generate language-encoded output. As another example, the machine learning model can process text or natural language data to generate latent text embedding output. As another example, the machine learning model can process text or natural language data to generate translation output. As another example, the machine learning model can process text or natural language data to generate classification output. As another example, the machine learning model can process text or natural language data to generate text segmentation output. As another example, the machine learning model can process text or natural language data to generate semantic intent output. As another example, the machine learning model can process text or natural language data to generate upscaled text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language). As yet another example, the machine learning model can process text or natural language data to generate predictive output.

[0077] In some implementations, the input to the machine learning model of this disclosure can be speech data. The machine learning model can process the speech data to generate output. As an example, the machine learning model can process speech data to generate speech recognition output. As another example, the machine learning model can process speech data to generate speech translation output. As another example, the machine learning model can process speech data to generate latent embedding output. As another example, the machine learning model can process speech data to produce encoded speech output (e.g., encoded and / or compressed representations of speech data, etc.). As another example, the machine learning model can process speech data to generate upgraded speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, the machine learning model can process speech data to generate text representation output (e.g., a text representation of the input speech data, etc.). As yet another example, the machine learning model can process speech data to generate predictive output.

[0078] In some implementations, the input to the machine learning model of this disclosure can be latent encoded data (e.g., a latent space representation of the input). The machine learning model can process the latent encoded data to generate an output. As an example, the machine learning model can process the latent encoded data to generate a recognition output. As another example, the machine learning model can process the latent encoded data to generate a reconstruction output. As yet another example, the machine learning model can process the latent encoded data to generate a search output. As yet another example, the machine learning model can process the latent encoded data to generate a reclustering output. As yet another example, the machine learning model can process the latent encoded data to generate a prediction output.

[0079] In some implementations, the input to the machine learning model of this disclosure can be statistical data. Statistical data can be, represent, or otherwise include data calculated and / or computed from some other data source. The machine learning model can process the statistical data to generate output. For example, the machine learning model can process the statistical data to generate an identification output. As another example, the machine learning model can process the statistical data to generate a prediction output. As another example, the machine learning model can process the statistical data to generate a classification output. As another example, the machine learning model can process the statistical data to generate a segmentation output. As another example, the machine learning model can process the statistical data to generate a visualization output. As another example, the machine learning model can process the statistical data to generate a diagnostic output.

[0080] In some implementations, the input to the machine learning model of this disclosure can be sensor data. The machine learning model can process the sensor data to generate output. As an example, the machine learning model can process sensor data to produce a recognition output. As another example, the machine learning model can process sensor data to generate a prediction output. As another example, the machine learning model can process sensor data to generate a classification output. As another example, the machine learning model can process sensor data to generate a segmentation output. As another example, the machine learning model can process sensor data to generate a visualization output. As another example, the machine learning model can process sensor data to generate a diagnostic output. As another example, the machine learning model can process sensor data to generate a detection output.

[0081] In some cases, machine learning models can be configured to perform tasks involving encoding input data for reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task could be an audio compression task. The input could include audio data, and the output could include compressed audio data. In another example, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In yet another example, the task could include generating an embedding of the input data (e.g., input audio or video data).

[0082] In some cases, the input includes visual data, and the task is a computer vision task. In other cases, the input includes pixel data from one or more images, and the task is an image processing task. For example, an image processing task could be image classification, where the output is a set of scores, each corresponding to a different object class and representing the likelihood of one or more images describing objects belonging to that object class. An image processing task could be object detection, where the image processing output identifies one or more regions in one or more images, and for each region, the region describes the likelihood of an object of interest. As another example, an image processing task could be image segmentation, where the image processing output defines a corresponding likelihood for each class in a predetermined set of classes for each pixel in one or more images. For example, the set of classes could be foreground and background. As another example, the set of classes could be object classes. As another example, an image processing task could be depth estimation, where the image processing output defines a corresponding depth value for each pixel in one or more images. As another example, an image processing task could be motion estimation, where the network input includes multiple images, and the image processing output defines the motion of a scene described at pixels between the images in the network input for each pixel of one of the input images.

[0083] In some cases, the input includes audio data representing spoken language, and the task is speech recognition. The output may include text mapped to spoken language. In some cases, the task involves encrypting or decrypting the input data. In some cases, the task involves microprocessor performance tasks such as branch prediction or memory address translation.

[0084] In some embodiments, any of the above-described inputs may be provided for the tagging task or other indexing task. For example, any of the above-described inputs or other inputs may be or include instances, such as untagged instances (e.g., missing some or all tags, such as missing expected tags). In some embodiments, the task is to process a query against an input instance. The output (e.g., or intermediate output) may include a data structure that associates untagged instances with one or more values ​​indicating their relationship to a query tag. In this way, for example, the task may be an indexing task to index untagged instances (e.g., tag data about tags not previously associated with instances) used to process tag data queries. The output may include counts or other summary outputs describing the relationship between untagged instances and query tags. The output may include retrievals of untagged instances determined to be associated with query tags. In some embodiments, the index may be transient (e.g., stored to obtain various metrics and / or analyses from processing queries against indexed instances and later unloaded) or stored for a duration exceeding the transient period (e.g., written to disk, etc.).

[0085] Figure 3 An example computing system that can be used to implement this disclosure is shown. Other computing systems may also be used. For example, in some implementations, the client computing system 102 may include a model trainer 160 and a training dataset 162. In such an implementation, the model 120 may be trained and used locally at the client computing system 102. In some such implementations, the client computing system 102 may implement the model trainer 160 to personalize the model 120 based on user-specific data.

[0086] Figure 4 A flowchart describing an example method 400 performed according to an example embodiment of the present disclosure is provided. Although for purposes of illustration and discussion... Figure 4 The steps are described in a specific order, but the methods disclosed herein are not limited to the order or arrangement specifically shown. The steps of example method 400 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of this disclosure.

[0087] In example 402, example method 400 includes receiving a tagged record. In example method 400, the tagged record may include a source activity and a target activity of the link. For example, the tagged record may include tagged record 170.

[0088] At 404, example method 400 includes updating one or more parameters of a first machine learning model using tagged records. In example method 400, the first machine learning model is configured to output data describing a target activity associated with a source activity. For example, the first machine learning model may include a transformation model (e.g., transformation model 122) for determining a transformation from a source activity (e.g., an activity on a source platform, such as source activity 140) to a related target activity (e.g., an activity in another pathway related to the topic of the source activity). In some embodiments, the source activity data includes data describing online activities, while the target activity includes offline activities. In some embodiments, the source activity data includes data describing online activities, while the target activity includes online activities.

[0089] In 406, example method 400 includes updating one or more parameters of a second machine learning model using labeled records. In example method 400, the second machine learning model is configured to output the distribution of the target activity over time. For example, the second machine learning model may include a lag model (e.g., lag model 126) for determining the temporal distribution of the target activity flowing from or otherwise related to the source activity.

[0090] In some embodiments of Example Method 400, the first machine learning model and the second machine learning model are part of a machine learning model framework configured to process queries on source activity data. For example, in some embodiments, processing the query includes receiving source activity data. In some embodiments, the source activity data may be associated with linked source activities. For example, in some embodiments, linked source activities form a subset of the source activity data (e.g., a subset of otherwise unlinked source activity data linked to a target activity). In some embodiments, the source activity data includes data describing online activities on a source system, and communication with the source system is restricted to indicating links between the source activity data and the target activity. In some embodiments, processing the query includes using the first machine learning model to determine a predicted target activity associated with the source activity data. In some embodiments, processing the query includes using the second machine learning model to generate a predicted temporal distribution of the target activity. In some embodiments, processing the query includes generating query results in response to the query based at least in part on the predicted target activity and the predicted temporal distribution of the target activity. In some embodiments, generating the query results includes sampling the output of the second machine learning model. In some embodiments, the query results include a data structure that associates the predicted target activity with multiple time periods.

[0091] In some embodiments of Example Method 400, the first machine learning model and the second machine learning model are trained using different updates. For example, in some embodiments, one or more parameters of the first machine learning model are updated independently of one or more parameters of the second machine learning model. For example, in some embodiments, updating one or more parameters includes updating one or more first parameters of the first machine learning model based at least in part on the target activity count on the training set for predicting target activities. In some embodiments, updating one or more parameters includes updating one or more second parameters of the second machine learning model based at least in part on the predicted temporal distribution of target activities.

[0092] In some embodiments of example method 400, both the first machine learning model and the second machine learning model receive the same set of input signals. In some embodiments, the input to the second machine learning model includes dates (e.g., in addition to the set also provided to the first machine learning model). In some embodiments, the output of the first machine learning model is input to the second machine learning model. In some embodiments, the output of the first machine learning model includes a target activity count.

[0093] Figure 5 A flowchart describing an example method 500 performed according to an example embodiment of the present disclosure is provided. Although for purposes of illustration and discussion... Figure 5 The steps are described in a specific order, but the method disclosed herein is not limited to the order or arrangement specifically shown. The steps of example method 500 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of this disclosure.

[0094] At 502, example method 500 includes receiving source activity data. In some embodiments, the source activity data (e.g., source activity 140) includes data describing online activity. In some embodiments, the source activity data includes data describing online activity associated with a first-party platform (e.g., client computing device 302).

[0095] At 504, example method 500 includes performing a query on a target activity related to the source activity data. In some embodiments, the target activity includes offline activities. In some embodiments, the query is an implicit query (e.g., implicit in the architecture and / or machine learning framework of a query processing subsystem such as query processing subsystem 120).

[0096] At 506, executing the query includes using a first machine learning model within a machine learning model framework to determine predicted target activities related to the source activity data. In some embodiments, the output of the first machine learning model includes a count of target activities (e.g., an estimate of the number of target activity events).

[0097] At 508, executing the query includes generating a predicted temporal distribution of the target activity (e.g., target activity 150) using a second machine learning model framework. In some embodiments, the output of the second machine learning model includes the temporal distribution of the target activity (e.g., a probability distribution).

[0098] In 510, example method 500 includes generating query results (e.g., query result 160) in response to a query based at least in part on predicted target activities and the predicted temporal distribution of the target activities. In some embodiments, the query results may include data describing the distribution of the predicted target activities over time. In some embodiments, the query results include a data structure that associates the predicted target activities with multiple time periods. In some embodiments, generating the query results includes sampling the output of a second machine learning model.

[0099] In some embodiments of Example Method 500, the first machine learning model and the second machine learning model are trained using different updates. For example, in some embodiments, updating one or more parameters includes updating one or more first parameters of the first machine learning model based at least in part on the target activity count on the training set for predicting target activities, and updating one or more second parameters of the second machine learning model based at least in part on the predicted temporal distribution of the target activities.

[0100] In some embodiments of example method 500, both the first machine learning model and the second machine learning model receive the same set of input signals. In some embodiments, the input to the second machine learning model includes dates (e.g., in addition to the set also provided to the first machine learning model). In some embodiments, the output of the first machine learning model is input to the second machine learning model. In some embodiments, the output of the first machine learning model includes a target activity count.

[0101] In some embodiments of example method 500, the source activity data includes data describing online activities on the source system, and communication with the source system is limited to indicating links between the source activity data and the target activity.

[0102] Figure 6 A flowchart describing an example method 600 for training an exemplary machine learning model framework (e.g., training a query processing subsystem 120, such as executing example methods 400 and / or 500) according to an example embodiment of the present disclosure. Although for illustrative and discussion purposes, Figure 6 The steps are described in a specific order, but the method of this disclosure is not limited to the order or arrangement specifically shown. The steps of example method 600 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of this disclosure.

[0103] Example method 600 may include any one or more portions of example method 400 and / or example method 500. For example, example method 600 may be included in or precede example method 400 and / or example method 500.

[0104] For example, at 602, example method 600 includes receiving a tagged record. In some embodiments, the tagged record (e.g., tagged record 170) may include source activity data (e.g., as received at 502). In example method 600, the tagged record includes linked source activity data and linked target activity data (e.g., source activity data linked to a target activity event).

[0105] In 604, example method 600 includes performing a training query on target activities associated with source activities. In some embodiments of example method 600, performing the training query includes using a first machine learning model to determine a training set of predicted target activities associated with linked source activity data. In some embodiments of example method 600, performing the training query includes using a second machine learning model to generate a training temporal distribution of the target activities.

[0106] In 610, example method 600 includes updating one or more parameters of a machine learning model framework based at least in part on a training set predicting target activities, a training temporal distribution of target activities, and linked target activity data. For example, in some embodiments, a first machine learning model and a second machine learning model are trained using different updates. In some embodiments, updating one or more parameters includes updating one or more first parameters of the first machine learning model based at least in part on the target activity counts on the training set predicting target activities. In some embodiments, updating one or more parameters includes updating one or more second parameters of the second machine learning model based at least in part on the predicted temporal distribution of target activities.

[0107] Other public information

[0108] The technologies discussed here refer to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, and divisions of tasks and functions between and within components. For example, the processes discussed here can be implemented using a single device or component, or a combination of multiple devices or components. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can run sequentially or in parallel.

[0109] While this subject matter has been described in detail with respect to its various specific example embodiments, each example is provided by way of explanation and not limitation. Changes, modifications, and equivalents to such embodiments will readily arise for those skilled in the art upon acquiring the foregoing understanding. Therefore, this subject matter disclosure does not exclude the inclusion of such modifications, modifications, and / or additions to this subject matter that will be obvious to those skilled in the art. For example, features shown or described as part of one embodiment may be used with another embodiment to produce yet another embodiment. Therefore, this disclosure is intended to cover such changes, modifications, and equivalents.

Claims

1. A computer-implemented method, comprising: Source activity data is received by a computing system comprising one or more processors; The computing system executes a query on a target activity related to the source activity data, wherein executing the query includes: The computing system uses a first machine learning model within a machine learning model framework to determine the predicted target activity associated with the source activity data; and The computing system generates the predicted temporal distribution of the target activity using a second machine learning model within the machine learning model framework; and The computing system generates query results in response to the query, based at least in part on the predicted target activity and the predicted time distribution of the target activity.

2. A computer-implemented method, comprising: A record containing linked source activity data and linked target activity data is received by a computing system including one or more processors; The computing system executes a training query for a target activity related to the source activity data, wherein executing the training query includes: The computing system uses a first machine learning model within a machine learning model framework to determine a training set of predicted target activities related to the source activity data of the link; and The computing system generates the training time distribution of the target activity using a second machine learning model within the machine learning model framework; and The computing system updates one or more parameters of the machine learning model framework based at least in part on the training set for predicting target activities, the training time distribution of target activities, and linked target activity data.

3. The computer-implemented method as described in claim 1, wherein, Generating the query results includes sampling the output of the second machine learning model.

4. The computer-implemented method as described in claim 1 or claim 3, wherein, The query results include a data structure that associates the predicted target activity with multiple time periods.

5. The computer-implemented method as described in claim 2, wherein, The first machine learning model and the second machine learning model are trained using different updates.

6. The computer-implemented method as described in claim 2, wherein, Updating one or more of the parameters includes: The computing system updates one or more first parameters of the first machine learning model based at least in part on the target activity counts on the training set for predicting target activities; and The computing system updates one or more second parameters of the second machine learning model based at least in part on the predicted temporal distribution of the target activity.

7. The computer-implemented method as described in claim 1 or claim 2, wherein, Both the first machine learning model and the second machine learning model receive the same set of input signals.

8. The computer-implemented method as described in claim 1 or claim 2, wherein, The input to the second machine learning model includes dates.

9. The computer-implemented method as described in claim 1 or claim 2, wherein, The source activity data includes data describing online activities on a source system, and communication with the source system is restricted to indicate links between the source activity data and the target activity.

10. The computer-implemented method as described in claim 1 or claim 2, wherein, The output of the first machine learning model is input into the second machine learning model.

11. The computer-implemented method as described in claim 1 or claim 2, wherein, The output of the first machine learning model includes the target activity count.

12. The computer-implemented method as described in claim 1 or claim 2, wherein, The source activity data includes data describing online activities, and the target activity includes offline activities.

13. The computer-implemented method as described in claim 1 or claim 2, wherein, The source activity data includes data describing online activities, and the target activity includes online activities.

14. A computer-implemented method, comprising: A computing system comprising one or more processors receives a record containing tags of linked source activities and linked target activities; The computing system updates one or more parameters of a first machine learning model using the marked records, the first machine learning model being configured to output data describing a target activity associated with a source activity; as well as The computing system uses the marked records to update one or more parameters of a second machine learning model configured to output the temporal distribution of the target activity, the second machine learning model being different from the first machine learning model.

15. The computer-implemented method of claim 14, further comprising: The computing system receives source activity data, wherein the source activity data is associated with the source activity of the link; The computing system uses the first machine learning model to determine the predicted target activity related to the source activity data; The computing system uses the second machine learning model to generate the predicted temporal distribution of the target activity; and The computing system generates query results in response to the query, based at least in part on the predicted target activity and the predicted time distribution of the target activity.

16. The computer-implemented method as described in claim 15, wherein, Generating the query results includes sampling the target activity and the predicted time distribution of the target activity by the computing system.

17. The computer-implemented method as described in claim 14, wherein, The parameters of the first machine learning model are updated independently of the parameters of the second machine learning model.

18. The computer-implemented method as described in claim 14, wherein, Both the first machine learning model and the second machine learning model receive the same set of input signals.

19. The computer-implemented method as described in claim 15, wherein, The source activity data includes data describing online activities on a source system, and communication with the source system is restricted to indicate links between the source activity data and the target activity.

20. A system comprising: One or more processors; as well as One or more memory devices storing computer-readable instructions that, when executed, cause the one or more processors to perform operations, including the methods as described in any of the preceding claims.

21. The system as described in claim 20, in, The one or more memory devices store a learning data structure for performing queries, the learning data structure including a first weight of a first machine learning model and a second weight of a second machine learning model; And among them, The operation further includes: The target activity is determined by transforming the input signal set using the learned data structure; and The predicted temporal distribution of the target activity is generated by transforming the input signal set using the learning data structure.

22. A computer-readable medium storing computer-readable instructions that, when executed, cause one or more processors to perform operations, including the method as claimed in any one of claims 1-2 and 14.

Images