Techniques for generating an analysis report

Abstract

Techniques for generating a website report are presented herein. The method can include accessing, by one or more computing devices, a plurality of non-identifying events. Each event in the plurality of non-identifying events can be associated with one or more characteristics. Further, the method can calculate the number of pseudo-users associated with the plurality of non-identifying events, using a machine-learned prediction model, based on the ratio of events to users and the total number of non-identifying events. Further, the method can include assigning a first event from the plurality of non-identifying events to a first pseudo-user based on one or more characteristics of the first event. Further, the method can include generating a report of the website. The report includes information derived from the first event assigned to the first pseudo-user.

Description

Technical Field

[0001] The present disclosure generally relates to automatically generating data analysis reports for websites. More specifically, the present disclosure relates to techniques for using a machine-learned prediction model to determine information (e.g., number of users, number of sessions) associated with a plurality of non-identifying events.

Background Art

[0002] Website owners may wish to understand what content users have accessed, which campaigns have performed best, and important touchpoints in driving conversions. However, it can be difficult to understand the actions users take after interacting with content items or impressions due to potential gaps in the information available to website owners. Additionally, users can interface with websites using different browsers and devices. Further, users may research a product online and then purchase it offline. Website owners need to make data-driven decisions and focus on improving the customer journey.

Summary of the Invention

[0003] Aspects and advantages of embodiments of the present disclosure are shown in part in the following description, or can be learned from the description, or can be learned through the practice of the embodiments.

[0004] The present disclosure provides a system and method for generating analysis data for an analysis report using a machine - learned prediction model. The analysis data can include user - identifying data and non - identifying events (e.g., events that may lack a stable user identifier). The machine - learned prediction model can estimate business metrics from non - identifying events. Business metrics can include the number of users, the number of sessions, along with other dimensions such as gender and advertising campaigns. The system can internally combine metrics from identifying data with estimates from non - identifying data (e.g., non - identifying events) to provide a complete analysis report for business decision - making.

[0005] One exemplary aspect of the present disclosure is directed to a computer - implemented method. The method can include accessing, by one or more computing devices, a plurality of non - identifying events. The plurality of non - identifying events has a total number of non - identifying events. For example, non - identifying events can be associated with events that do not have a user identifier (e.g., a stable user identifier). Each event in the plurality of non - identifying events can be associated with one or more characteristics. Further, the method can include calculating, using a machine - learned prediction model, the number of pseudo - users associated with the plurality of non - identifying events based on the event - to - user ratio and the total number of non - identifying events. Further, the method can include assigning, using a machine - learned prediction model, a first event from the plurality of non - identifying events to a first pseudo - user based on one or more characteristics of the first event. Further, the method can include generating an analysis report for a website. The analysis report can include information derived from the number of pseudo - users and the first event assigned to the first pseudo - user.

[0006] Other exemplary aspects of the present disclosure are directed to a computing system including one or more processors and one or more non-transitory computer-readable media. The one or more non-transitory computer-readable media may collectively store a machine-learned prediction model and instructions that, when executed by the one or more processors, cause the computing system to perform operations. The machine-learned prediction model is configured to generate an event-to-user ratio based on data derived from a plurality of identification events associated with an identified user of a website when receiving an identifier while browsing the website. The operations may include accessing a plurality of non-identification events. For example, the non-identification events may be associated with events without a user identifier (such as a stable user identifier). The plurality of non-identification events has a total number of non-identification events. Each event in the plurality of non-identification events may be associated with one or more characteristics. Further, the operations may include using the machine-learned prediction model to calculate the number of pseudo-users associated with the plurality of non-identification events based on the event-to-user ratio and the total number of non-identification events. Further, the operations may include using the machine-learned prediction model to assign a first event from the plurality of non-identification events to a first pseudo-user based on one or more characteristics of the first event. Further, the operations may include generating an analysis report for the website. The analysis report may include information derived from the number of pseudo-users and the first event assigned to the first pseudo-user. In some embodiments, the processing parameters of the system may be adjusted based on data obtained from the generated analysis report.

[0007] Still other exemplary aspects of the present disclosure are directed to one or more non-transitory computer-readable media that, when executed by one or more computing devices, cause the one or more computing devices to perform operations. The operations can include accessing a plurality of non-identifying events. For example, the non-identifying events can be associated with events without a user identifier (such as a stable user identifier). The plurality of non-identifying events has a total number of non-identifying events. Each event in the plurality of non-identifying events is associated with one or more characteristics. Further, the operations can include using a machine-learned prediction model to calculate the number of pseudo-users associated with the plurality of non-identifying events based on the event-to-user ratio and the total number of non-identifying events. Further, the operations can include using a machine-learned prediction model to assign a first event from the plurality of non-identifying events to a first pseudo-user based on one or more characteristics of the first event. Further, the operations can include generating an analysis report for a website. The analysis report can include information derived from the number of pseudo-users and the first event assigned to the first pseudo-user.

[0008] In some embodiments, the machine-learned prediction model can determine the event-to-user ratio based on data derived from a plurality of identifying events associated with the identified users of the website that received an identifier when browsing the website. In some examples, the data derived from the plurality of identifying events can be first party data from the website.

[0009] In some embodiments, the first event can be associated with a first characteristic. Further, the method can further include selecting a subset of similar users from a plurality of identified users based on the first characteristic of the first event. The subset of similar users can be associated with the first characteristic. Further, the method can include updating an event-to-user ratio based on data derived from the subset of similar users.

[0010] In some embodiments, the first event can be adding an item to a shopping cart on a website. Further, the subset of similar users can be users who also added an item to the shopping cart on the website.

[0011] In some embodiments, the first characteristic can be the web browser associated with the first event. Alternatively, in some embodiments, the first characteristic can be the country of origin associated with the first event. Alternatively, in some embodiments, the first characteristic can be the display resolution associated with the first event.

[0012] In some embodiments, the subset of similar users can be associated with a first dimension. Further, the method can further include using a trained machine learning prediction model to assign the first dimension to a first pseudo-user based on the first characteristic. Further, the analysis report can further include information derived from the first dimension assigned to the first pseudo-user.

[0013] In some embodiments, the first dimension can be the first visit date associated with the first event. Further, the first dimension can be assigned to the first pseudo-user based on a probability distribution derived from a plurality of identified events. In some embodiments, the first dimension can be the gender associated with the first pseudo-user.

[0014] In some embodiments, the method may further include using a machine - learned prediction model to calculate the number of sessions associated with a plurality of unidentifiable events based on the ratio of events to sessions and the total number of unidentifiable events. Further, the analysis report may further include information derived from the number of sessions.

[0015] In some embodiments, the machine - learned prediction model can determine the ratio of events to sessions based on data derived from a plurality of identifiable events associated with an identified user of a website when browsing the website and receiving an identifier.

[0016] In some embodiments, the method may further include using a machine - learned prediction model to assign a first pseudo - user to a plurality of pseudo - sessions. Further, the method may include assigning a first event to a first pseudo - session from the plurality of pseudo - sessions based on a first characteristic of the first event. Further, the analysis report may include information derived from the first pseudo - user assigned to the plurality of pseudo - sessions and the first event assigned to the first pseudo - session.

[0017] In some embodiments, the method may further include determining a series of events based on the configuration of a website. Further, the method may include using a machine - learned model to determine a second event from a plurality of unidentifiable events based on the series of events. Further, the method may include using a machine - learned prediction model to assign the second event to the first pseudo - session. The analysis report may further include information derived from the second event assigned to both the first pseudo - session and the first pseudo - user.

[0018] In some embodiments, the first event can be associated with a second dimension, and the method can further include using a machine-learned prediction model to assign the second dimension to a first pseudo-session. The analysis report can further include information derived from the second dimension assigned to both the first pseudo-session and the first pseudo-user.

[0019] In some embodiments, the method can further include presenting the analysis report on a display of the client computing device in response to a request received from the client computing device.

[0020] In some embodiments, the machine-learned prediction model can assign the first event to a first pseudo-user in real time at a time event. Further, the method can include accessing additional non-identifying events for a particular period after the time event. Further, the method can include reassigning the first event to a second pseudo-user based on the additional non-identifying events. Further, the method can include generating an updated report for the website. The updated report can include information derived from the first event reassigned to the second pseudo-user. Thereafter, the method can include replacing the analysis report presented on the display with the updated report.

[0021] In some embodiments, the machine-learned prediction model can be a regression model.

[0022] In some embodiments, the method can further include calculating a correction factor based on data obtained over multiple days. Further, the method can include updating the number of pseudo-users associated with a plurality of non-identifying events based on the correction factor. Further, the method can include determining the number of unique pseudo-users who visited a website over multiple days based on the correction factor. Further, the analysis report can include information derived from the number of unique pseudo-users who visited a website over multiple days.

[0023] Other aspects of the disclosure are directed to various systems, devices, non-transitory computer-readable media, user interfaces, and electronic devices.

[0024] These and other features, aspects, and advantages of the various embodiments of the disclosure will become better understood with reference to the following detailed description of the invention and the accompanying claims. The accompanying drawings, which are incorporated herein and form a part of this specification, illustrate exemplary embodiments of the disclosure and, together with the detailed description, serve to explain the relevant principles.

[0025] A detailed description of embodiments directed to those of ordinary skill in the art is set forth in this specification with reference to the accompanying drawings.

Brief Description of the Drawings

[0026]

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0027] Exemplary embodiments according to aspects of the present disclosure relate to a computing device having a machine - learned predictive model for generating an analysis report of a website. The report can be generated based on identified events and non - identified events. Identified events can be associated with users of a website who have consented to send online activities as first - party data to the publisher of the website. Non - identified events can be associated with events lacking a user identifier. The consent mode is an example where an event may lack a user identifier. In some examples, the user identifier is a stable user identifier. A stable (e.g., permanent) user identifier can be the same user identifier across events within a browser session in the case of consent, the same user identifier for events over multiple days in the case of platform constraints, or the same user identifier for events across multiple devices (e.g., desktop, mobile, etc.). In some embodiments, the machine - learned predictive model can acquire analysis data without an identifier and perform modeling to estimate behavioral data. The analysis data without an identifier can be first - party data obtained from a customer (e.g., an administrator of a website or a mobile application). For example, the machine - learned predictive model can model non - identified events associated with a pseudo - user based on the behavior of similar users who allow a user identifier. The training data used to train the machine - learned predictive model is based on data derived from identified events. The techniques described herein can predict analysis data associated with non - identified events by using a machine - learned predictive model. The machine - learned predictive model can estimate multiple metrics across multiple dimensions from non - identified events to generate an analysis report.

[0028] In some embodiments, the system can utilize a hybrid of a probabilistic hard-coded algorithm and a machine-learned prediction. For example, the system can use a machine-learned prediction model to assign unidentifiable events to pseudo-users, pseudo-sessions, and unobservable dimensions. Further, the machine-learned prediction model can determine the ratios and algorithms described herein for assigning unidentifiable events. In some examples, the ratios and hard-coding algorithms can be pre-computed by the machine-learned prediction model. Using both types of approaches (e.g., real-time determination and pre-computed ratios / algorithms), the tasks of the machine-learned prediction model are simplified, and the model can be made smaller, easier to train, and trained more quickly. Further, by using both types of approaches, the system can generate an analysis report in real-time from unidentifiable events. In some embodiments, the processing parameters of the system can be adjusted based on data obtained from the generated analysis report.

[0029] A machine-learned prediction model can analyze a vast amount of historical data, identify correlations and trends between key data points, and use those insights to make accurate predictions about behavior, thereby generating analysis data. A machine-learned prediction model can fill in the gaps of events without user identifiers. A machine-learned model can utilize first-party data obtained by a website to fill in the gaps of the customer journey and determine insights. The first-party data can include labeled data (e.g., observable measurements) that can be used to determine (e.g., estimate, predict) information about unidentifiable events (e.g., unlabeled data, data lacking user identifiers). For example, when a user transitions between devices, from online to offline, browser restrictions, and various consent choices, gaps in the customer journey can occur.

[0030] The systems and methods described herein can improve the prediction of behavioral modeling of unidentifiable events (e.g., unlabeled data). Behavioral modeling can be associated with unidentifiable events and unobservable dimensions, which may not have ground truth for training a machine learning model. Ground truth refers to the actual nature of the problem that is the target of a machine learning model, as reflected by a relevant dataset associated with the given use case. Behavioral modeling can provide insights into actions beyond conversions that a user can perform on a website or within a mobile application. A machine-learned prediction model can utilize data derived from events with identifiers to model and analyze events without identifiers. For example, the techniques described herein enable a website to associate a user with a session and answer questions such as "How many new users did I acquire in my last campaign?"

[0031] In some embodiments, a company may experience a loss of data from analytics reporting that is proportional to the number of events without identifiers. This results in an incomplete measurement scenario and may prevent the company from obtaining answers to questions such as the following. a) How many active users are there per day? b) How many new users did I acquire in my last campaign? c) What was the user journey from accessing my website to actually making a purchase? d) How many of my site visitors are based in Germany compared to the UK? e) What are the differences in user behavior between mobile visitors and web visitors?

[0032] For example, having an accurate count of the active users per day can be used by the system to determine how much processing, network bandwidth, and other computing resources may be needed to reduce crashes of websites and mobile applications. In some embodiments, the methods described in FIGS. 3-5 can further include adjusting processing parameters based on data from the generated analysis reports.

[0033] In some embodiments, the machine-learned prediction model attempts to fill in missing data by modeling events without user identifiers based on events with user identifiers. The training data used for modeling can be based on consented user data associated with characteristics that can train the model. The characteristics may be analysis reports and data sets related to the website and / or mobile application. The reports and user interfaces generated and presented may vary depending on the type of characteristics selected for display by the client.

[0034] For example, the machine-learned prediction model estimates data based on user and session metrics, such as the ratio of active users and conversions per day that may be unobservable when identifiers (e.g., analytics cookies, user IDs) are not available.

[0035] The machine-learned prediction model can generate modeled data for non-identifying events (e.g., unlabeled data). Further, the first-party data obtained from the website can be referred to as labeled data. For example, labeled data includes data obtained when a user visits the client's website and consents to analytics cookies. Further, when the user consents to personalize using an identifier, the data is referred to as labeled data. In some examples, the user can give consent for personalization by signing up for the first-party website or mobile application.

[0036] Alternatively, if the user does not consent to the use of analytics cookies or equivalent mobile application identifiers, the events are not associated with a user identifier and the data is referred to as non-identifying events (e.g., unlabeled data). For example, if the system collects 10 page view events, the system cannot observe and report whether those events are associated with 10 users or just 1 user. Instead, the system can use a machine-learned prediction model to analyze the non-identifying events based on the behavior of similar users who have accepted the allowed analytics cookies or equivalent mobile application identifiers.

[0037] In some embodiments, the systems and methods according to the exemplary aspects of the present disclosure can obtain the above advantages by using a machine-learned model framework to generate analysis data and / or reports based on labeled (e.g., identifying events) and unlabeled data (e.g., non-identifying events). For example, the machine-learned model framework can include a prediction model that can be trained using the labeled data. In this way, for example, the machine-learned model framework can learn to use the prediction model to generate predictions and analysis data using the unlabeled data.

[0038] In some embodiments, exemplary systems and methods according to the illustrative aspects of the present disclosure can provide for improved storage, management, retrieval, and cross-referencing of data structures in a memory (e.g., a database). For example, an exemplary database can include real-world data structures that describe various unidentifiable events (e.g., unlabeled data). Other databases can also include data structures that describe identifiable events (e.g., labeled data). Based on the labeled data, an exemplary computing system according to the present disclosure can learn an intermediate set of data structures (e.g., a set of learned parameters of a machine-learned predictive model) to map unidentifiable events to pseudo-users, pseudo-sessions, and unobservable dimensions. In some embodiments, for example, the intermediate set of data structures can function to provide an association between the unlabeled data of the database and one or more transformation labels to enable improved storage and / or retrieval of the unlabeled data (e.g., index-based storage based on one or more labels, search based on one or more labels, etc.).

[0039] In some embodiments, for example, the intermediate set of data structures can function to provide for the processing and execution of queries on unidentifiable events. For example, the queries can include queries for obtaining analysis data associated with the unidentifiable events. The unidentifiable events can be assigned to pseudo-users, pseudo-sessions, and unobservable dimensions to rapidly generate analysis data and / or reports. Advantageously, however, the intermediate set of data structures can functionally map the unidentifiable events to pseudo-users, pseudo-sessions, and unobservable dimensions. In this way, for example, exemplary systems and methods according to aspects of the present disclosure can provide for the execution and processing of queries on an input data set even when such queries are not available otherwise (e.g., for embodiments with poor data or limited communication).

[0040] In some embodiments, exemplary systems and methods according to the exemplary aspects of the present disclosure can provide for determining relevance from among non-identifying events (e.g., unlabeled data). For example, relevance can be determined along dimensions of no labeling or incomplete labeling in non-identifying event data. For example, temporal relevance can be determined for non-identifying event data even when the data lacks a complete (or any) labeling of temporal relationships. For example, in some embodiments, non-identifying event data can include a timestamp (e.g., date, time, date and time) associated with an event, but the non-identifying event data can lack a timestamp for any subsequent event even when the prediction model determines that there may be a series of events.

[0041] In some embodiments, a machine-learned prediction model can be updated by modifying one or more of the model's parameters. For example, the system can check the accuracy of the model by performing holdout validation and can modify the model parameter(s) based on the holdout validation. Holdout validation can maintain the accuracy of the model by comparing the estimated user data to a portion of the observed user data held out from model training, and that information is used to adjust the model (e.g., modify one or more parameters). Further, the system can communicate changes that can affect the data to the client.

[0042] The systems and methods of the present disclosure provide several technical effects and advantages. Aspects of the present disclosure can provide several technical improvements to machine learning prediction models by reducing the processing time for generating reports in real time. By using different types of ratios to determine the number of users and sessions, the system can reduce the amount of modeling required to generate data for reports. When some can be pre-computed, by using different types of ratios to train and / or execute the model, the computational resources (e.g., processor time, memory usage, etc.) required to train and / or execute the model can be reduced. The systems and methods described herein can improve the processing speed of generating reports and also reduce the computing resources required to perform modeling to generate data for reports. As a result, the system can achieve state-of-the-art performance while maintaining the accuracy of the prediction data. Further, the systems and methods described herein can adjust the processing parameters of the system based on the data from the generated analysis reports.

[0043] Referring now to the drawings, exemplary embodiments of the present disclosure will be discussed in more detail.

[0044] Embodiments of Correction over Multiple Days Assuming that the pseudo-user identifier can last for only one day, but in embodiments of correction over multiple days, given that the agreed-upon user identifier can persist over time, the system can update (e.g., correct, etc.) the analysis report by using more accurate data accessed over multiple days. Thus, when the system determines and reports the unique number of users who have agreed over any date range, the system can simply count the number of unique identifiers within that time window.

[0045] For example, if there are 100 users who consented and visited the website two days ago, 120 users who consented and visited the website the day before, and 30 users who visited on both days, then the number of unique users who consented over the two days is (100 + 120 - 30) = 190.

[0046] In some embodiments, the system does not allow the identifier assigned to a pseudo-user to persist across a day boundary. Thus, in some examples, the system includes a correction factor (also referred to as "correction for multiple days") for adjusting the overcount of pseudo-users over any date range. The correction factor can be the ratio of the unique consented users over the date range to the consented users for a single day over the date range.

[0047] Using the above example, over a two-day period, the correction factor is (100 + 120 - 30) / (100 + 120) = 190 / 220 = 0.86.

[0048] Thus, if the system determines that there were 220 pseudo-users two days ago and 250 pseudo-users yesterday, the corrected number of de-duplicated pseudo-users over the two days is (220 + 250) * 0.86 = 404. Thus, the system can determine that there were 404 unique pseudo-users over the two days.

[0049] Furthermore, after the de-duplication process, the system can determine that there are 46 pseudo-users who visited the website on both days (220 pseudo-users on the first day + 250 pseudo-users on the second day - 404 unique pseudo-users over the two days).

[0050] In some embodiments, when a report is generated by a customer, the correction for multiple days can be applied at query execution time (e.g., the system can calculate the correction when the report is requested and not store the correction factor in the system's backend).

[0051] In some embodiments, the method can further include calculating a correction factor based on data obtained over multiple days. Further, the method can include updating the number of pseudo-users associated with a plurality of non-identifying events based on the correction factor. Further, the method can include determining the number of unique pseudo-users who visited a website over multiple days based on the correction factor. Further, the analysis report can include information derived from the number of unique pseudo-users who visited a website over multiple days.

[0052] Exemplary Devices and Systems FIG. 1A shows a block diagram of an exemplary computing system 100 that performs image editing, according to an exemplary embodiment of the present disclosure. System 100 includes a user computing device 102, a server computing system 130, and a training computing system 150 communicatively coupled via a network 180.

[0053] The user computing device 102 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0054] The computing device 102 includes one or more processors 112 and a memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or multiple processors operably connected. The memory 114 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The memory 114 can store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0055] In some embodiments, the user computing device 102 can store or include one or more models 120. For example, the model 120 (e.g., a prediction model) can be various machine-learned models such as a neural network (e.g., a deep neural network), or other types of machine-learned models including non-linear models and / or linear models, or can include them in other ways. The neural network can include a feed-forward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Exemplary models 120 are described with reference to FIGS. 2-5.

[0056] In some embodiments, the one or more models 120 are received from the server computing system 130 via the network 180, stored in the user computing device memory 114, and then used by the one or more processors 112 or implemented in other ways. In some embodiments, the user computing device 102 can implement multiple parallel instances of a single model 120.

[0057] More specifically, the model 120 can be trained using the training computing system 150 with the training dataset 162 to train the parameters of the model to optimize the model. The training computing system 162 can add efficiency and accuracy to the training model depending on the observed data. Further, the training data 162 can also be the first party data. Further, the training data 162 can include data obtained when the user visits the client's website and agrees to the analytics cookie. Further, the training data 162 can include data obtained when the user accepts the identifier.

[0058] Additionally, or alternatively, one or more models 140 can be included in the server computing system 130 that communicates with the user computing device 102 according to a client-server relationship, or can be stored and implemented in other ways, and thereby implemented. For example, the model 140 can be implemented by the server computing system 140 as part of a web service (e.g., a website). Thus, one or more models 120 can be stored and implemented on the user computing device 102, and / or one or more models 140 can be stored and implemented on the server computing system 130.

[0059] In addition, the user computing device 102 can include one or more user input components 122 that receive user input. For example, the user input component 122 can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that senses the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve to implement a virtual keyboard. Other exemplary user input components include a microphone, a conventional keyboard, or other means by which a user can provide user input.

[0060] The server computing system 130 includes one or more processors 132 and a memory 134. The one or more processors 132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and can be one processor or multiple processors operably connected. The memory 134 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The memory 134 can store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0061] In some embodiments, the server computing system 130 includes or is otherwise implemented with one or more server computing devices. When the server computing system 130 includes multiple server computing devices, such server computing devices can operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0062] As described above, the server computing system 130 can store or otherwise include one or more machine-learned models 140. For example, the model 140 can be various machine-learned models or, alternatively, can include various machine-learned models. Exemplary machine-learned models include neural networks or other multi-layer non-linear models. Exemplary neural networks include feed-forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. The exemplary model 140 is described with reference to FIGS. 2-6.

[0063] The user computing device 102 and / or the server computing system 130 can train the models 120 and / or 140 through interaction with a training computing system 150 communicatively coupled via a network 180. The training computing system 150 can be separate from the server computing system 130 or can be part of the server computing system 130.

[0064] The training computing system 150 includes one or more processors 152 and a memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be a single processor or multiple processors operably connected. The memory 154 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The memory 154 can store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some embodiments, the training computing system 150 includes or is implemented by one or more server computing devices.

[0065] The training computing system 150 can include a model trainer 160 that trains the machine-learned models 120 and / or 140 stored in the user's computing device 102 and / or the server computing system 130 using various training or learning techniques such as, for example, backpropagation of error. For example, a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent can be used to iteratively update the parameters over a number of training iterations.

[0066] In some embodiments, performing backpropagation may include performing backpropagation that is truncated over time. The model trainer 160 can perform several generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained.

[0067] In particular, the model trainer 160 can train the image editing models 120 and / or 140 based on a set of training data 162. The training data 162 can include, for example, observational data and / or first party data.

[0068] In some embodiments, when the user provides consent, the user computing device 102 can provide training examples. Thus, in such embodiments, the model 120 provided to the user computing device 102 can be trained by the training computing system 150 with respect to user-specific data received from the user computing device 102. In some embodiments, this process can be referred to as personalization of the model.

[0069] The model trainer 160 includes computer logic utilized to provide the desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in some embodiments, the model trainer 160 includes a program file stored on a storage device, loaded into memory, and executed by one or more processors. In other embodiments, the model trainer 160 includes one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium such as RAM, a hard disk, or an optical or magnetic medium.

[0070] Network 180 can be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. Generally, communication via Network 180 can be performed via any type of wired or wireless link using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0071] FIG. 1A illustrates one exemplary computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some embodiments, user computing device 102 can include model trainer 160 and training dataset 162. In such embodiments, model 120 can be trained and used locally on user computing device 102. In some of such embodiments, user computing device 102 can implement model trainer 160 to individualize model 120 based on user-specific data.

[0072] FIG. 1B shows a block diagram of an exemplary computing device 10 implemented in accordance with an exemplary embodiment of the present disclosure. Computing device 10 can be a user computing device or a server computing device.

[0073] Computing device 10 includes several applications (e.g., applications 1 to N). Each application includes its own machine learning library and one or more trained machine learning models. For example, each application can include a trained machine learning model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and the like.

[0074] As illustrated in FIG. 1B, each application can communicate with several other components of the computing device, such as one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, each application can communicate with each device component using an API (e.g., a public API). In some embodiments, the API used by each application is specific to that application.

[0075] FIG. 1C represents a block diagram of an exemplary computing device 50 implemented in accordance with an exemplary embodiment of the present disclosure. Computing device 50 can be a user computing device or a server computing device.

[0076] The computing device 50 includes several websites and / or mobile applications (e.g., applications 1 to N). Each website and / or mobile application can communicate with the central intelligence layer. Exemplary websites can include e-commerce (e.g., shopping) websites, media streaming websites, social network websites, etc. Exemplary applications can include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc. In some embodiments, each application can communicate with the central intelligence layer (and the model(s) stored therein) using an API (e.g., a common API across all applications).

[0077] The central intelligence layer includes multiple machine-learned models. For example, as shown in FIG. 1C, each machine-learned model (e.g., model) can be provided for each website and / or application and managed by the central intelligence layer. In other embodiments, two or more websites and / or mobile applications can share a single machine-learned model. For example, in some embodiments, the central intelligence layer can provide a single model (e.g., a single model) for all of the websites and / or applications. In some embodiments, the central intelligence layer is included within or otherwise implemented by the operating system of the computing device 50.

[0078] The Central Intelligence Layer can communicate with the Central Device Data Layer. The Central Device Data Layer can be a centralized repository of data for the computing device 50. As shown in FIG. 1C, the Central Device Data Layer can communicate with some other components of the computing device, such as, for example, one or more sensors, a context manager, a device status component, and / or additional components. In some embodiments, the Central Device Data Layer can communicate with each device component using an API (e.g., a private API).

[0079] Exemplary Model Arrangement FIG. 2 shows a block diagram of an exemplary machine-learned prediction model 200 for determining analysis data for reports, according to an exemplary embodiment of the present disclosure. In some embodiments, a computing system (user computing device 102, server computing device 130, training computing device 150, computing device 10, computing device 50) can include the machine-learned prediction model 200.

[0080] In some embodiments, the machine-learned prediction model 200 can be a four-layer model. The machine-learned prediction model 200 can include a first-layer model for determining a pseudo-user (e.g., the first user 210) associated with a plurality of non-identifying events of a website. A pseudo-user can be a user associated with a non-identifying event. For example, when the system collects 10 page view events, the system can determine the number of pseudo-users (i.e., 1 to 10 people) associated with the 10 page view events.

[0081] In some embodiments, the machine - learned prediction model 200 can include a model of a second layer to determine sessions (e.g., the first session 220, the second session 222) associated with a plurality of non - identifying events of a website. A pseudo - user can be associated with one or more sessions. For example, as shown in FIG. 2, the first user 210 can be associated with the first session 220 and the second session 222. Continuing with the example of 10 page - view events, the system can determine the number of sessions associated with the 10 page - view events. In some examples, since one pseudo - user can have multiple sessions, the number of pseudo - users is less than the number of sessions. Sessions are sometimes referred to as user traffic.

[0082] In some embodiments, the machine - learned prediction model 200 can include a model of a third layer to assign non - identifying events (e.g., the first event 230, the second event 232, the third event 234) to pseudo - users and / or sessions. For example, as shown in FIG. 2, the first event 230, the second event 232, and the third event 234 are assigned to both the first session 220 and the first user. FIGS. 3 - 5 illustrate a method for assigning non - identifying events to pseudo - users and / or sessions according to an exemplary embodiment of the present disclosure.

[0083] In some embodiments, the machine-learned prediction model 200 may include a fourth layer model to assign unobservable dimensions (e.g., first visit date, number of website visits per day, previous event, consecutive events, gender) to the pseudo-users and / or sessions. The unobservable dimensions may be assigned based on characteristics associated with events already assigned to the pseudo-users or sessions. In some examples, the system may include hundreds of unobservable dimensions that the system can assign to the pseudo-users or sessions. For example, as shown in FIG. 2, the unobservable dimension 250 is assigned to both the first session 220 and the first user. FIGS. 3-5 illustrate a method for assigning unobservable dimensions to pseudo-users and / or sessions according to exemplary embodiments of the present disclosure.

[0084] Exemplary method FIG. 3 shows a flowchart diagram of an example for generating an analysis report of a website according to an exemplary embodiment of the present disclosure. FIG. 3 shows steps executed in a particular order for purposes of illustration and explanation, but the method of the present disclosure is not limited to the particular order or arrangement shown. The various steps of method 300 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0085] In some embodiments, method 300 can be executed by a computing system such as computing device 102, server computing system 130, training computing system 150, computing device 10, computing device 50, etc. Further, the computing system can use one or more processors (e.g., processors 112, 132, 152 (plural)) to execute method 300.

[0086] At 302, the computing system can access a plurality of un-identified events. For example, an un-identified event can be associated with an event without a user identifier (e.g., a stable user identifier). In some examples, if a user does not consent to the use of analytics cookies or equivalent mobile application identifiers, the event can be without a user identifier. The plurality of un-identified events has a total number of un-identified events. Each event in the plurality of un-identified events can be associated with one or more characteristics.

[0087] In some examples, the machine-learned prediction model can be a regression model. Further, an exemplary machine-learned prediction model includes a neural network or other multi-layer non-linear model. Exemplary neural networks include feed-forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks.

[0088] In some examples, the machine-learned prediction model can be trained using various training or learning techniques, such as, for example, backpropagation of error. For example, a loss function can be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent can be used to iteratively update the parameters over a number of training iterations.

[0089] At 304, the computing system can use the machine-learned prediction model to calculate the number of pseudo-users associated with the plurality of un-identified events based on the event-to-user ratio and the total number of un-identified events. In some examples, the computing system can use the machine-learned prediction model to calculate the number of pseudo-users associated with the plurality of un-identified events based on the event-to-user ratio.

[0090] In some embodiments, the machine-learned prediction model can determine an event-to-user ratio based on data derived from a plurality of identification events associated with an identified user of an identified website. For example, a user can accept an analytics cookie when browsing a website. In other examples, a user can accept a user identifier when interfacing with or browsing a mobile application. In some examples, the data derived from the plurality of identification events can be first-party data from a website or a mobile application.

[0091] In operation 306, the computing system uses the machine-learned prediction model to assign a first event from a plurality of non-identification events to a first pseudo-user based on one or more characteristics of the first event.

[0092] In some embodiments, the first event can be associated with a first characteristic. Further, the computing system can select a subset of similar users from a plurality of identified users based on the first characteristic of the first event. The subset of similar users can be associated with the first characteristic. Further, the computing system can update the event-to-user ratio based on data derived from the subset of similar users. The updated event ratio can be utilized by the computing system to determine the number of pseudo-users in operation 304.

[0093] In some embodiments, the first characteristic is a web browser associated with a first event. Alternatively, in some embodiments, the first characteristic can be the country of origin associated with the first event. Alternatively, in some embodiments, the first characteristic is the display resolution associated with the first event. The machine-learned prediction model can assign a plurality of events having the first characteristic to a pseudo-user and / or a pseudo-session. For example, a plurality of events having a specific display resolution can be assigned to a first pseudo-user, and other events having different display resolutions are filtered out from being assigned to the first pseudo-user because the user does not have different display resolutions during the session.

[0094] In some examples, the first event can be adding an item to a shopping cart on a website, and the subset of similar users are users who have added an item to the shopping cart on the website. In some examples, the first event can be purchasing an item in the shopping cart on the website, and the subset of similar users are users who have purchased an item in the cart on the website. The machine-learned prediction model can determine a journey for a session to predict a correlation between the first event and a previous non-identifying event. The journey of the session can be utilized to predict a correlation relationship between the first event and a subsequent non-identifying event. For example, the journey of the session can include clicking on a product, then adding the product to the shopping cart, and then purchasing the product from the shopping cart. Further, the events can be associated with the journey, including but not limited to, entering credit card information, entering a shipping address, and other events associated with purchasing a product.

[0095] In some embodiments, a subset of similar users may be associated with a first dimension. For example, the first dimension may be a non-observable dimension such as the first visit date. In other examples, the first dimension may be the gender associated with the first pseudo-user. Further, the computing system can use a machine-learned prediction model to assign the first dimension to the first pseudo-user based on the first characteristic. In some examples, the first dimension may be the first visit date associated with the first event, and the first dimension is assigned to the first pseudo-user based on a probability distribution derived from a plurality of identification events. For example, the probability distribution can indicate that the first visit date is likely to be a certain number of days ago, and that date can be assigned as the first visit date of the first pseudo-user. Further, the analysis report can provide information derived from the first dimension assigned to the first pseudo-user.

[0096] At 308, the computing system can generate an analysis report of the website. In some examples, the computing system can generate an analysis report of a mobile application. The analysis report can include the number of pseudo-users and information derived from the first event assigned to the first pseudo-user. In some examples, the computing system can generate an analysis report of a mobile application.

[0097] In some embodiments, the method can further include calculating a correction factor based on data obtained from a plurality of days. Further, the method can include updating the number of pseudo-users associated with a plurality of non-identification events based on the correction factor. Further, the method can include determining the number of unique pseudo-users who visited the website over a plurality of days based on the correction factor. Further, the analysis report can include information derived from the number of unique pseudo-users who visited the website over a plurality of days.

[0098] In some embodiments, the method further includes adjusting processing parameters (e.g., processing power, memory capacity, network bandwidth) based on data derived from the analysis report generated at 308.

[0099] FIG. 4 shows a flowchart diagram for assigning events to sessions according to an exemplary embodiment of the present disclosure. FIG. 4 shows steps executed in a particular order for purposes of illustration and explanation, but the method of the present disclosure is not limited to the particular order or arrangement shown. The various steps of method 400 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0100] In some embodiments, the method of operation 400 can be combined with the operations of method 300 and / or method 500. For example, operation 402 can be executed after operation 304 of method 300.

[0101] At 402, the computing system can determine the event-to-session ratio based on data derived from a plurality of identification events associated with the identified user of the website that received the identifier, using a machine-learned prediction model.

[0102] At 404, the computing system can calculate the number of sessions associated with a plurality of un-identified events based on the event-to-session ratio and the total number of un-identified events, using a machine-learned prediction model. Further, the analysis report generated at 308 can include information derived from the number of sessions.

[0103] At 406, the computing system can assign a first pseudo-user to a plurality of pseudo-sessions, using a machine-learned prediction model.

[0104] At 408, based on the first characteristic of the first event, the computing system can assign the first event to a first pseudo - session from a plurality of pseudo - sessions. Further, the analysis report generated at 308 may include information derived from the first pseudo - user assigned to the plurality of pseudo - sessions and the first event assigned to the first pseudo - session.

[0105] At 410, the computing system can determine a series of events based on the configuration of a website. For example, a series of events can include a session where a user starts a journey (i.e., the first event) on the homepage, then the user clicks on a product page (i.e., the second event), then the user selects a product from the product page and adds it to the shopping cart (i.e., the third event), then the user clicks on the checkout page and purchases the selected product (i.e., the fourth event), and then the user enters credit card and shipping information to complete the purchase of the selected product (e.g., the fifth event).

[0106] At 412, the computing system can use a machine - learned model to determine a second event from a plurality of un - identified events based on a series of events. Continuing with the example of product purchase, the computing system can assign both the first and second events to the first pseudo - user, where the first event is opening the homepage of the website and the second event is opening the product page of the website.

[0107] At 414, the computing system can assign a second event to a first pseudo - session using a machine - learned prediction model. Continuing with the example of a product purchase, the computing system can assign both the first event and the second event to the first pseudo - session, where opening a homepage is the first event and opening a product page is the second event. Further, the analysis report generated at 308 can include information derived from the second event that is assigned to both the first pseudo - session and the first pseudo - user.

[0108] In some embodiments, the first event can be associated with a second dimension. Further, the computing system can use a machine - learned prediction model to assign the second dimension to the first pseudo - session. Further, the analysis report generated at 308 can include information derived from the second dimension that is assigned to both the first pseudo - session and the first pseudo - user.

[0109] FIG. 5 shows a flowchart diagram for presenting a report in real - time and then updating the report based on additional information, according to an exemplary embodiment of the present disclosure. FIG. 5 shows steps that are executed in a particular order for purposes of illustration and explanation, but the methods of the present disclosure are not limited to the specifically shown order or arrangement. The various steps of method 500 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0110] In some embodiments, the method of operation 400 can be combined with the operations of method 300 and / or method 400. For example, operation 502 can be executed after operation 308 of method 300.

[0111] At 502, in response to a request received from a client computing device, the computing system can present an analysis report on the display of the client computing device. In some examples, the analysis report exists in real time or near real time. The presentation of the analysis report may be at a time event. Further, a machine-learned prediction model can assign a first event to a first pseudo-user in real time at the time event.

[0112] At 504, the computing system can access additional non-identifying events for a specific period after the time event. For example, the computing system can access all of the non-identifying events for a specific period (e.g., 12 hours, 24 hours). The additional non-identifying events can improve the predictions of the machine-learned prediction model.

[0113] At 506, the computing system can reassign the first event to a second pseudo-user based on the additional non-identifying events. For example, a machine-learned model can make a first prediction in real time regarding the first event using limited information obtained in real time. Further, the machine-learned model can make a more accurate prediction after the time event based on additional non-identifying events captured over a specific period.

[0114] At 508, the computing system can generate an updated report for the website. For example, the updated report can include information derived from the first event that is reassigned to the second pseudo-user.

[0115] At 510, the computing system can replace the analysis report presented on the display at 502 with the updated report.

[0116] FIG. 6 shows an illustration 600 of a graphical user interface 610 presenting a first report and a second report, according to an exemplary embodiment of the present disclosure. In some embodiments, the graphical user interface 610 may present the website owner with the option to view a first report 620 based on available first-party data and / or to view a second report 630 generated by a machine-learned prediction model. The first report 620 may be generated based on data derived from a plurality of identification events associated with an identified user of the website that has received an identifier. The second report 630 can be generated based on the techniques described in FIGS. 3-5. As shown in FIG. 6, the system can seamlessly integrate the modeled data and the observed data in a combined or separate report.

[0117] Additional Disclosure In addition to the above, when and if the systems, programs, or functions described herein may enable the collection of user information (e.g., information about a user's social network, social actions or activities, occupation, user preferences, or a user's current location), and both when it may and when it may not, controls may be provided to the user to enable the user to select whether to send content or communications from the server to the user. Further, certain data may be processed in one or more ways such that information that can identify an individual is removed before the data is stored or used. For example, a user's identifying information may be processed so that information that can identify the individual user cannot be determined, or, if location information is obtained (such as at the city, zip code, or state level), the user's geographical location may be generalized so that the user's specific location cannot be determined. Thus, the user may control what information is collected about the user, how that information is used, and what information is provided to the user.

[0118] The technology described in this specification pertains to servers, databases, software applications, and other computer-based systems, as well as the actions being performed and the information being sent to and from such systems. The flexibility inherent in computer-based systems enables a wide variety of possible configurations, combinations, and partitions of tasks and functions among components. For example, the processes discussed in this specification can be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications can be implemented in a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0119] The subject matter of this disclosure has been described in detail with respect to its various specific and illustrative embodiments, but each example has been provided for explanatory purposes and is not intended to limit the disclosure. Those of ordinary skill in the art, upon reaching the foregoing understanding, can readily create modifications, variations, and equivalents to such embodiments. Accordingly, the disclosure of the subject matter does not exclude including such modifications, variations, and / or additions to the subject matter that would be apparent to those of ordinary skill in the art. For example, features illustrated or described as part of one embodiment can be used in other embodiments to create still further embodiments. Accordingly, the disclosure is intended to cover such modifications, variations, and equivalents.

Claims

1. A method implemented on a computer, comprising: accessing, by one or more computing devices, a plurality of unidentifiable events, wherein the plurality of unidentifiable events has a total number of unidentifiable events, and each event of the plurality of unidentifiable events is associated with one or more characteristics; using a machine-learned prediction model to calculate the number of pseudo-users associated with the plurality of unidentifiable events based on the ratio of events to users and the total number of the unidentifiable events; using the machine-learned prediction model to assign a first event from the plurality of unidentifiable events to a first pseudo-user based on the one or more characteristics of the first event; and generating an analysis report of a website, the analysis report including information derived from the number of the pseudo-users and the first event assigned to the first pseudo-user. A method implemented on a computer.

2. The method implemented on a computer according to claim 1, wherein the unidentifiable events are associated with events without user identifiers, and the machine-learned prediction model determines the ratio of events to users based on data derived from a plurality of identifiable events associated with a plurality of identifiable users who received identifiers when browsing the website.

3. The method implemented on a computer according to claim 2, wherein the data derived from the plurality of identifiable events is first-party data from the website.

4. The first event is associated with a first characteristic, and the method further comprises: selecting, based on the first characteristic of the first event, a subset of similar users from the plurality of identifiable users, the subset of similar users being associated with the first characteristic; and updating the ratio of events to users based on data derived from the subset of similar users. The method implemented on a computer according to claim 2 or claim 3.

5. The method implemented on a computer according to claim 4, wherein the first event is adding an item to the shopping cart of the website, and the subset of the similar users are users who have added an item to the shopping cart of the website.

6. The method implemented on a computer according to claim 4 or claim 5, wherein the first characteristic is a web browser associated with the first event, a country of origin associated with the first event, or a display resolution associated with the first event.

7. The subset of the similar users is associated with a first dimension, and the method further comprises using the machine-learned prediction model to assign the first dimension to the first pseudo-user based on the first characteristic, The method implemented on a computer according to any one of claims 4 to 6, wherein the analysis report further comprises information derived from the first dimension assigned to the first pseudo-user.

8. The method implemented on a computer according to claim 7, wherein the first dimension is the first visit date associated with the first event, and the first dimension is assigned to the first pseudo-user based on a probability distribution derived from the plurality of identification events.

9. The method further comprises using the machine-learned prediction model to calculate the number of sessions associated with the plurality of non-identification events based on the ratio of events to sessions and the total number of the non-identification events, The method implemented on a computer according to any one of claims 1 to 8, wherein the analysis report further comprises information derived from the number of sessions.

10. The method implemented on a computer according to claim 9, wherein the machine-learned prediction model determines the ratio of events to sessions based on data derived from a plurality of identification events associated with an identified user of the website who received an identifier when browsing the website.

11. The method further comprises using the machine-learned prediction model to assign the first pseudo-user to a plurality of pseudo-sessions, and assigning the first event to a first pseudo-session from the plurality of pseudo-sessions based on a first characteristic of the first event. The method implemented on a computer according to claim 10, wherein the analysis report further includes information derived from the first pseudo-user assigned to the plurality of pseudo-sessions and the first event assigned to the first pseudo-session.

12. The method includes: determining a series of events based on the configuration of the website; using the machine-learned model to determine a second event from the plurality of non-identified events based on the series of events; using the machine-learned prediction model to assign the second event to the first pseudo-session, The method implemented on a computer according to claim 11, wherein the analysis report further includes information derived from the second event assigned to the first pseudo-session and the first pseudo-user.

13. The first event is associated with a second dimension, and the method further includes: using the machine-learned prediction model to assign the second dimension to the first pseudo-session; The method implemented on a computer according to claim 11 or claim 12, wherein the analysis report further includes information derived from the second dimension assigned to the first pseudo-session and the first pseudo-user.

14. The method includes: presenting the analysis report on a display of the client computing device in response to a request received from the client computing device, the method implemented on a computer according to any one of claims 1 to 13.

15. The machine-learned prediction model assigns the first event to the first pseudo-user in real time at a time event, and the method includes: accessing additional non-identified events during a specific period after the time event; reassigning the first event to a second pseudo-user based on the additional non-identified events; generating an updated report of the website, the updated report including information derived from the first event reassigned to the second pseudo-user, and Replacing the analysis report presented on the display with the updated report, further comprising the computer-implemented method according to claim 14.

16. The computer-implemented method according to any one of claims 1 to 15, wherein the machine-learned prediction model is a regression model.

17. The method is Calculating a correction factor based on data obtained from a plurality of days, and Updating the number of pseudo-users associated with the plurality of non-identifying events based on the correction factor, further comprising the computer-implemented method according to any one of claims 1 to 16.

18. The method is Determining the number of unique pseudo-users who visited the website over the plurality of days based on the correction factor, and The analysis report further includes information derived from the number of unique pseudo-users who visited the website over the plurality of days, the computer-implemented method according to claim 17.

19. A computing system, One or more processors, and One or more non-transitory computer-readable media, the one or more non-transitory computer-readable media are A machine-learned prediction model configured to generate an event-to-user ratio based on data derived from a plurality of identifying events associated with identified users of a website that received an identifier, a machine-learned prediction model, When executed by the one or more processors, collectively store instructions that cause the computing system to perform operations, the operations being Accessing a plurality of non-identifying events, the plurality of non-identifying events having a total number of non-identifying events, each event of the plurality of non-identifying events being associated with one or more characteristics, accessing, Using the machine-learned prediction model to calculate the number of pseudo-users associated with the plurality of non-identifying events based on the event-to-user ratio and the total number of non-identifying events, Using the machine-learned prediction model to assign a first event from the plurality of non-identifying events to a first pseudo-user based on the one or more characteristics of the first event, and Generating an analysis report of the website, the analysis report including information derived from the number of the pseudo-users and the first event assigned to the first pseudo-user, and generating. A computing system including this.

20. One or more non-transitory computer-readable media including instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations including: Accessing a plurality of non-identifying events, the plurality of non-identifying events having a total number of non-identifying events, and each event of the plurality of non-identifying events being associated with one or more characteristics, and accessing. Using a machine-learned prediction model to calculate the number of pseudo-users associated with the plurality of non-identifying events based on the event-to-user ratio and the total number of the non-identifying events. Using the machine-learned prediction model to assign a first event from the plurality of non-identifying events to a first pseudo-user based on the one or more characteristics of the first event, and Generating an analysis report of the website, the analysis report including information derived from the number of the pseudo-users and the first event assigned to the first pseudo-user, and generating. One or more non-transitory computer-readable media including this.

Images