Privacy-preserving identity resolution using page content

Abstract

Various embodiments discussed herein are directed to improving existing techniques by extracting or detecting identifiers from pages regardless of the device or platform the user is using, and then anonymizing such identifiers in order to determine whether the same user has accessed different pages, or whether the current user in a current computer session is the same user as in a past computer session.

Description

[Background technology]

[0001] background Various computer-implemented technologies track user devices and activities over computer networks across various web sessions. For example, some technologies use cookies, which are typically small text files stored on a user device by a web browser at the request of a particular web server. Cookies can be used to perform various actions, such as remembering and maintaining user preferences for websites, remembering login details, and determining who is a new or returning visitor. Summary of the Invention

[0002] overview This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used solely as an aid in determining the scope of the claimed subject matter.

[0003] Various embodiments discussed herein are directed to improving upon existing techniques by extracting or detecting one or more identifiers (e.g., phone numbers, email addresses, or credit card numbers) from a page (e.g., a web page) regardless of the device or platform the user is using, and then anonymizing (e.g., hashing) such potentially unique identifiers to determine whether the same user accesses different pages across different computer sessions. For example, during a first computer session, a first user may access mobile application A on their mobile phone. During such first computer session, the user may enter a shipping address on the first app page while electronically purchasing a product. Subsequently, during a second computer session, the same first user may visit web application C using web browser D hosted on their desktop computer. During this second computer session, the first user may enter the same shipping address on the web page.

[0004] Existing technologies cannot detect that the same first user was involved in these different computer sessions because, among other reasons, these computer sessions are device or platform specific. However, certain embodiments described herein anonymize (e.g., generate a one-way hash of) the delivery address (and / or other potentially unique identifiers). Such anonymized identifiers function as pseudonymous identifiers that remain consistent across all platforms and devices without explicitly identifying the user. These anonymized values ​​allow certain embodiments to associate the first and second computer sessions without ever knowing the actual delivery address or identities of the associated users. Some embodiments generate a score indicating the probability that the same user accessed page content from multiple sessions, based at least in part on comparing the anonymized identifiers detected across the multiple sessions. Such functionality has the technical effects of improved process security, improved accuracy or reduced error rates for tracking users, increased scalability, improved human interaction, and improved computer resource consumption, among other technical effects.

[0005] BRIEF DESCRIPTION OF THE DRAWINGS The present invention will now be described in detail with reference to the accompanying drawings. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary computing architecture suitable for implementing some embodiments of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram illustrating how hashes are generated for different pages accessed by a user across different computer sessions, according to some embodiments. [Figure 3] FIG. 10 is a block diagram illustrating exemplary inputs and outputs for generating a probability score that multiple computer sessions involved the same user, according to some embodiments. [Figure 4] 10 is a screenshot of an example page showing certain field values ​​that can be hashed, according to some embodiments. [Figure 5] FIG. 1 is a block diagram of a modified BERT model or encoder that uses a specific input to predict certain natural language characters and detects specific fields that a page contains, according to some embodiments. [Figure 6] FIG. 1 is a schematic diagram illustrating how a neural network makes a particular training and deployment prediction given a particular input, according to some embodiments. [Figure 7] FIG. 1 is a schematic diagram of an exemplary network graph, according to some embodiments. [Figure 8] FIG. 10 is a flow diagram of an example process for training a machine learning model on various pages, according to some embodiments. [Figure 9] FIG. 10 is a flow diagram of an example process for generating a score indicative of the probability that the same user accessed a first page and a second page or accessed content from different computer sessions, according to some embodiments. [Figure 10] FIG. 1 is a block diagram illustrating an exemplary operating environment suitable for implementing some embodiments of the present disclosure. [Figure 11] FIG. 1 is a block diagram of an example computing device suitable for use in implementing some embodiments described herein. DETAILED DESCRIPTION OF THE INVENTION

[0007] Detailed Description To satisfy statutory requirements, the present specification provides specific descriptions of aspects of the present disclosure. However, such descriptions are not intended to limit the scope of this patent. Rather, the inventors contemplate other ways in which the claimed subject matter may be embodied, including different steps or combinations of steps similar to those described herein, in combination with other current or future technologies. Furthermore, although the terms "step" and / or "block" may be used herein to refer to different elements of the method employed, these terms should not be construed as implying any particular order among or between the various steps disclosed herein unless and except where the order of individual steps is explicitly recited. Each method described herein may include computational processes that may be performed using any combination of hardware, firmware, and / or software. For example, various functions may be performed by a processor executing instructions stored in memory. The method may also be embodied as computer-usable instructions stored on a computer storage medium. The method may be provided as a standalone application, a service, or a hosted service (standalone or in combination with another hosted service), or as a plug-in to another product, to name a few examples.

[0008] Existing tracking technologies, such as third-party cookies, are becoming outdated. As mentioned above, cookies can be used to track user devices. There are several types of cookies, including first-party cookies and third-party cookies. "Third-party" cookies are created and placed by web applications other than the one the user is visiting to perform specific actions, such as cross-site tracking, ad serving, and retargeting. Third-party cookies typically function by embedding JavaScript from one web application into another. Third-party cookies store data remembered between browsing sessions. First-party cookies, on the other hand, are generated by the host domain the user is currently visiting. However, due to various concerns, such as user privacy, various technologies are eliminating third-party cookies (known as "cookie deprecation") and other data objects. This is because user IDs are typically derived directly and these cookies track users from one network site to another, which many users find annoying.

[0009] Without cookies, existing technologies incorporate features associated with reduced accuracy or increased error rates when it comes to tracking users. For example, some technologies consider only static metadata such as the user device's IP address, user agent data (i.e., web browser name, operating system ID, and device type), or the screen resolution of the device the user is currently using. However, these and some cookie-based technologies provide very low accuracy in tracking users. This is because, with the advent of unique user devices (e.g., smartwatches, wearable devices, mobile phones, etc.), the same user tends to navigate the web using different devices or different platforms (e.g., different operating systems, different browsers, web applications that run on a browser versus mobile applications that do not require a browser, etc.). For example, solutions that simply track the user device's IP address may not be able to track users across different sessions because the user may use different devices (and therefore different IP addresses) across multiple sessions. In another example, for users using local native mobile applications, user agent data may not be extractable because the user must use a web browser. This can lead to further inaccuracies depending on the application's use case, such as sending irrelevant ads for retargeting, reduced ability to maintain user preferences for specific websites, reduced ability to determine if a user is a new or returning visitor, and reduced ability to remember login details (e.g., usernames and passphrases).

[0010] Relatedly, existing technologies are also not scalable. To track users, web application owners typically must configure their own data objects (e.g., cookies) to fit specific platform or other parameters. For example, a data object that records user activity may require the user to use a specific web browser because the data object resides with (or communicates with) that particular web browser. In another example, some data objects only record user activity using a particular operating system. Thus, tracking users is not scalable to larger numbers of users outside these parameter limits.

[0011] Because some of these technologies are not scalable or require the configuration of certain parameters, these solutions still require manual user input, which not only negatively impacts user interaction but also unnecessarily consumes computer resources. For example, users using local mobile apps are often required to re-enter their login credentials for each session because cookies or other data object technologies cannot track user activity when the user is not using the browser. Consequently, users are often left frustrated by having to repeatedly enter such information, often negatively impacting the user experience. At the computational level, such manual computer user input incurs repetitive I / O costs (e.g., excessive physical read / write head movement on non-volatile disks) because each time an operating system or other component processes a user request to store or access user credentials (or other information, such as user preferences), the user device must contact storage devices to perform read or write operations, and such operations are expensive because they are error-prone and eventually wear out components such as read / write heads. Furthermore, each repetitive input also requires the data to be stored in memory, thereby unnecessarily consuming storage space.

[0012] As described herein, various embodiments of the present disclosure provide one or more technical solutions having technical effects in light of these technical problems as well as other problems. For example, certain embodiments are directed to extracting identifiers from page content, regardless of the device or platform the user is using, and then anonymizing (e.g., hashing) such identifiers to determine whether the same user accesses different pages across different computer sessions. In some implementations, the identifiers are unique or substantially unique to a user. For example, during a first computer session, a first user may visit mobile application A on their mobile phone that includes operating system B. During such first computer session, the user may subscribe to mobile application A by entering their email address (e.g., John.doe@email.com) on the first app page. Subsequently, during a second computer session, the same user (first user) may visit web application C using web browser D hosted on their desktop computer. During this second computer session, the first user may purchase a product and use the same email address (e.g., John.doe@email.com) on the web page. With existing technology, there is no way to connect a web browser D / web application C with a first user's cell phone / mobile application A (or operating system B). However, certain embodiments anonymize (e.g., generate a one-way hash of) the email address (among other data). Such anonymized value then serves as a pseudonymous identifier that remains consistent across all platforms and devices. These anonymized values ​​allow certain embodiments to piece together a first computer session and a second computer session without ever knowing the actual email addresses or identities of the associated users.In other words, certain embodiments generate a score indicative of the probability that the same user accessed page content from multiple sessions based at least in part on comparing anonymized identifiers across multiple sessions.

[0013] Certain embodiments have the technical effect of improving user privacy or improving process security compared to existing techniques. In some embodiments, this is because cookies are not used at all. Furthermore, unlike certain cookies, certain embodiments do not directly derive a user ID. Therefore, one technical solution is the concept of extracting specific values ​​of specific fields, or unique identifiers associated with a user that do not directly identify the user but are still unique enough to not be redundant across many users. For example, in some examples, such unique identifiers or values ​​include a specific user's email address, a specific user's zip code, a specific user's mailing address, a credit card number, and / or a specific user.

[0014] Another technical solution for improving process security is the concept of causing the anonymization of such values ​​or unique identifiers (e.g., by encryption, obfuscation, or masking). Thus, sensitive user information, such as phone numbers, email addresses, or credit card information, cannot be discovered (or is unlikely to be discovered). As a result, a cyber attacker cannot programmatically access resources (e.g., bank accounts or other accounts) because the anonymized data is too obfuscated to make sense. Another related technical solution for improving process security is the concept of performing the actual anonymization in a client application (e.g., a web browser, extension, or app plug-in) on the user device. In other words, in some embodiments, the user device performs the anonymization instead of the remote server. In this way, plaintext sensitive information, such as credit card numbers, is not transmitted over a computer network to the remote server. Thus, it can be stored on the remote server in an anonymized form, as opposed to plaintext, making it impossible or unlikely for a cyber attacker to intercept the plaintext data during transmission or reducing the likelihood of the data being discovered.

[0015] Certain embodiments also improve the accuracy or error rate of tracking users compared to existing techniques. Instead of considering only device- or platform-specific static metadata, certain embodiments additionally or alternatively extract page content to track users regardless of the device or platform they are using. Thus, one technical solution is to extract a set of values ​​for one or more fields from a web or app page or to detect a unique identifier. Additionally or alternatively, another technical solution is to generate a score indicating the probability that the same user accessed both a first web or app page and a second web or app page, based at least in part on a comparison between anonymized data (representing the set of extracted values). Such an embodiment has the technical effect of reducing the error rate for tracking users, as extracting page information to track users is device- and platform-independent, meaning that users are more likely to be tracked across multiple computer sessions. Either technical solution, alone or in combination, provides additional precision depending on the application use case, such as sending relevant advertisements for retargeting, improving the ability to maintain user preferences for a particular website, improving the ability to determine if a user is a new or returning visitor, improving the ability to remember login details (e.g., username and passphrase), etc. The reason is that extracting page content to track a user is device or platform agnostic, so that all user activity is captured across all computer sessions, even if the user uses different devices or platforms.

[0016] Certain embodiments are also more scalable than existing technologies. Web application owners do not need to configure their own data objects, such as cookies, to conform to specific parameters. This is because certain embodiments execute code (e.g., JavaScript) when a user is on a site that captures the entire Document Object Model (DOM) (or other data object), which captures the logical structure of a page / document, and serializes the DOM in the order in which the computer user's activities are performed. This approach accurately records what a user viewed and / or did, and does so regardless of the user's platform or device type. Therefore, a technical solution employs the ability to capture the logical structure of a page / document in the order in which the computer user's activities were performed. Therefore, no specific web browser, operating system, etc. is required. Therefore, tracking users is more scalable than existing technologies.

[0017] Certain embodiments also improve human interaction and computer resource consumption based on scalability, or device and platform agnosticism. As described above, certain embodiments are more scalable than existing technologies and therefore require less manual user input, which improves user interaction and reduces computer resource consumption. For example, using the discussion above, certain embodiments track user activity even when the user is not using a particular browser, so a user using a local mobile app does not need to re-enter their login credentials for each session. Thus, the user experience is less redundant because the user does not have to repeatedly enter such information. At the level of computational technical effect, such reduced manual computer user input translates to reduced I / O costs (e.g., fewer physical read / write head movements on non-volatile disks) because the operating system or other component processes fewer user requests to enter user credentials or other information. This means that user devices contact storage devices less often to perform read or write operations. Thus, these embodiments are less prone to errors and less wear on read / write heads or other storage device components. Furthermore, less manual computer user input means less data is written to local memory, thereby increasing storage capacity.

[0018] Turning now to FIG. 1 , a block diagram is provided illustrating aspects of an exemplary computing system architecture suitable for implementing some embodiments of the present disclosure, generally designated as system 100. System 100 represents only one example of a suitable computing system architecture. Other configurations and elements may be used in addition to or instead of those shown, and some elements may be omitted entirely for clarity. Furthermore, like system 100, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location, according to various embodiments.

[0019] Exemplary system 100 is described with respect to FIG. 10 and includes a network 110 that communicatively couples components of system 100, including session recording component 102, field extraction component 104, value anonymization component 106, user computer session stitching component 108, presentation component 120, and storage 105. In some embodiments, these components are embodied as a collection of compiled computer instructions or functions, program modules, computer software services, or configurations of processes running on one or more computer systems, such as computing device 11 described with respect to FIG. 11 and user device 02a and / or server 06 of FIG. 10.

[0020] In some embodiments, the functions performed by the components of system 100 relate to one or more personal assistant applications, services, or routines. Such applications, services, or routines, among other things, may operate on one or more user devices (such as user device 02a in FIG. 10 ), servers (such as server 06 in FIG. 10 ), may be distributed across one or more user devices and servers, or may be implemented in the cloud. Furthermore, in some embodiments, these components of system 100, including one or more servers (such as server 06 in FIG. 10 ) and client devices (such as user device 02a in FIG. 10 ), are distributed across a network, distributed in the cloud, or reside on a user device, such as user device 02a in FIG. 10 . Furthermore, these components, the functions performed by these components, or the services performed by these components are implemented at an appropriate abstraction layer, such as the operating system layer, application layer, and / or hardware layer of a computing system. Alternatively or additionally, in some embodiments, the functionality of these components and / or the embodiments described herein are performed at least in part by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), and complex programmable logic devices (CPLDs). Additionally, although functionality is described herein with respect to specific components shown in exemplary system 100, it is contemplated that in some embodiments the functionality of these components may be shared or distributed across other components.

[0021] Continuing with FIG. 1 , the session recording component 102 is generally responsible for listening to (e.g., capturing or recording) and storing in computer storage (e.g., RAM or disk) various computer user activities and / or various pages (and their contents) that a user interacts with during a given computer session, regardless of device or platform. For example, the session recording component 102 may capture that a user clicks the same button X on page Y a certain number of times, where page Y, its contents, and the user's click on button X are all recorded. As used herein, "computer user activity" refers to any suitable user computer input, such as clicking, browsing, purchasing, scrolling, drilling, keyboard entry (of natural language letters, words, numbers, or symbols), dragging, etc. As used herein, a "page" refers to any suitable instance of an entire computer application workflow, such as a particular web page of a web application, a particular app page of an application (e.g., an "activity" in Android), an instance of a user interface (e.g., a screen), a window in a desktop application, etc. In some embodiments, the session recording component 102 stores such information as individual data records (e.g., database rows) whose key or unique identifier is the session ID so that user privacy is protected. Thus, the session recording component 102 has the authority to listen to the content of the page and the computer user activity performed by the user on that page.

[0022] In some embodiments, the session recording component 102 functions by capturing computer user activities and page interactions from the user's perspective in the order in which the user engages in those activities. As an illustrative example, some embodiments execute JavaScript (or other computer programming language) when a user is on a web application (or other application), which captures the entire Document Object Model (DOM) (or other data object that captures the logical structure of the page / document), serializes the DOM according to the order in which the computer user activities were performed, and then sends it to the server. Doing so accurately captures what the user was looking at and / or doing at any given time. As used herein, "DOM" refers to an application programming interface (API) for HTML and XML web documents. The DOM defines the logical structure of a document and how a user can access and manipulate such a document, such as through computer user activities. A web page is a document displayed in a browser window or as HTML source. In both cases, it is the same document, but the DOM representation allows manipulation of that document. As an object-oriented representation of a web page, the DOM can be modified with scripting languages ​​such as JavaScript.

[0023] An example of a program that includes the functionality of the session recording component 102 according to some embodiments is CLARITY®, developed by MICROSOFT® of Redmond, Washington. CLARITY is a tool that captures how users actually use a website or application. CLARITY automatically generates heat maps for every page to show where people click on a page, what people ignore on a page (e.g., by cool colors as opposed to red, which indicates a high number of clicks), and how far people scroll on a page.

[0024] CLARITY also records actual sessions to understand how a user engages in computer user activities (from the user's perspective) step-by-step across sessions. In other words, CLARITY records / stores each page (and all of its content) that a user accesses, how the user interacts with that page (e.g., selecting specific buttons), when the user performs those computer user activities, and where on the page the user interacts—all in the order in which the user performs them. In other words, recording an actual computer session involves recording multiple visual snapshots or screenshots of a user's entire session, including each page the user navigates to and all of the computer user activities the user performs on each page. Consequently, in some embodiments, the field extraction component 104 “replays” such actual recordings to see what the user was looking at and doing in terms of computer user activities, i.e., to detect the respective values ​​of each page the user navigates to. CLARITY also provides other powerful insights, such as “dead clicks” (e.g., clicks that have no effect) and “rage clicks” (e.g., when the user rapidly clicks within the same area or UI element).

[0025] In some embodiments, the session recording component 102 performs its function in an alternative or additional manner. For example, some embodiments use any suitable software tracking tool. In another example, in some embodiments, the session recording component 102 is included as a browser extension and records session data across all web sessions.

[0026] The field extraction component 104 is generally responsible for performing a computer search on pages recorded by the session recording component 102 and / or detecting specific content (e.g., fields and / or corresponding values ​​or unique identifiers) on such pages. For example, during a first session, a user may have entered a phone number into the page. The field extraction component 104 then detects the phone number and predicts, via a probability score, that it is a phone number as opposed to another type of value, such as a credit card. In some embodiments, the field extraction component 104 selects only certain content of a page for downstream processing (by components 106 and 108) while ignoring other pieces of content on the same page. For example, in some embodiments, to select which values ​​to hash by the value anonymization component 106, the field extraction component 104 searches for and / or selects content that is more indicative of a unique identifier for a user, while non-extracted content is more likely to be shared by multiple users or non-identical users across two or more sessions. For example, in some embodiments, the field extraction component 104 searches for and detects credit card numbers, phone numbers, email addresses, etc., entered by a user throughout their computer user activity, which is likely to be unique to a single user and consistent across sessions. As described in more detail below, such information is passed to the value anonymization component 106 and the user computer session stitching component 108.

[0027] In some embodiments, the field extraction component 104 further ignores other field values ​​on the page, such as the "total cost" amount, date, etc., for downstream processing because these values ​​do not indicate the user's identity and are unlikely to be entered or consistent across future sessions. Computer applications and pages can vary widely and may contain unique information. However, certain fields and values ​​tend to be consistent across applications, such as the requirement that a user enter a credit card number, phone number, email address, etc. Thus, in some embodiments, the field extraction component 104 searches for or selects these fields and / or unique identifiers for downstream processing and ignores or does not search for other fields and / or values.

[0028] In some embodiments, as described in more detail below, the field extraction component 104 performs its function through one or more machine learning models to learn which particular fields and / or values ​​to extract and / or which particular fields / values ​​to ignore based on training the models to understand what particular fields or values ​​look like and / or where different fields / values ​​tend to be located (e.g., by DOM structure). In this way, the machine learning models can learn to distinguish between unique identifiers. For example, as described in more detail below, some embodiments use a Bidirectional Encoder Representation from Transformers (BERT) model that has been fine-tuned to recognize that particular fields and / or values ​​belong to or are components of individual fields.

[0029] In some embodiments, the field extraction component 104 assigns the type of site or page the user visited, which dictates which fields and / or values ​​the field extraction component 104 searches for or considers. For example, in response to receiving an indication that the user visited an electronic marketplace web application or page (e.g., a checkout page), the field extraction component 104 searches for a "credit card" number value / field, a "shipping address" field / value, and / or a "billing address" value / field. Such functionality is thus governed by programmatic rules and conditions that trigger the search for specific content.

[0030] In some embodiments, pages recorded by the session recording component 102 include attachments (e.g., PDF files) or other data objects that are not immediately machine-readable. In these embodiments, the field extraction component 104 performs optical character recognition or other computer vision functions via an optical character recognition (OCR) module (not shown) to convert human natural language text into computer-readable text. In the context of OCR, the field extraction component 104 detects natural language characters (e.g., words and numbers) within the document and converts such characters into a machine-readable format so that further analysis can occur with the value anonymization component 106 and the user-computer session stitching component 108. As used herein, a "document" refers to a digital image or other data object that includes one or more natural language characters, such as numbers, English words, or English phrases. In some examples, a document represents or initially exists as a real-world piece of paper / document or other medium that has been copied or scanned.

[0031] In some embodiments, the processor executing the OCR module converts one or more raw documents into another format (e.g., via PymuPDF) in preparation for further processing by the OCR module. For example, the OCR module converts any document that is not an image (e.g., a JPEG document) into an image of black and white pixel values ​​(e.g., a bitmap image), in which dark areas are identified as characters that need to be recognized and light areas are identified as background). Often, certain formats (e.g., JPEG) make it difficult to extract information and / or the original format consumes unnecessary computer memory resources. Therefore, data can be converted from one format to another to extract all document features, conserve memory resources, etc., while maintaining high image quality. For example, the OCR module can automatically convert a multi-page PDF document into an image file output format (e.g., JPG, PNG, BMP, TIFF, etc.) via a PDF-JPG converter, for example. JPEG is a file format that can contain images using lossy image compression techniques of 4:1 to 10:1. Through this compression technique, embodiments reduce the image size of images contained within a PDF document without compromising image quality.

[0032] In some embodiments, the OCR module additionally or alternatively performs image quality functions that modify the appearance of the document by converting a color document to grayscale, performing desaturation (removing color), modifying brightness, modifying contrast to correct contrast, etc. In response, in some embodiments, the OCR module executes a computer process that rotates the document image to a uniform orientation, a process called "deskewing" the image. Occasionally, documents uploaded by users are slightly rotated or flipped in the vertical or horizontal plane and at various angles such as 45°, 90°, etc. Accordingly, some embodiments deskew the image to reorient it to a uniform orientation (e.g., straight edges or landscape orientation). In some embodiments, in response to the deskew operation, some embodiments remove background noise (e.g., by Gaussian and / or Fourier transforms). Often, when a document is uploaded, such as by scanning or photographing from a camera, it is common for the resulting image to contain unwanted dots or other marks due to printer malfunctions. To separate it from this meaningless noise interference, some embodiments clean the image by removing these marks. In response to removing the background noise, some embodiments extract characters from the document image and place the extracted characters in another format, such as JSON, which can be used as input for other machine learning models, such as a lightweight modified BERT model for language prediction, as described in more detail below.

[0033] In some embodiments, the field extraction component 104 additionally (or instead) includes an object detection module for detecting documents or objects (e.g., a "credit card" field) within a document or page (e.g., via a machine learning model). For example, in some embodiments, in response to (or before) the OCR module performing its function, text and object detection can be performed to convert 2D grayscale images into structured text associated with their bounding boxes and coordinates. For example, some embodiments use computer vision, object detection, and / or image classification techniques (e.g., by implementing a convolutional neural network (CNN)). In an illustrative example of the object detection function, certain embodiments use one or more machine learning models (e.g., CNNs) to generate bounding boxes that define boundaries and encompass computer objects representing document features (e.g., phone number values, email address values, etc.) or the document itself. These machine learning models can also generate classification predictions that computer objects are specific features. In computer vision applications, the output of object detection can be encompassed by bounding boxes.

[0034] A bounding box describes or defines the boundaries of an object in terms of its location (e.g., 2D or 3D coordinates) (as well as the height and width of the bounding box). For example, a bounding box can be a rectangular box determined by its x-axis and y-axis coordinates. Such bounding boxes provide an object recognition system with an indication of spatial distinctions between objects that aid in detecting objects within a document or the document itself. In an illustrative example, a first bounding box is generated over the front of a document, which triggers image capture or OCR of the document. In another example, a second bounding box is generated over a "total amount" representation and labeled "Amount," and a third bounding box is generated over an object (e.g., an image of a mountain) and labeled "Mountain," e.g., represented by a logo.

[0035] In some embodiments, the processor executing the OCR module of the field extraction component 104 detects or extracts each word and / or number in a document using any suitable function, such as pattern recognition and / or feature detection. For pattern recognition, the OCR module is provided with examples of natural language text in various fonts and formats, which are used to compare and recognize characters in a scanned document. For feature detection, the OCR module applies rules regarding specific letter or number characteristics to recognize characters in a scanned document. Features may include angled lines, crossed lines, curved lines, etc. within a character for comparison. For example, a capital "A" may be stored as two diagonal lines intersecting a horizontal line running through the middle. In some embodiments, the output of the OCR module is each extracted field and / or value in the document and the address (e.g., X and Y coordinates) of each extracted value and / or field in the document. As used herein, a "field" refers to a category, attribute, column, or data type that is typically expressed in natural language (but in some cases is invisible). For example, a document may include a "phone number" field and corresponding value. A "value" is a particular natural language indicator, numeric value, and / or symbol that belongs to or is a component of a particular field. For example, the phone number value for the "Phone Number" field described above might be "480-276-7179." In some embodiments, the output of the OCR module is fields and values ​​for further downstream processing, as described in more detail below.

[0036] The value anonymization component 106 is generally responsible for anonymizing the content extracted by the field extraction component 104. "Anonymized" data is data in which at least one value has been changed, such that the output data is substantially different from the input but still represents the input. For example, an email address of John.smith@email.com would be hashed to the value "8acsqfn3v," which still represents the email address even though the payload value is entirely different. An anonymized value represents any value that has been obfuscated, encrypted, hashed, or otherwise modified. Some embodiments bucket the data after anonymization. Bucketing data means reserving a certain amount of memory to hold a single data item (e.g., a single hash value) or multiple data items (e.g., combined hashes).

[0037] In some embodiments, the value anonymization component 106 anonymizes the page content using any suitable hashing algorithm, such as a one-way hash. A one-way hash function (e.g., a message digest) is a mathematical function that takes a variable-length input string and converts it into a fixed-length binary sequence that is computationally difficult to reverse, i.e., to generate the original string from the hash. In other words, a one-way hash function is a mathematical function that generates a fingerprint of the input, but there is typically no way to calculate the original input. If the input is the same, the hash is always the same. If the input changes even slightly, even by a single character, the output hash is typically different. In this way, the hash cannot be linked to a user. In some embodiments, a hash is used to verify that input A is identical to input B, but cannot be used to recover the input from the output like encryption, which is another mathematical function. Exemplary hashing algorithms included by the value anonymization component 106 are checksums, universal hash function families, cyclic redundancy checks, non-cryptographic hash functions, keyed cryptographic hash functions, and unkeyed cryptographic functions.

[0038] In some embodiments, the value anonymization component 106 resides on the server side and / or the client side. For example, in some embodiments, the value anonymization component 106 causes anonymization by instructing a client application on a user device to perform the anonymization over the computer network 110, but the value anonymization component 106 does not perform the anonymization itself. In other examples, the value anonymization component 106 causes anonymization by directly performing the anonymization function itself (e.g., on the client side).

[0039] The user computer session stitching component 108 is generally responsible for generating a score (e.g., a confidence score) that indicates the probability that the same user accessed content in different computer sessions (or different pages regardless of computer session). In some embodiments, such functionality is based at least in part on comparing hashes of different pages and sessions, as performed by the value anonymization component 106. Thus, in some embodiments, the user computer session stitching component 108 makes a programmatic call to the value anonymization component 106, which then fetches data records from computer storage that indicate the different sessions and hash values, and then returns the records or hash values ​​to the user computer session stitching component 108 for comparison.

[0040] In some embodiments, as described in more detail below, the user computer session stitching component 108 additionally or alternatively generates its scores using one or more machine learning models. For example, in some embodiments, a machine learning model takes as input a combination of specific field values ​​(e.g., phone number, credit card number, zip code, email address, residential address) and other signals such as IP address and screen resolution for each page of multiple computer sessions to learn node activation / deactivation weights for predicting that the same user has accessed content across multiple computer sessions. For example, in multiple sessions representing the travels of a single user, the model learns that the field (or a hash thereof) that changes least between different sessions (most indicative of user identity) is the phone number (activating the node), whereas other fields, such as the IP address, change significantly between sessions (deactivating the node) because the same user tends to use different devices across multiple sessions. Thus, nodes representing phone numbers are weighted highly for prediction, and nodes representing IP addresses are weighted less. In other words, the same phone number entered by a user across multiple computer sessions is more suggestive that the same user accessed the content from both computer sessions, while different IP addresses detected from multiple sessions are less suggestive that different users accessed the content from different computer sessions.

[0041] Alternatively or additionally, in some embodiments, the user computer session stitching component 108 generates its score at least in part by using an arithmetic, difference, or overlap algorithm that indicates the difference between the anonymized values. For example, in some embodiments, the user computer session stitching component 108 uses a Jaccard index to determine the character overlap (i.e., union) between the compared anonymized values. For example, using the discussion above, if each character in the hash "Hext5" is compared to each character in the hash "Hext5Rj9804," the overlap or similarity is a total of five characters (i.e., "Hext5") between the two hashes. The greater the character overlap or union (Jaccard index) between the multiple anonymized values, the higher the score or probability that the same user accessed the content across multiple computer sessions.

[0042] The exemplary system 100 also includes a presentation component 120 that is generally responsible for presenting content and related information, such as web or app pages and their site map elements (e.g., as illustrated by screenshot 400 in FIG. 4 ), to a user. In some embodiments, the presentation component 120 includes one or more applications or services on a user device, across multiple user devices, or in the cloud. For example, in one embodiment, the presentation component 120 manages the presentation of content to a user across multiple user devices associated with that user. Based on content logic, device capabilities, associated logical hubs, the user's inferred logical location, and / or other user data, the presentation component 120 may determine the context of the presentation, such as on which user device to present the content, and how (or how much content, in what format, which may depend on the user device or context) and / or when to present it. In particular, in some embodiments, the presentation component 120 applies content logic to device capabilities, associated logical hubs, inferred logical location, or sensed user data to determine how to present the content.

[0043] In some embodiments, presentation component 120 generates (or causes the generation of) user interface features associated with the page. Such features may include user interface elements (such as graphical buttons, sliders, menus, audio prompts, alerts, alarms, vibrations, pop-up windows, notification or status bar items, in-app notifications, or other similar features for interfacing with the user), queries, and prompts. In some embodiments, a personal assistant service or application working with presentation component 120 determines when and how to present the content. In such embodiments, the content, including content logic, can be understood as recommendations to presentation component 120 (and / or the personal assistant service or application) about when and how to present notifications, which can be overridden by the personal assistant app or presentation component 120.

[0044] Exemplary system 100 also includes storage 105. Storage 105 generally stores information including data, computer instructions (e.g., software program instructions, routines, or services), data structures, and / or models (e.g., neural network 605 and modified BERT model of FIG. 5 ) used in embodiments of the technology described herein. In some embodiments, storage 105 represents any suitable data repository or device, such as a database, a data warehouse, RAM, a cache, a disk, a RAID, and / or a storage network (e.g., a storage area network (SAN)). In some embodiments, storage 105 contains data records (e.g., database rows) containing any suitable information described herein, such as session data stored by session recording component 102, predefined fields parsed by field extraction component 104, hash values / unique identifiers in anticipation of records being requested over computer network 110 and retrieved for hash comparison by user-computer session stitching component 108, etc. As described herein, in some embodiments, each record is called up or requested and returned over computer network 110 depending on the component that needs it.

[0045] By way of example and not limitation, the data contained within storage 105 may be generally referred to throughout as data. In some embodiments, data may include mobile device location information, smartphone data (such as phone status, charging data, date / time, or other information derived from a smartphone), computer user activity information including computer user activity occurring on multiple user devices (e.g., app usage, online activity, searches, voice data such as automatic speech recognition, activity logs, communication data including calls, texts, instant messages, and emails, website posts, other records related to events, or other activity-related information), user history, session logs, application data, contact data, record data, notification data, social network data, news (including popular or trending items on search engines or social networks), home sensor data, appliance data, global positioning system (GPS) data, and the like. Any such data sensed or determined from sensors (referred to herein as sensor data) may include, but is not limited to, vehicle signal data, traffic data, weather data (including forecasts), wearable device data, other user device data (which may include device settings, profiles, network connections such as Wi-Fi network data, or configuration data, model number, firmware, or data about the device, e.g., device pairing, such as when a user has a mobile phone paired with a Bluetooth headset), gyroscope data, accelerometer data, other sensor data that may be sensed or detected by sensors, data derived based on other data (e.g., location data that may be derived from Wi-Fi, cellular network, or IP address data), and virtually any other source of data that may be sensed or determined as described herein.

[0046] 2 is a schematic diagram illustrating how hashes are generated for different pages accessed by a user across different computer sessions, according to some embodiments. At a first point in time, a user engages in a first computer session 220. For example, the user logs into a first mobile application (e.g., a banking application) using a mobile device 206, initiating the first computer session 220. Thereafter, while navigating the web application, the user enters various values ​​into a page 202, such as "123-4567" (the user's phone number), "j.doe@email.com" (the user's email address), and "64086" (the user's zip code). In some embodiments, as described in FIG. 1, the session recording component 102 captures value and field information from the page 202, such as "phone number," "email," and "zip code," as well as the specific values ​​entered by the user. Then, in some embodiments, as described in FIG. 1, the field extraction component 104 determines which fields (and corresponding values) are most relevant for hashing and selects fields accordingly, such as the "phone number," "email," and "zip code" fields. In response, in some embodiments, the value anonymization component 106 hashes each value in each field. As shown, the phone number is hashed to "3XtaC4TL," the email address is hashed to "FZTL3RSB," and the zip code is hashed to "B3LS6Fak." In some embodiments, the value anonymization component 106 combines such hashes into a combined hash 204.

[0047] In various embodiments, “combining” hashes includes any suitable function, such as adding hashes, concatenating hashes, generating a new hash from the individual hashes, etc. For example, one example of adding hashes is adding the individual hashes of fields, combining a phone number hash, an email hash, and a zip code hash into one large value (i.e., “3XtaC4TLFZTL3RSBB3LS6Fak”). In this way, the combined hash becomes more unique each time an individual hash is included in combined hash 204. As described below, in some embodiments, combined hash 204 is stored in computer storage (e.g., as an individual data record) for later access and comparison to other hashed values. The user can then end first computer session 220 by any suitable action, such as logging out of the first mobile application.

[0048] At a second time after the first time, the same user using laptop 212 engages in a second computer session 230 (e.g., by logging into a second application, such as a web application, via a web browser). The user navigates to page 208, and the session recording component 102 described in FIG. 1 captures that the user again enters their phone number (123-4567). The field extraction component 140 determines that this field, among other fields, is the only relevant field for hashing, and the value anonymization component 106 hashes the phone number into hash 210 (i.e., "3XTac4TL").

[0049] In response, in some embodiments, user computer session stitching component 108 compares hash 210 to combined hash 204, as described in FIG. 1. As shown in FIG. 2, the phone number hash (i.e., 3XTac4TL) extracted from first session 220 on page 202 is the same as phone number hash 210 extracted from second session 230 on page 208. Thus, in some embodiments, regardless of whether the remainder of combined hash 204 matches hash 210, because combined hash 204 contains the same value as hash 210, user computer session stitching component 108 generates a score indicating that the same user likely visited both pages 202 and 208 and / or was involved in both computer sessions 220 and 230. This contemplates situations where hosting applications, which may be very different, do not require users to enter the same field values. In this case, a single matching hash may alone be associated with a high probability that the same user visited both pages or was part of both computer sessions. In this way, if, for example, only a phone hash is calculated for page 208, a probability score is still calculated. However, adding a credit card hash, an email address hash, etc. (as with combined hash 204) makes the hash more unique, thereby increasing the probability that the content is being accessed by the same user. The more hashed field values ​​that match, the higher the probability or confidence that a particular user was involved in a particular computer session. In this way, particular embodiments generate progressively higher confidence scores for each hash value found when one computer session matches another hash value from another computer session.

[0050] In some embodiments, the more hashes that match, the higher the probability that the same user visited both pages. For example, if an embodiment also detects that the zip code "64086" (i.e., its hash "B3LS6Fak") is also derived from page 208, there is a higher probability that the user visited both pages compared to detecting only the phone number.

[0051] Figure 3 is a block diagram illustrating example inputs and outputs for generating a probability score that multiple computer sessions involved the same user, according to some embodiments. In some embodiments, the field extraction model / layer 304 includes the functionality described with respect to the field extraction component 104 of Figure 1. Similarly, in some embodiments, the anonymization model / layer 306 includes the functionality described with respect to the value anonymization component 106 of Figure 1. In some embodiments, the user computer session stitching model / layer 308 includes the functionality described with respect to the user computer session stitching component 108 of Figure 1.

[0052] At a first point in time, the field extraction model / layer 304 takes the page 302 (e.g., screenshot 400 of FIG. 4 ) as input and extracts a particular value (i.e., the phone number 123-4567) from the page 302. At a second point in time, the anonymization model / layer 306 takes the phone number 123-4567 as input and anonymizes the phone number to “4aXpf82c.” At a third point in time, the user computer session stitching model / layer 308 takes the hash “4aXpf82c” as input and compares it to one or more anonymized values ​​of other field values ​​from previous computer sessions or information entered on previous pages. Based at least in part on the comparison, the user computer session stitching model / layer 308 generates a score that is a probability of X (e.g., 96) that the same user was involved in this computer session (and / or on this page) and previous computer sessions (and / or on different pages). For example, the same hash may have been generated in a previous session, indicating that the same phone number was entered, causing a higher score for the probability that the same user accessed both pages. As described in more detail below, there may be additional or alternative functions for generating the score, such as detecting the same IP address of the device for each computer session, comparisons between other field values, etc.

[0053] FIG. 4 is a screenshot 400 of an example page illustrating certain field values ​​that may be hashed, according to some embodiments. As shown in screenshot 400, the page includes multiple fields and values. For example, the “Membership Number” field includes value 408 (i.e., 123123123123). In some embodiments, the field extraction component 104 of FIG. 1 generates a score indicating that the “Payment Method,” “Shipping Address,” and “Billing Address” fields (and / or their corresponding values) correspond to (e.g., are components of or match) one or more predetermined fields. A “predetermined” field may be any appropriate field learned through training by one or more machine learning models (described in more detail below) and / or a program-designated field that is searched or selected on each page the user navigates. As described herein, such searched fields indicate fields that certain embodiments anonymize for comparison. In this manner, as shown in FIG. 4, some fields, such as 402, 404, and 406, are extracted and anonymized, while other fields, such as field 408, are ignored.

[0054] In some embodiments, determining whether a field "corresponds" to (e.g., matches) a given field includes determining whether the field matches the exact syntax (e.g., character-by-character) of the keyword itself (e.g., by term frequency-inverse document frequency (TF-IDF)). For example, a processor executing field extraction component 104 (FIG. 1) may take the term "email address" (i.e., the given field) as input and find this exact phrase in document 400 that matches. Alternatively or additionally, in some embodiments, determining whether a field corresponds to a given field includes determining whether one or more words in page 400 have the same semantic meaning as the given field, even though they may have different syntax (e.g., different characters and / or added / subtracted words). For example, the given field is "phone number," but no such string with an exact syntax match is found in page 400, although a similar phrase such as "telephone number" may be found in the page. In these embodiments, both "phone number" and "telephone number" are semantically similar.

[0055] As shown in screenshot 400, certain embodiments anonymize field value 402 (i.e., credit card number) to "v66si6t," field value 404 (i.e., phone number) to "gkdnf53x," and field value 406 (i.e., zip code) to "v9a9cw." As noted above, in some embodiments, user computer session stitching component 108 (FIG. 1) combines these hashes or uses any one of these hashes to generate a score indicating whether the same user was part of multiple computer sessions. In some embodiments, rather than hashing the individual field values ​​as shown in screenshot 400, user computer session stitching component 108 first combines each of values ​​402, 404, and 406 (in their natural language or numeric form), for example, and then creates a single hash of all values. Alternatively or additionally, some embodiments convert or encode each of the values ​​402, 404, and 406 into a feature vector and then concatenate the feature vectors for downstream analysis by a machine learning model, as described in more detail below.

[0056] 5 is a block diagram of a modified BERT model or encoder that uses a particular input to predict certain natural language characters and detect the particular fields that a page contains, according to some embodiments. In some embodiments, this model represents or includes the functionality described with respect to the field extraction component 104 of FIG. 1 and / or the field extraction model / layer 304 of FIG. 3.

[0057] First, an input 501 natural language corpus (e.g., various Wikipedia English words or BooksCorpus) is converted into feature vectors and embedded in input embeddings 402 (FIG. 4), and the meaning of each natural language word (e.g., English semantics) is derived during pre-training. In some embodiments, corpus documents such as textbooks, journals, blogs, social media feeds, etc. are ingested by the model to understand the English language.

[0058] In some embodiments, unlike existing long short-term memory (LSTM) models, for example, each word or character in the input 501 is mapped in parallel or simultaneously into an input embedding 502. The input embedding 502 maps words to a feature vector that represents the word. However, the same word (e.g., "apple") in different sentences may have different meanings (e.g., telephone and fruit). Therefore, a positional encoder 504 may be implemented. The positional encoder 504 is a vector that gives context to a word (e.g., "apple") based on the word's position in the sentence. For example, for the message "I just sent the document," since "I" is at the beginning of the sentence, an embodiment may indicate a position in the embedding that is closer to "just" as opposed to "document." Some embodiments generate the positional encoder vector using a sine / cosine function as follows:

number

[0059] After passing the input 501 through the input embedding 502 and applying the positional encoder 504, the output is a word embedding feature vector that encodes positional information or context based on the positional encoder 504. These word embedding feature vectors are then passed to the encoder block 506, where they pass through a multi-head attention layer 506-1 and a feedforward layer 506-2. The multi-head attention layer 506-1 is generally responsible for focusing on or processing certain portions of the feature vector that represent specific parts of the input 501 by generating attention vectors. For example, in a question-answering system, the multi-head attention layer 506-1 determines how relevant the ith word (or a particular word within a block) is to answer a question or how relevant it is to other words in the same block or other blocks, and its output is an attention vector. For every word, some embodiments generate an attention vector that captures the contextual relationship between it and other words within the same sentence, block, and / or line. For a given word, some embodiments calculate a weighted average or aggregate attention vectors of other words that contain the given word (e.g., other words in the same line or block) to calculate a final attention vector.

[0060] In some embodiments, single-head attention involves abstract vectors Q, K, and V that extract different components of a particular word, which are then used to calculate the attention vector for all words using the following formula:

number

[0061] With multi-head attention, there are multiple weighting matrices Wq, Wk, and Wv, and therefore multiple attention vectors Z per word. However, the neural network can only expect one attention vector per word. Therefore, another weighting matrix Wz is used to ensure that the output is still one attention vector per word. In some embodiments, some form of normalization (e.g., batch normalization and / or layer normalization) is performed after layers 506-1 and 506-2 to smooth the loss surface and facilitate optimization while using larger learning rates.

[0062] Layers 506-3 and 506-4 represent residual connection and / or normalization layers, where normalization recenters and rescales or normalizes the data across feature dimensions. Feedforward layer 506-2 is a feedforward neural network applied to all of the attention vectors output by multi-head attention layer 506-1. Feedforward layer 506-2 can convert the attention vectors into a format that can be processed by the next encoder block, or make a prediction at 508. For example, given that a user currently (or previously) types a first natural language sequence, "The due date is...", encoder block 506 predicts that the next natural language sequence (or field type) will be a specific date or specific words based on past documents containing the same or similar language as the first natural language sequence.

[0063] In some embodiments, the encoder block 506 includes pre-training and fine-tuning to learn a language (pre-training) and perform field detection (prediction) at 508 (fine-tuning). In some embodiments, pre-training is performed to understand a language, and fine-tuning is performed to learn a specific task, such as learning answers to a set of questions (in a question-answering system), and / or to learn specific field types on a given page in order to detect fields as described herein.

[0064] In some embodiments, the encoder block 506 learns the language and context of words in pre-training by training simultaneously or at the same time based on two unsupervised tasks (MLM and NSP). Regarding input and output, in pre-training, the natural language corpus of input 501 can be various historical documents such as textbooks, magazines, and periodicals, to output predicted natural language characters in 508 (no field detection at this point). The encoder block 506 takes a sentence, paragraph, or line (e.g., contained in input 501) and replaces random words with a mask. The goal is to output the value or meaning of the masked token. For example, if a line reads "Please [MASK] this document quickly," the predicted value of "mask" is "send." This helps the encoder block 506 understand the bidirectional context of the sentence, paragraph, or line of the document. In the case of NSP, the encoder 506 takes as input two or more elements, such as a sentence, line, or paragraph, and determines, for example, whether the second line in the document actually follows (e.g., is directly below) the first line in the document. This helps the encoder block 506 understand the context across all elements of the document, not just within a single element. Using both of these together, the encoder block 506 derives a superior understanding of natural language.

[0065] In some embodiments, during pre-training, the input to the encoder block 506 is a set (e.g., two) masked sentences (sentences with one or more masks), which may alternatively be partial strings or paragraphs. In some embodiments, each word is represented as a token, and portions of the token are masked. Each token is then converted into a word embedding (e.g., 502). On the output side, a binary output for the prediction of the next sentence is generated. For example, if masked line 2 follows (e.g., was directly below) masked block 1, this component may output, for example, 1. The output is a word feature vector corresponding to the output of the machine learning model function. Therefore, the number of input word feature vectors is the same as the number of output word feature vectors.

[0066] In some embodiments, the initial embedding (e.g., input embedding 502) is constructed from three vectors: a token embedding, a segment or context question embedding, and a positional embedding. In some embodiments, the following functions occur in a pre-training phase: The token embedding is a pre-trained embedding. The segment embedding is the sentence number (including the input 501) that is encoded into a vector (e.g., assuming a top-down and right-to-left approach, first sentence, second sentence, etc.). The positional embedding is a vector representing the position of a particular word within such a sentence, which may be generated by the positional encoder 504. These three embeddings are added or concatenated to generate an embedding vector that is used as input into the encoder block 506. Because all of the vectors are fed into the encoder block 506 simultaneously and the language model needs to preserve some order, the segment embedding and positional embedding are used for temporal ordering.

[0067] In pre-training, the output is typically a binary value C (in the case of NSP) and various word vectors (in the case of MLM). Training minimizes loss (e.g., cross-entropy loss). In some embodiments, all feature vectors are the same size and generated simultaneously. Each word vector can then be passed to the output of a fully connected layer with the same number of neurons as the number of tokens in the vocabulary.

[0068] Some embodiments are further responsible for fine-tuning the encoder block 506 after it has been pre-trained. With respect to input and output, the input 501 currently includes different pages (e.g., web or app pages such as screenshot 400), each of which contains multiple fields and / or values, and the output 508 currently includes predicted field types (detected fields) located within the corresponding pages. Once pre-training is performed, the encoder block 506 is trained on very specific tasks, such as question answering, modified NSP or MLM, or detecting which fields are contained within a page. In a question answering task, the model receives a question about text content (e.g., "Given page X, does this page contain a 'phone number' field?") and marks or tags the beginning and end of the answer within the page within the document (e.g., the phone number "123,4567"). For example, in question answering, some embodiments replace the fully connected output layer of the encoder block 506 used in pre-training with a brand new set of output layers that can output the answer to a given question. Supervised training can then be performed using the question-answering dataset.

[0069] Thus, certain embodiments modify the model for fine-tuning by modifying the input and output layers. That is, for example, the input is changed from masked Sentence 1 and Sentence 2 tokens to "Question" and "Sentence" containing the answer (or potential answer) as tokens. At the output layer, certain embodiments output the start and end words (or characters) that encapsulate the answer (field). In some embodiments, such question-answer pairs are specifically labeled as complete or incomplete (e.g., answered or unanswered).

[0070] In an illustrative example of fine-tuning or inference using encoder block 506, some embodiments may take multiple pages of field values ​​(fine-tuning) in input 501, including phone numbers, and learn that the field (or field type) of which a value is most likely to be a component is likely to be X. For example, given training based on QA pairs, the model may predict that the value "123-4567" is likely to be a phone number or a component of a phone number. In this way, language models can be trained and fine-tuned not only to understand natural language, but also to predict which field or category a particular value belongs to / is a component of.

[0071] It will be appreciated that different models / training processes may be used instead or in addition to the question answering system or model of FIG. 5 . For example, in some embodiments, an annotator (either programmatic or human) is used to label different fields in a fine-tuned dataset of pages, and predictions 508 are based on learned weights associated with different features of the labeled fields. For example, the annotator may label each web or app page with a “phone number” field (capturing different phone numbers), a “credit card” field (capturing different credit card numbers), etc. In this way, the model learns weights or features that indicate particular annotated fields. For example, for pages with multiple annotated phone number fields, the model may learn that these phone number fields always contain three digits followed by a dash followed by four additional digits. Thus, any input that fits this learned pattern will generate a score that is more indicative of the particular field value being a component of a “phone number” field, whereas inputs that contain natural language characters and do not follow one or more of these patterns will be more indicative of another field (or not a phone number).

[0072] Some embodiments additionally or alternatively analyze the payload of the field itself to generate a score for detecting the field / field type. For example, in some embodiments, certain embodiments use a term frequency-inverse document frequency (TF-IDF) algorithm. The TF-IDF algorithm involves numerical statistics that infer how important a query word or term is to a dataset (e.g., a page). "Term frequency" indicates how often a query term (e.g., a given field) appears in the dataset, which is then divided by the length of the dataset (i.e., the total amount of terms in the dataset). "Inverse document frequency" infers how important a term is by reducing the weight of frequently used or common terms such as "the" and "of" that are abundant in the dataset but are of little importance to the relevance of the query. In these embodiments, for example, if a given field is "credit card," these algorithms can use term frequency to find an exact string match (i.e., "credit card") within the page, remove all other words within the page using inverse document frequency, and responsively detect the presence of a "credit card" field on the page based on the exact match found within the page.

[0073] Some embodiments alternatively or additionally use natural language processing (NLP) to find semantically similar words. For example, a given field (the field that an embodiment searches for in a document) might be "Email Address," and a page might contain only one field called "Email." Particular embodiments can determine that these are the same field (or that the "Email" field is also an "Email Address" field because they are semantically similar). For example, some embodiments first tokenize fields on a page into their constituent words, numbers, and symbols, and then tag some or each of the words with a part-of-speech (POS) identifier. In various embodiments, "tokenization" or syntactic analysis corresponds to a computer-implemented process of segmenting content into words, sentences, symbols, character sequences, and / or other elements of content. This can include a set of rules for parsing a message, such as the order of words and / or parts of speech (POS). For example, in the sentence "The girl jumped for joy," the syntax may correspond to the word order in a subject-verb-adverb (or subject, verb, object, etc.) structure. In various embodiments, each word on the page is tagged with an identifier, such as a POS identifier.

[0074] In some embodiments, NLP derives the semantic and syntactic content of semi-structured or unstructured data (e.g., data in an image file). This is in contrast to analyzing "structured" data, such as data in a database. NLP can be configured to parse content to determine semantic context (e.g., the meaning of words by analyzing each word in a document relative to each other and relative to training data) and syntactic context (e.g., a set of rules governing sentence structure in a given language). NLP is configured to recognize keywords, contextual information, and metadata tags associated with one or more portions of a set of data. In certain embodiments, NLP analyzes summary information, keywords, and text descriptions contained in a set of data and uses syntactic and semantic elements present in this information to identify contexts of interest. Syntactic and semantic elements can include information such as word frequency, word meaning, text font, italics, hyperlinks, proper nouns, noun phrases, parts of speech (e.g., nouns, adverbs, adjectives, etc.), and / or the context of surrounding words. Other syntactic and semantic elements are possible.

[0075] In some embodiments, NLP includes named entity recognition (NER). NER is an information extraction technique that identifies elements or "entities" within natural language text and classifies them into predefined categories. Such predefined categories can be indicated in corresponding tags or labels. An entity can be, for example, a person's name, a specific organization, a specific location, a specific time, a specific quantity, a specific monetary value, a specific percentage, a specific page, etc. Similarly, corresponding tags or labels can be a specific person, organization, location, time, price (or other billing data), etc. In the present context of the present disclosure, for example, these tags or labels can indicate whether a certain extracted attribute corresponds to a "credit card" field on a page, the name of an entity (e.g., a specific company), line item information (e.g., a description of the service or item being billed), the address of an entity, or a specific date listed on a page.

[0076] In some embodiments, the field extraction component 104 (FIG. 1) detects particular fields using the distance between both the field name (e.g., "Phone Number") and the corresponding value (e.g., "123-4567"). For example, some embodiments determine the location of each value relative to the field name on the page. For example, some embodiments determine a direction and distance (e.g., cosine distance) of the field name "Total Amount" from the value written as "$200.00." Based on such location and a first rule, some embodiments generate a first score for each value, the first score indicating at least in part whether the individual value is within a threshold location relative to one or more keywords. For example, the value "$200.00" (rather than the value "5") may be given the highest score because it is directly to the left of and at the same height as the field name "Total Amount," as opposed to being outside the distance threshold and misaligned.

[0077] 6 is a schematic diagram illustrating how a neural network 605 makes a particular training and deployment prediction given particular inputs, according to some embodiments. In one or more embodiments, the neural network 605 represents or includes at least some of the functionality described with respect to the user computer session stitching component 108 of FIG. 1 and / or the user computer session stitching model / layer 308 of FIG. 3.In some embodiments, the neural network 605 may be implemented using a variety of techniques, including supervised learning (e.g., using logistic regression, using backpropagation neural networks, random forests, decision trees, etc.), unsupervised learning (e.g., using the Apriori algorithm, using K-means clustering), semi-supervised learning, reinforcement learning (e.g., using the Q-learning algorithm, using temporal difference learning), regression algorithms (e.g., least squares, logistic regression, stepwise regression, multivariate adaptive regression splines, local estimation scatterplot smoothing, etc.), instance-based methods (e.g., k-nearest neighbors, learning vector quantization, self-organizing maps, etc.), regularization methods (e.g., ridge regression, least absolute shrinkage and selection operators, elastic nets, etc.), decision tree learning methods (e.g., classification and regression trees, iterative dichotomization, etc.), and decision tree learning methods (e.g., classification and regression trees, iterative dichotomization, etc.). 3, C4.5, chi-squared automatic interaction detection, decision stumps, random forests, multivariate adaptive regression splines, gradient boosting machines, etc.), Bayesian methods (e.g., naive Bayes, averaged 1-dependent estimators, Bayesian belief networks, etc.), kernel methods (e.g., support vector machines, radial basis functions, linear discriminant analysis, etc.), clustering methods (e.g., k-means clustering, expectation maximization, etc.), association rule learning algorithms (e.g., apriori algorithm, eclat algorithm, etc.), artificial neural network models (e.g., perceptron, backpropagation, Hopfield network, etc.), The term may represent or include alternative or additional model functionality such as: self-organizing maps, learning vector quantization, etc.), deep learning algorithms (e.g., restricted Boltzmann machines, deep belief networks, convolutional networks, stacked autoencoders, etc.), dimensionality reduction methods (e.g., principal component analysis, partial least squares regression, Sammon mapping, multidimensional scaling, projection pursuit, etc.), ensemble methods (e.g., boosting, bootstrap aggregation, AdaBoost, stacking, gradient boosting machines, random forests, etc.), and / or any suitable type of machine learning algorithm.

[0078] In various embodiments, the neural network 605 is trained using one or more data sets of training data inputs 615 to make training predictions 607 with acceptable losses, which later aid in deployment to make correct inference predictions 609. In one or more embodiments, learning or training involves minimizing a loss function between a target variable (e.g., an incorrect prediction that a user is part of two sessions) and an actual predicted variable (e.g., a correct prediction that a user is part of multiple sessions). Based on the loss determined by the loss function (e.g., mean squared error loss (MSEL), cross-entropy loss, etc.), the loss function learns to reduce the error of the prediction over multiple epochs or training sessions, such that the neural network 605 learns which features and weights indicate correct inferences given the inputs. Therefore, to reduce prediction error, it is desirable to achieve as close to 100% confidence in a particular classification or inference as possible. In an illustrative example, the neural network 605 learns over several epochs that for a given session or page sequence (simulating the pages a user navigates through during a session), the most likely user is X. It will be appreciated that in some embodiments, the pages of multiple computer sessions in the deployment input 603 and training data input represent anonymized values ​​(e.g., anonymized by the anonymization model / layer 306 of FIG. 3).

[0079] Following processing of the first round / epoch of training (e.g., "training data input" 615), the neural network 605 makes predictions that may or may not be at an acceptable loss function level. For example, the neural network 605 may process the training data input 615. The neural network 605 may then predict that the user has accessed content from another computer session. This process may be repeated over multiple iterations or epochs until an optimal or correct predictor is learned (e.g., by maximizing reward and minimizing loss) and / or until the loss function reduces the error in the predictions to an acceptable confidence level.

[0080] In one or more embodiments, the neural network 605 converts or encodes the runtime deployment input 603 and the training data input 615 (e.g., via convolutional layers) into corresponding feature vectors in a feature space. A "feature vector" (also referred to as a "vector"), as described herein, may include one or more real numbers, such as a series of floating values ​​or integers (e.g., [0,1,0,0]), that represent one or more other real numbers, words in a natural language (e.g., English), and / or other character sequences (e.g., symbols (e.g., @, !, #), phrases, and / or sentences, etc.). Such natural language words and / or character sequences correspond to a set of features, which are encoded or converted into corresponding feature vectors so that a computer can process the corresponding extracted features. For example, embodiments may parse, tokenize, and encode each value or other content in a page into one or more feature vectors.

[0081] In some embodiments, the neural network 605 learns by training, parameters, or weights, such that similar features are close to each other in feature space (e.g., by Euclidean distance or cosine distance) by minimizing a loss according to a loss function (e.g., Triplet loss or GE2E loss). Such training is based on one or more of the training data inputs 615 provided to the neural network 605.

[0082] One or more embodiments determine one or more feature vectors representing the input 615 in vector space by aggregating (e.g., mean / median or dot product) the feature vector values ​​to arrive at a particular point in the feature space. For example, certain embodiments formulate a dot product of the pages of a single session (representing all pages navigated during the session) and / or formulate a dot product of the field values ​​of a single page (e.g., all of the values ​​in screenshot 400 of FIG. 4), and then aggregate these values ​​into a single feature vector.

[0083] In one or more embodiments, the neural network 605 learns features from the training data input 615 and responsively applies weights to them during training. A "weight" in the context of machine learning may represent the importance or significance of a feature or feature value for a prediction. For example, each feature may be associated with an integer or other real number, with larger real numbers making the feature more important to the prediction. In one or more embodiments, weights in neural networks or other machine learning applications may represent the strength of connections between nodes or neurons from one layer (input) to the next layer (output). A weight of 0 may mean that the input does not change the output, whereas a weight greater than 0 may change the output. The larger the value of the input, or the closer the value is to 1, the more the output changes or increases. Similarly, negative weights are possible. A negative weight may proportionally reduce the value of the output. For example, the more the value of the input increases, the smaller the value of the output. A negative weight may contribute to a negative score.

[0084] In another illustrative example of training, one or more embodiments learn feature vector embeddings based on learning (e.g., deep learning) to detect similar features between training data inputs 615 in a feature space using a distance measure such as cosine (or Euclidean) distance. For example, the training data inputs 615 are converted from strings or other formats into vectors (e.g., a set of real numbers) where each value or set of values ​​represents an individual feature in the feature space (e.g., individual field values ​​and / or individual pages across multiple sessions). The feature space (or vector space) contains a collection of feature vectors, and each vector may be oriented or embedded within the space based on the aggregate similarity of the features in the feature vectors. Certain feature characteristics for each target prediction may be learned or weighted across various training stages or epochs. For example, for a first field value (e.g., a phone number) in multiple sessions in the training data inputs 615 created by a user in , the neural network 605 may learn that the same phone number entered across multiple sessions means that the same user accessed the pages in these different sessions with a probability of 90% or greater. As a result, this pattern may be weighted (e.g., the connections of the node may be strengthened to values ​​closer to 1), while the connections of other nodes (e.g., representing other unimportant fields (such as dates)) may be weakened to values ​​closer to 0). In this way, embodiments learn weights corresponding to different features such that similar features found in the input contribute positively to the prediction.

[0085] In some embodiments, such training is supervised using annotations or labels. Alternatively or additionally, in some embodiments, such training is not supervised using annotations or labels, but may include, for example, clustering different, unknown clusters of data points together. For example, in some embodiments, training includes (or is preceded by) annotating / labeling each page (and thus fields and values) with a specific unique identifier (representing a particular user) so that the neural network 605 learns which fields and values ​​the same user enters, which are used to modify the weights / neural node connections for future predictions. For example, the neural network 605 may learn that multiple users tend to use different email addresses across computer sessions or page navigation. Thus, the neural network 605 adjusts weights accordingly or deactivates nodes at 607 so that email addresses are not a strong signal to use for prediction training.

[0086] In one or more embodiments, following training of the neural network 605, the neural network 605 receives one or more of the deployment inputs 603 (e.g., while deployed). When a machine learning model is deployed, it has typically been trained, tested, and packaged so that it can process data it has never seen before. In response, in one or more embodiments, the deployment inputs 603 are automatically converted into one or more feature vectors that are mapped within the same feature space as the vectors representing the training data inputs 615 and / or the training predictions 607. In response, one or more embodiments determine a distance (e.g., Euclidean distance) between the one or more feature vectors and the training data inputs 615 or other vectors representing the predictions, which is used to generate one or more of the inferred predictions 609.

[0087] In an illustrative example, neural network 605 receives an indication that a user (whose identity is unknown) accessed a first web or app page of a first computer application in a first computer session. Neural network 605 then determines the distance (e.g., Euclidean distance) between a vector representing runtime deployment input 603 and training data input 615. Based on this distance being within a threshold distance, particular embodiments generate a score in inferential prediction 609 that indicates the probability that the current user of the current computer session is the same user as the past computer session.

[0088] In certain embodiments, the inferential prediction 609 can be either hard (e.g., class membership is binary "yes" or "no") or soft (e.g., there is a probability or likelihood associated with the label). Alternatively or additionally, transfer learning may be performed. Transfer learning is the concept of reusing a pre-trained model for a new, related problem (e.g., a new video encoder, new feedback, etc.).

[0089] Figure 7 is a schematic diagram of an exemplary network graph 700, according to some embodiments. In some embodiments, the network graph 700 is a structure used by the user computer session stitching component 108 of Figure 1, the user computer stitching model / layer 308 of Figure 3, or included within the deployment input 603 of Figure 6 to access historical computer user activity and / or page content for generating scores. A network graph is a graphical representation or visualization for a set of objects, where pairs of objects are connected by links or "edges." Interconnected objects are represented by points called "vertices," and the links connecting those vertices are called "edges." Each node or vertex represents a specific location in one-, two-, or three-dimensional (or any other dimensional) space. A vertex is a point where one or more edges meet. An edge connects two vertices. Specifically, network graph 700 (an undirected graph) includes nodes or vertices 702 (representing John Doe), 704 (representing Application C), 706 (representing Application A), 708 (representing Application B), and node 710 (representing Page Z of Application B).

[0090] The network graph 700 embodies the relationships between user John Doe (or a unique identifier representing John Doe in a privacy-preserving manner) and various applications, and stores (or points to) John Doe's computer user activity by application or page. In this way, for example, the user computer session stitching component 108 (FIG. 1) "walks" the network graph 700 to find all pages of all applications that a particular user has interacted with in order to compare past hash values ​​(e.g., phone numbers) with current hash values ​​(e.g., near-real-time phone numbers entered by the user). This contemplates situations where a user may enter specific values ​​at different times, meaning that a single given session may not provide the specific field values ​​necessary for comparison of anonymized values. Rather, the history of computer sessions may reveal the specific field values ​​necessary for comparison. For example, a user may enter a specific phone number when registering during a first computer session of application A. However, during the next several computer sessions, the user may not enter that phone number again. Particular embodiments still capture the entire history of a user's interactions with applications.

[0091] Instead of or in addition to network graph 700, some embodiments use other computer session tracking features, such as computer user activity logs (e.g., browser logs or application logs), which provide a time-stamp indication of the exact computer user activities performed by a user. Some embodiments construct network graph 700 in response to user computer session stitching component 108 (FIG. 1) generating a particular score. For example, based on determining (via hash comparison) that John Doe accessed both application A and application C in different sessions, certain embodiments generate nodes 706 and 704 that link to node 702 representing all hash values ​​(particular field values) common between the two applications. In other words, instead of naming node 702 "John Doe," it is named by the hash value (e.g., a hashed phone number) common between applications A and C, indicating that the same user entered the same phone number between the two different sessions when interacting with applications A and C.

[0092] FIG. 8 is a flow diagram of an example process 800 for training a machine learning model on various pages, according to some embodiments. Process 800 (and / or any of the functions described herein) may be performed by processing logic including hardware (e.g., circuits, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executed on a processor to perform a hardware simulation), firmware, or a combination thereof. While certain blocks described in this disclosure are referenced in a particular quantity and order, it will be understood that any block may occur substantially in parallel with, before, or after any other block. Furthermore, there may be more (or fewer) blocks than shown. Additional blocks may include blocks embodying any functionality described herein (e.g., as described with respect to FIGS. 1-7). Computer-implemented methods, systems (including at least one computing device having at least one processor and at least one computer-readable storage medium), and / or computer-readable media described herein may perform or be caused to perform process 800 or any other functionality described herein.

[0093] In some embodiments, process 800 is performed by the modified BERT model of FIG. 5 to learn various fields, or by the neural network 605 of FIG. 6 to learn which fields, values, and / or other signals indicate that the same user accessed content from different computer sessions. According to block 802, multiple pages with one or more labels are received, each page including one or more values ​​(e.g., of a particular field), which may or may not be anonymized. For example, a page identical to screenshot 400 of FIG. 4 is received, with visual annotations (labels) indicating that value 402 is a "credit card number," value 404 is a "phone number," and value 406 is a "zip code" in preparation for learning the specific fields. In another example, screenshot 400 is instead or additionally labeled as "User X," indicating that user X entered all of the information in screenshot 400. In some embodiments, such labeling is performed on multiple pages across multiple sessions to predict whether specific values ​​on different pages or computer sessions were entered by the same user. In yet another example, screenshot 400 is alternatively or additionally labeled with a specific computer session ID to distinguish between different sessions in which the user is engaged.

[0094] Pursuant to block 804, a ground truth is derived based on one or more features extracted from the page. For example, each field value (e.g., phone number, credit card number, zip code, etc.) of each page and computer session is encoded into a feature vector and embedded in a feature space to represent the ground truth.

[0095] Pursuant to block 806, pairs of training sets are identified. In some embodiments, such pairs of training sets are entirely different pages from the plurality of pages received in block 802 that were used to derive the ground truth. In an illustrative example of block 806, pairs of two training sets of the same type are paired, such as two pages that the same user interacted with during different sessions. In another example, pairs of two training sets of different types are paired, such as two pages that different users interacted with during different sessions.

[0096] Pursuant to block 808, a machine learning model is trained based at least in part on learning weights associated with the extracted features. In other words, various embodiments learn embeddings of the training set pairs relative to the ground truth. For example, some embodiments determine the distance between a ground truth feature vector representing the labeled page and a feature vector representing one of the training set pairs. Based on a loss (e.g., difference in distance between the ground truth and the training set pair) determined by a loss function (e.g., mean squared error loss (MSEL), cross-entropy loss), the loss function is learned to reduce errors in predictions over multiple epochs or training sessions. For example, some embodiments train a neural network with a mean squared error loss,

number

number

[0097] 9 is a flow diagram of an example process 900 for generating a score indicative of the probability that the same user accessed a first page and a second page or accessed content from different computer sessions, according to some embodiments. In some embodiments, process 900 occurs at runtime or model deployment time, such as when process 800 (FIG. 8) for training a machine learning model has already occurred.

[0098] Pursuant to block 903, particular embodiments receive an indication (e.g., a flag, Boolean value, or other value) that a user (via a first user device associated with the user) has accessed a first page, the first page access occurring during a first computer session. As described herein, "accessing" a page includes any suitable computer user activity, such as a user logging into a particular site (e.g., by entering a URL) or a user making a particular selection when the page is rendered to the user. In some embodiments, the first page is a "web page" of a web application. In other embodiments, the first page is an "app page" of a mobile or other application.

[0099] A "computer session," as described herein, may be initiated and terminated in any suitable manner. For example, in some embodiments, a computer session begins when a user logs into a site or is recognized by the site as a returning user associated with activity on the site. In some embodiments, a computer session is considered to have ended when a user logs off the site (e.g., exits a window), logs into another site or opens a different window, or becomes inactive (or idle) on the site for a predetermined period of time. For example, after 30 minutes of idle time with no computer user activity (e.g., no queries or clicks received), certain embodiments automatically terminate a session. However, it will be appreciated that in a multi-device world, the traditional definition of a computer session is becoming increasingly inapplicable. More broadly, in some embodiments, a computer session encompasses the idea that a user is attempting to accomplish a particular task, possibly spanning multiple devices and / or an extended period of time. A user may resume a computer session on a different device, after a period of time, etc. A user may have many parallel sessions ongoing at the same time, for example. A session may include user phases, such as a discovery phase, an exploration phase, a follow-up phase, etc.

[0100] In some embodiments, responsive to block 903, some embodiments record computer user activities in a data record for the first computer session. In some embodiments, such functionality includes functionality such as that described with respect to session recording component 102 of FIG. 1. For example, in some embodiments, recording the computer user activities includes capturing a Document Object Model (DOM) of the first page and serializing the DOM according to the order in which the computer user activities are performed, where the serialization corresponds to storing in computer storage a visual representation of each input of the computer user activities performed by the user on the first page. For example, in block 903, in response to receiving an indication that a user has logged into an application and generated screenshot 400 of FIG. 4, certain embodiments record or store in computer storage each input made by the user, such as values ​​402, 404, and 406, capturing an in-visual sequence of when and how the user interacted with screenshot 400, as well as screenshot 400 itself, among other screenshots.

[0101] In accordance with block 905, some embodiments extract a first value of a first field from the first page. As used herein, a "value" includes any numeric value and / or natural language character and / or string. And, as noted above, a "field" generally represents a category of data typically expressed in natural language. For example, referring again to FIG. 4, value 402 includes a sequence of digits representing a credit card number. In some embodiments, block 905 includes functionality described with respect to field extraction component 104 of FIG. 1, field extraction model / layer 304 of FIG. 4, and / or modified BERT model of FIG. 5. In some embodiments, "extracting" includes copying the first value from the first page into a data structure. In some embodiments, "extracting" in block 905 alternatively or additionally includes "detecting" a unique identifier associated with the user in the first page, where the unique identifier is the first value.

[0102] A “unique identifier” refers to any suitable value associated with a user, such as a user's credit card number, a user's phone number, a user's email address, a user's zip code, a user's residential address, or a user's name. A unique identifier tends to be unique or specific to a user, as opposed to a general value (e.g., an order total, an order date, or the name of the country or city where the user lives) that may be shared by other users or that may change significantly between computer sessions. For example, certain embodiments detect (e.g., by a machine learning model) the value 123-4567, which is the user's phone number. In some embodiments, the “extracting” in block 905 alternatively or additionally includes determining that a first value of the plurality of values ​​includes a unique identifier associated with the user. For example, some embodiments determine or extract, by a modified BERT model, that the first value shown above (i.e., 123-4567) is a phone number, as opposed to any other field.

[0103] Continuing with block 905, some embodiments detect an indication of an attachment in the first page. In response to the detection, some embodiments convert the attachment into a machine-readable bitmap image. At least in part responsive to the converting, some embodiments detect a first value in the machine-readable bitmap image, the first value being contained within the attachment, and extracting the first value is based at least in part on the detecting. Such functionality contemplates that a particular page may have an embedded attachment (e.g., a PDF) that is not machine-readable in its native form. As such, as described with respect to field extraction component 104 of FIG. 1, some embodiments perform OCR and other functions to extract the corresponding content and make the content machine-readable.

[0104] In some embodiments, block 905 (or "extract" in block 905) includes or is followed by the following function: in response to receiving an indication that a user has accessed a first page, particular embodiments generate a second score indicating whether a first field or first value in the first page corresponds to (e.g., matches) a predetermined field, and extracting the first value of the first field from the first page is based on the second score indicating that the first value or first field corresponds to the predetermined field. Such function is described herein with respect to Term Frequency-Inverse Document Frequency (TF-IDF) or semantic similarity embodiments, where a "predetermined field" (e.g., a field name) is stored in computer storage, and a determination is made whether the extracted first field or first value syntactically matches or is semantically similar to the predetermined field. In other words, in some embodiments, particular embodiments extract a particular value or field only if there is a syntactic or semantic match. This anticipates developer rules or conditions to extract or select only certain fields for hashing, such as fields that are strongly indicated to be unique or consistent across multiple computer sessions (e.g., credit card numbers) as opposed to fields that are not strongly indicated to be unique or consistent across multiple computer sessions (e.g., dates), as described herein.

[0105] In some embodiments, the "extract" in block 905 is based on training a machine learning model to learn to indicate that an individual value of a plurality of values ​​is a component of an individual field of a plurality of fields. In some embodiments, this includes the functionality described with respect to the modified BERT model of FIG. 5, which is pre-trained to understand natural language and fine-tuned to predict the field type of a given value. In an illustrative example of the functionality of learning that a value is a component of (e.g., belongs to) a certain field based on a model trained to recognize patterns in credit card numbers (e.g., always having certain types of characters (digits) and certain amounts of digits), the model learns that this value is a credit card, as opposed to, for example, a phone number.

[0106] In some embodiments, the first field (or any field described herein) is one of an email address, a zip code (e.g., as shown by value 406 in FIG. 4), a postal address (e.g., as shown under "Billing Address" in FIG. 4), a credit card number (e.g., as shown by value 402 in FIG. 4), a phone number (e.g., as shown by value 404 in FIG. 4), a username, and / or a natural language name (e.g., that directly identifies the user).

[0107] In accordance with block 907, some embodiments cause a first hash (e.g., a one-way hash) to be calculated (e.g., at least in part in response to the extraction in block 905), where the first hash at least partially represents the first value. For example, referring again to Figure 4, credit card value 402 is hashed to v66si6t. In some embodiments, block 907 includes any of the functionality described with respect to value anonymization component 102 of Figure 1, combined hash 204 of Figure 2, single phone hash 210 of Figure 2, and / or anonymization model / layer 306 of Figure 3.

[0108] It will be appreciated that the values ​​do not necessarily have to be hashed, and that any suitable anonymization may be performed on the values, such as obfuscating, encrypting, masking, etc. For example, in some embodiments, at least in part, in response to detecting the unique identifier on the first page, some embodiments cause a first anonymized value to be calculated, the first anonymized value at least partially representing the unique identifier. For example, a full credit card number may be masked or hidden except for the last four digits. In another example, at least in part, in response to determining that a first value of the plurality of values ​​includes a unique identifier associated with a user, some embodiments cause the unique identifier Z to be anonymized.

[0109] In some embodiments, the first hash "at least partially represents the first value" because in some embodiments the first hash represents only the first value (e.g., as shown by phone hash 210 in FIG. 2), or in other embodiments, the first hash not only represents / includes the first value but also includes hashes of other field values ​​(e.g., as shown by combined hash 204 in FIG. 2). For example, some embodiments extract a third value of a second field from the first page. In response to extracting the third value, some embodiments cause a first hash to be calculated by combining a first subhash representing the first value with a second subhash representing the third value. Some embodiments extract a fourth value from the second page. In response to extracting the fourth value, some embodiments cause a second hash to be calculated by combining a third subhash representing the second value with a fourth subhash representing the fourth value. All of this functionality is described in FIG. 2, particularly computer session 220. A "subhash" is any individual hash of a particular value belonging to a particular field. A "subhash" may also be part of a larger combined hash. For example, in Figure 2, the "subhash" for the phone number 123-4567 is 3XTac4TL, which in some embodiments is included in combined hash 204. As another example, each of the hashes shown in Figure 4 is a subhash in some embodiments. Thus, in some embodiments, the first hash in block 907 represents a concatenation of the values ​​of multiple fields on the first page, or alternatively, represents only the first value.

[0110] In some embodiments, the first hash (and / or the second hash) is a one-way hash as described herein. In some embodiments, the first hash (and / or the second hash) is calculated on a user device associated with the user. As described herein, this has the technical effect of improving computer security because the plaintext value is not transmitted over a network to the server.

[0111] In accordance with block 909, some embodiments receive another indication that at least a second page has been accessed, the access of the second page occurring during a second computer session. For example, one week after a user accesses the first page, the same or a different user may be accessing a page of the same computer application as the first page or a page of a different computer application. However, it will be appreciated that in some embodiments, the access of the second page alternatively occurs during the same first computer session. In other words, in some embodiments, process 900 is performed for different pages accessed during the same computer session, as opposed to different computer sessions.

[0112] As described herein, one technical solution is the ability to capture page values ​​or fields within any subsequent computer session, even if the user is using a different device or platform. For example, in some embodiments, a first page accessed during a first computer session represents an app page contained within a mobile application, and a second page accessed during a second computer session is a web page contained within a web application. Furthermore, in some embodiments, during the first computer session, the user accessed the mobile application using a mobile device (e.g., a smartphone) and accessed the web application through a browser on a computing device (e.g., a laptop or desktop) different from the mobile device. This illustrates the technical effect that embodiments are platform and device agnostic, as described herein, to improve the accuracy and scalability of user tracking.

[0113] Pursuant to block 911, some embodiments extract a second value from the second page. In some embodiments, such “extracting” in block 911 includes or represents the same functionality as described with respect to the extraction in block 905. In some embodiments, such extracting in block 911 includes functionality described with respect to the field extraction component 104 of FIG. 1, the field extraction model / layer 304 of FIG. 3, or the modified BERT model of FIG. 5. In accordance with block 913, some embodiments cause a second hash (or anonymized value) to be calculated (e.g., at least in part in response to extracting the second value), the second hash at least partially representing the second value. In some embodiments, block 913 represents the same functionality as described with respect to block 907. In some embodiments, block 913 includes functionality described with respect to the value anonymization component 106 of FIG. 1, or the anonymization model / layer 306 of FIG. 3.

[0114] Based at least in part on the comparison between the first hash and the second hash (or the first anonymized value and the second anonymized value), some embodiments generate a score indicating the probability that the same user accessed both the first page and the second page, in accordance with block 915. In some embodiments, block 915 represents or includes functionality described with respect to user computer session stitching component 108 of FIG. 1, user computer session stitching model / layer 308 of FIG. 3, or inference prediction 609 of neural network 605 of FIG. 6.

[0115] In some embodiments, the "score" includes or partially indicates the probability that a user (who accessed a first page) also accessed a second page. In other words, it captures the probability that a current user who accessed content in a second computer session is the same user who accessed the content during a first computer session. In some embodiments, the "score" includes or partially indicates that a user accessed content in both a first computer session and a second computer session, regardless of, for example, the individual pages accessed by the user. In some embodiments, the "score" directly indicates a confidence or probability (e.g., 95%) generated by a probabilistic model. In alternative or additional embodiments, the "score" indicates a particular distance (e.g., Euclidean distance) between feature vectors as described herein. In some embodiments, the "score" alternatively or additionally represents a machine learning prediction output, such as a classification probability of a particular user class (in the case of a classifier) ​​or a probability that a data point (representing a second page or second computer session) belongs to a particular cluster representing a particular user (e.g., in the case of K-means clustering).

[0116] In some embodiments, the “comparison” in block 915 includes or represents comparing the first anonymized value (or first anonymized unique identifier) ​​to a second anonymized value (or second anonymized unique identifier), where the second anonymized value represents, at least in part, another unique identifier derived during access to or input (e.g., by a user) on the second page during the second computer session. Referring again to FIG. 2 , in an exemplary illustration of such a comparison, particular embodiments compare hash 3XTac4TL (a phone number hash), extracted during computer session 220 and as part of hash 204, with hash 210, which is the exact same value. Thus, based on the hash values ​​matching, particular embodiments generate a high score, indicating a high likelihood that the same user accessed the content in both computer session 220 and computer session 230.

[0117] In some embodiments, generating the score in block 915 is based on training a machine learning model (e.g., as illustrated by process 800 in FIG. 8) to learn whether different computer sessions (and / or pages) were engaged in by the same user. In some embodiments, this includes functionality described with respect to process 800 or neural network 605 in FIG. 6. For example, the "score" in certain embodiments includes or represents a prediction, based on training a model, that the current user of the current computer session (e.g., the one accessing the second computer session or second page) is the same user who accessed the first page or other content in a past computer session (e.g., the first computer session).

[0118] In some embodiments, after generating the score in block 915 and recording the computer user activity on the first page and the second page (e.g., via CLARITY), some embodiments cause at least one of: transmitting a targeted advertisement to a user device associated with the user based on obtaining a record including the computer user activity and the score; generating a fraud detection report; and storing user preferences in computer storage. In an illustrative example of an embodiment of a targeted advertisement, based on the score indicating that the same user likely accessed the first page and the second page, particular embodiments then analyze the computer user activity to determine that the user purchased several merchandise containing the name and logo of a professional football team. Accordingly, particular embodiments cause an advertisement to be presented (e.g., on the second page or during a second computer session) for purchasing products bearing the name and logo of the same professional football team by communicating with the publication server over network 110 of FIG. 1 so that the publication server can directly serve the advertisement.

[0119] In another example of a fraud detection embodiment, based on a high probability score indicating that a first page is highly associated with fraud (e.g., based on the IP address of the user device that accessed the first page being the subject of past fraud detection) and a score in block 915 indicating that the same user accessed a second page, particular embodiments cause an administrator device to present a notification indicating that a transaction or computer user activity occurring on the second page is likely fraudulent. In yet another example of user preference, based on a score indicating that the same user likely accessed the first and second pages, particular embodiments then analyze the computer user activity to determine that the user has indicated a preference for a weather pagelet to be presented. Accordingly, particular embodiments cause the presentation of a weather pagelet during the second computer session or on the second page. It will be understood that these actions are merely representative, and that any suitable action may be taken (e.g., during the second computer session) based on the score and computer user activity, such as determining whether a user is a new or returning visitor, tracking user behavior for analysis, remembering items in a shopping cart based on actions from a previous computer session, remembering login details, etc.

[0120] Embodiments of the present disclosure may be described in the general context of computer code or machine-usable instructions, including computer-usable or computer-executable instructions, such as program modules, executed by a computer or other machine, such as a smartphone, tablet PC, or other mobile device, server, or client device. Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs particular tasks or implements particular abstract data types. Embodiments of the present disclosure may be practiced in a wide variety of system configurations, including mobile devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including memory storage devices.

[0121] Some embodiments may include an end-to-end software-based system operable within the system components described herein to operate computer hardware to provide system functionality. At a low level, a hardware processor can execute instructions selected from a machine language (also called machine code or native) instruction set for a given processor. The processor recognizes the native instructions and performs corresponding low-level functions, e.g., related to logic, control, and memory operations. Low-level software written in machine code can provide more complex functionality to higher-level software. Thus, in some embodiments, computer-executable instructions can include any software, including low-level software written in machine code, high-level software such as application software, and any combination thereof. In this regard, system components can manage resources and provide services for system functionality. Other variations and combinations thereof are contemplated in embodiments of the present disclosure.

[0122] Other embodiments In some embodiments, a computerized system, such as the computerized system described in any of the above embodiments, includes at least one computer processor and one or more computer storage media storing computer usable instructions that, when used by the at least one computer processor, cause the at least one computer processor to perform operations, including receiving an indication that a user has accessed a first web or app page of a first computer application, the access of the first web or app page occurring during a first computer session; extracting a first value of a first field from the first web or app page; computing a first hash at least partially in response to the extracting, the first hash at least partially representing the first value; and receiving another indication that at least a second web or app page of the first computer application or another computer application has been accessed. receiving a web or app page access request, wherein access of the second page occurs during a second computer session; extracting a second value from the second web or app page; computing a second hash at least in part in response to extracting the second value, the second hash at least in part representing the second value, the first hash and the second hash protecting the privacy of the one or more users; and generating a score indicative of a probability that the same user accessed both the first web or app page and the second web or app page based at least in part on a comparison between the first hash and the second hash.

[0123] Advantageously, these and other embodiments of the computerized systems described herein have the technical effects of improved process security, improved accuracy or reduced error rates for tracking users, increased scalability, improved human interaction, and improved consumption of computer resources, among other technical effects, as described in more detail herein.

[0124] In any combination of the above embodiments of the computerized system, the operations further include detecting an indication of the attachment in the first web or app page; responsive to the detecting, converting the attachment into a machine-readable bitmap image; and, at least partially responsive to the converting, detecting a first value in the machine-readable bitmap image, wherein the first value is included in the attachment and extracting the first value is based at least in part on the detecting.

[0125] In any combination of the above embodiments of the computerized system, the operations further include extracting a third value of the second field from the first web or app page; in response to extracting the third value, causing a first hash to be calculated by combining a first sub-hash representing the first value and a second sub-hash representing the third value; extracting a fourth value from the second page; and in response to extracting the fourth value, causing a second hash to be calculated by combining the third sub-hash representing the second value and the fourth sub-hash representing the fourth value.

[0126] In any combination of the above embodiments of the computerized system, the first hash represents one of a concatenation of values ​​of each of the multiple fields of the first web or app page or the first value alone.

[0127] In any combination of the above embodiments of the computerized system, the operations further include, in response to receiving an indication that the user has accessed a first web or app page of the first computer application, generating a second score indicating whether each field or value in the first web or app page corresponds to a predetermined field, wherein extracting the first value of the first field from the first web or app page is based on the second score indicating that the first value or first field corresponds to the predetermined field.

[0128] In any combination of the above embodiments of the computerized system, extracting the first value of the first field from the first web or app page is based on training a machine learning model to learn that each value of the plurality of values ​​is indicative of a component of an individual field of the plurality of fields.

[0129] In any combination of the above embodiments of the computerized system, the first field includes one of an email address, a zip code, a postal address, a credit card number, a phone number, a username, or a natural language name.

[0130] In any combination of the above embodiments of the computerized system, generating the score is based on training a machine learning model to learn whether different computer sessions were engaged in by the same user.

[0131] In any combination of the above embodiments of the computerized system, the first web or app page is part of a mobile application, the second web or app page is part of a web application, and the user accesses the mobile application using a mobile device and the web application through a browser on a computing device different from the mobile device.

[0132] In any combination of the above embodiments of the computerized system, the operations further include, in response to receiving an indication that a user has accessed a first web or app page of a first computer application, recording computer user activity in a data record for the first computer session, the data record excluding cookies.

[0133] In any combination of the above embodiments of the computerized system, recording the computer user activities includes capturing a Document Object Model (DOM) of the first web or app page and serializing the DOM according to the order in which the computer user activities are performed, and capturing and serializing the DOM corresponds to storing in computer storage a visual representation of each input of the computer user activities performed by the user on the first web or app page.

[0134] In any combination of the above embodiments of the computerized system, the operations further include causing at least one of transmitting targeted advertisements to a user device associated with the user based on obtaining the records and generating the score, generating a fraud detection report, and storing user preferences in computer storage.

[0135] In any combination of the above embodiments of the computerized system, each of the first hash and the second hash is a one-way hash, and each of the one-way hashes is calculated on a user device associated with the user.

[0136] In some embodiments, a computer-implemented method, such as the computer-implemented method described in any of the above embodiments, includes receiving an indication that a user accessed a first page of a first computer application, the access of the first page occurring during a first computer session by a first user device associated with the user; detecting a first unique identifier associated with the user on the first page; calculating, at least in part, a first anonymized value in response to the detecting, the first anonymized value at least in part representing the unique identifier; comparing the first anonymized value with a second anonymized value, the second anonymized value at least in part representing a second unique identifier derived during the access of the second page; and generating a score at least in part indicative of a probability that the user accessed the second page based at least in part on the comparing.

[0137] Advantageously, these and other embodiments of the computer-implemented methods described herein have the technical effects of improved process security, improved accuracy or reduced error rates for tracking users, increased scalability, improved human interaction, and improved consumption of computer resources, among other technical effects, as described in more detail herein.

[0138] In any combination of the above embodiments of the computer-implemented method, the computer-implemented method further includes detecting a third unique identifier in the first page; in response to detecting the third unique identifier, causing a first anonymized value to be calculated by combining a first subhash representing the first unique identifier and a second subhash representing the third unique identifier; detecting a fourth unique identifier in the second page; and in response to detecting the fourth unique identifier, causing a second anonymized value to be calculated by combining the third subhash representing the second unique identifier and a fourth subhash representing the fourth unique identifier.

[0139] In any combination of the above embodiments of the computer-implemented method, the first anonymized value represents one of a concatenation of each value of a plurality of unique identifiers of the first page, or the first unique identifier alone.

[0140] In any combination of the above embodiments of the computer-implemented method, detecting the first unique identifier is based on training a machine learning model to learn that the first unique identifier is a component of a particular field of the plurality of fields.

[0141] In any combination of the above embodiments of the computer-implemented method, the first unique identifier includes one of an email address, a zip code, a postal address, a credit card number, a phone number, a username, or a natural language name.

[0142] In any combination of the above embodiments of the computer-implemented method, generating the score is based on training a machine learning model to learn whether different computer sessions were engaged in by the same user.

[0143] In some embodiments, one or more computer storage media, such as the one or more computer storage media described in any of the embodiments above, include computer-executable instructions embodied on the one or more computer storage media that, when executed by one or more processors, cause the one or more processors to perform operations including receiving an indication that a first page of a first computer application is being accessed, the first page including a plurality of values, the access of the first page being included in a first computer session; determining that a first value of the plurality of values ​​includes a unique identifier associated with a user; causing anonymization of the unique identifier, wherein anonymizing the unique identifier protects the privacy of the user; causing the anonymization; comparing the anonymized unique identifier with a second anonymized unique identifier, the second anonymized unique identifier representing a value entered on the second page during the second computer session; and generating a score indicative of a probability that the user accessed the content in both the first computer session and the second computer session based at least in part on the comparing.

[0144] Advantageously, these and other embodiments of the computer storage media described herein have the technical effects of improved process security, improved accuracy or reduced error rates for tracking users, increased scalability, improved human interaction, and improved consumption of computer resources, among other technical effects, as described in more detail herein.

[0145] Exemplary Computing Architecture and Apparatus Turning now to FIG. 10 , a block diagram illustrating an exemplary operating environment 10 in which some embodiments of the present disclosure may be employed is shown. It should be understood that this and other configurations described herein are provided by way of example only. Other configurations and elements (e.g., machines, interfaces, functions, sequences, and groupings of functions) in addition to or instead of those shown may be used, and some elements may be omitted entirely for clarity. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location. Various functions described herein as being performed by certain entities may be performed by hardware, firmware, and / or software. For example, some functions may be performed by a processor executing instructions stored in memory.

[0146] Among other components not shown, the exemplary operating environment 10 includes several user devices, such as user devices 02a and 02b through 02n, several data sources (e.g., databases or other data stores, such as 105), such as data sources 04a and 04b through 04n, a server 06, sensors 03a and 07, and a network 110. It should be understood that the environment 10 shown in FIG. 10 is an example of one suitable operating environment. Each of the components shown in FIG. 10 may be implemented by any type of computing device, such as computing device 11 described with respect to FIG. 11. These components may communicate with each other via a network 110, which may include, without limitation, a local area network (LAN) and / or a wide area network (WAN). In some implementations, the network 110 includes the Internet and / or a cellular network, among any of a variety of possible public and / or private networks.

[0147] It should be understood that within the scope of this disclosure, any number of user devices, servers, and data sources may be employed within operating environment 10. Each may include a single device or multiple devices working together in a distributed environment. For example, server 06 may be provided by multiple devices arranged in a distributed environment that collectively provide the functionality described herein. In addition, other components not shown may also be included in the distributed environment.

[0148] The user devices 02a and 02b through 02n may be client devices on the client side of the operating environment 10, while the server 06 may reside on the server side of the operating environment 10. The server 06 may include server-side software designed to function in combination with the client-side software on the user devices 02a and 02b through 02n to implement any combination of features and functionality discussed in this disclosure. This division of the operating environment 10 is provided to illustrate one example of a suitable environment, and each implementation does not require that the server 06 and any combination of the user devices 02a and 02b through 02n remain separate entities. In some embodiments, one or more servers 06 represent one or more nodes in a cloud computing environment. Consistent with various embodiments, a cloud computing environment includes a network-based distributed data processing system that provides one or more cloud computing services. Furthermore, a cloud computing environment may include hundreds or thousands of computers or more located in one or more data centers and configured to share resources over one or more networks 110.

[0149] In some embodiments, the user device 02a or server 06 may alternatively or additionally include one or more web servers and / or application servers that facilitate the delivery of web or online content to browsers installed on the user device 02b. Often, the content may include static and dynamic content. When a client application, such as a web browser, requests a website or web application via a URL or search term, the browser typically contacts a web server to request static content or basic components of the website or web application (e.g., HTML pages, image files, video files, etc.). An application server typically delivers any dynamic portions of a web application or the business logic portion of a web application. Business logic can be described as functionality that manages communication between user devices and data stores (e.g., databases). Such functionality may include business rules or workflow (e.g., code indicating the order of processes with conditional if / then statements, while statements, etc.).

[0150] User devices 02a and 02b through 02n may include any type of computing device available to a user. For example, in one embodiment, user devices 02a through 02n may be computing devices of the type described herein with respect to Figure 11. By way of example and not limitation, a user device may be embodied as a personal computer (PC), a laptop computer, a cell phone or mobile device, a smartphone, a tablet computer, a smart watch, a wearable computer, a personal digital assistant (PDA), a music player or MP3 player, a global positioning system (GPS) or device, a video player, a portable communications device, a gaming device or system, an entertainment system, a vehicle computer system, an embedded system controller, a camera, a remote control, a barcode scanner, a computerized measurement device, an instrument, a consumer electronics device, a workstation, or any combination of these listed devices, or any other suitable computing device.

[0151] Data sources 04a and 04b through 04n may include data sources and / or data systems configured to make data available to any of the various components of operating environment 10 or system 100 described with respect to FIG. 1. Examples of data sources 04a through 04n may be one or more of a database, file, data structure, corpus, or other data store. Data sources 04a and 04b through 04n may be separate from user devices 02a and 02b through 02n and server 06, or may be incorporated and / or integrated into at least one of these components. In one embodiment, data sources 04a through 04n include sensors (such as sensors 03a and 07) that may be integrated with or associated with user devices 02a, 02b, or 02n, or server 06.

[0152] In some embodiments, operating environment 10 is utilized to implement one or more of the components of system 100 described in Figure 1, including a component for generating a score indicative of whether the same user has accessed multiple sessions, as described herein. Operating environment 10 may also be utilized to implement aspects of processes 800 (Figure 8), 900 (Figure 9), and / or any other functionality described with respect to Figures 1-9.

[0153] Having described various implementations, an exemplary computing environment suitable for implementing embodiments of the present disclosure will now be described. With reference to FIG. 11, an exemplary computing device is provided and is generally referred to as computing device 11. Computing device 11 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the present disclosure. Computing device 11 should not be interpreted as having any dependency or requirement relating to any one or combination of illustrated components.

[0154] Turning to FIG. 11 , computing device 11 includes a bus 19 that directly or indirectly couples the following devices: memory 12, one or more processors 14, one or more presentation components 16, one or more input / output (I / O) ports 18, one or more I / O components 20, and an exemplary power supply 22. Bus 19 represents what may be one or more buses (such as an address bus, a data bus, or a combination thereof). While various blocks in FIG. 11 are shown as lines for clarity, in practice these blocks represent logical components and not necessarily actual components. For example, a presentation component, such as a display device, may be considered an I / O component. Additionally, a processor has memory. The inventors herein recognize that such is the nature of the art and reiterate that the diagram of FIG. 11 is merely illustrative of an exemplary computing device that may be used in connection with one or more embodiments of the present disclosure. No distinction is made between categories such as "workstations," "servers," "laptops," "handheld devices," or other computing devices, as all are within the scope of FIG. 11 and are intended with respect to "computing device."

[0155] The computing device 11 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computing device 11, including both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the computing device 11. Computer storage media do not themselves include signals. Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable media.

[0156] Memory 12 includes computer storage media in the form of volatile and / or non-volatile memory. Memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical disk drives, or other hardware. Computing device 11 includes one or more processors 14 that read data from various entities, such as memory 12 or I / O components 20. Presentation component 16 presents data indications to a user or other device. Exemplary presentation components include a display device, a speaker, a printing component, a vibrating component, etc.

[0157] The I / O port 18 allows the computing device 11 to be logically coupled to other devices, including I / O components 20, some of which may be integrated. Exemplary components include a microphone, joystick, gamepad, satellite dish, scanner, printer, wireless device, etc. The I / O component 20 may provide a natural user interface (NUI) that processes air gestures, voice, or other physiological input generated by a user. In some examples, the input may be transmitted to an appropriate network element for further processing. The NUI may implement any combination of voice recognition, touch and stylus recognition, facial recognition, biometric recognition, on-screen as well as adjacent-to-screen gesture recognition, air gestures, head and eye tracking, and touch recognition associated with a display on the computing device 11. The computing device 11 may be equipped with a depth camera, such as a stereoscopic camera system, an infrared camera system, an RGB camera system, or a combination thereof, for gesture detection and recognition. Additionally, the computing device 11 may be equipped with an accelerometer or gyroscope that enables motion detection. The output of the accelerometer or gyroscope may be provided to a display of the computing device 11 for rendering immersive augmented or virtual reality.

[0158] Some embodiments of the computing device 11 may include one or more radios 24 (or similar wireless communication components). The radios 24 transmit and receive radio or wireless communications. The computing device 11 may be a wireless terminal adapted to receive communications and media over various wireless networks. The computing device 11 may communicate via wireless protocols such as Code Division Multiple Access ("CDMA"), Global System for Mobile ("GSM"), or Time Division Multiple Access ("TDMA") to communicate with other devices. The wireless communications may be short-range connections, long-range connections, or a combination of both short-range and long-range wireless telecommunication connections. References to "short" and "long" types of connections do not refer to a spatial relationship between the two devices. Instead, short-range and long-range are broadly referred to as different categories or types of connections (i.e., primary and secondary connections). A short-range connection may include, by way of example and not limitation, a Wi-Fi® connection to a device (e.g., a mobile hotspot) that provides access to a wireless communication network, such as a WLAN connection using the 802.11 protocol, and a Bluetooth connection to another computing device is a second example of a short-range or near-field wireless communication connection. A long-range connection may include, by way of example and not limitation, a connection using one or more of the CDMA, GPRS, GSM, TDMA, and 802.16 protocols.

[0159] While various components have been identified herein, it should be understood that any number of components and configurations may be employed to achieve the desired functionality within the scope of the present disclosure. For example, components in the illustrated embodiments are shown with lines for conceptual clarity. Other configurations of these and other components may also be implemented. For example, while some components are depicted as single components, many of the elements described herein may be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location. Some elements may be omitted entirely. Furthermore, various functions described herein as being performed by one or more entities may be performed by hardware, firmware, and / or software, as described below. For example, various functions may be performed by a processor executing instructions stored in memory. As such, other configurations and elements (e.g., machines, interfaces, functions, sequences, and groupings of functions) in addition to or in place of those shown may be used.

[0160] The embodiments of the present disclosure have been described with the intent to be illustrative, not restrictive. The embodiments described in the preceding paragraphs may be combined with one or more of the specifically described alternatives. In particular, the claimed embodiments may include reference to other embodiments in the alternative. The claimed embodiments may define further limitations to the claimed subject matter. Alternative embodiments will become apparent to readers of this disclosure after and because of their reading. Alternative ways of implementing the foregoing subject matter may be accomplished without departing from the scope of the appended claims. Certain features and subcombinations are useful and may be employed without reference to other features and subcombinations and are considered to be within the scope of the claims.

[0161] As used herein, the term "set" may be employed to refer to an ordered (i.e., sequential) or unordered (i.e., non-sequential) collection of objects (or elements), such as, but not limited to, data elements (e.g., events, clusters of events, etc.). A set can contain N elements, where N is any non-negative integer. That is, a set can contain 1, 2, 3, ... objects and / or elements, where N is a positive integer with no upper bound. Thus, as used herein, a set can contain only a single element. In other embodiments, a set can contain significantly more than 1, 2, or 3 elements. As used herein, the term "subset" is a set contained within another set. A subset can be, but need not be, a proper or strict subset of the other set in which it is contained. That is, if set B is a subset of set A, then in some embodiments set B is a proper or strict subset of set A. In other embodiments, set B is a subset of set A, but is not a proper or strict subset of set A.

Claims

1. At least one computer processor, When used by the aforementioned at least one computer processor, Receiving an indication that a user has accessed a first web or app page of a first computer application, wherein such access to the first web or app page occurs during a first computer session. In response to receiving the indication that the user has accessed the first web or app page of the first computer application, generate a first score indicating whether each field or value in the first web or app page corresponds to a predetermined field. Extracting the first value of the first field from the first web or app page, at least in part on the first score indicating that the first value or the first field corresponds to the predetermined field. The method involves calculating a first hash in at least partially in response to extracting a first value, wherein the first hash represents at least partially the first value. Receiving another indication that at least a second web or app page of the first computer application or another computer application has been accessed, wherein the access to the second web or app page occurs during a second computer session. Extracting a second value from the second web or app page, A second hash is calculated in at least partially response to the extraction of the second value, the second hash represents at least partially the second value, and the first and second hashes protect the privacy of one or more users, and To generate a second score indicating the probability that the same user accessed both the first web or app page and the second web or app page, based at least in part on a comparison between the first hash and the second hash. One or more computer storage media that store computer-available instructions that cause at least one computer processor to perform an operation including the above, A system that includes this.

2. The aforementioned operation, To detect the indication of an attachment on the first web or app page, In response to the detection, convert the attached file into a machine-readable bitmap image, and Responding at least partially to the conversion, detecting the first value in the machine-readable bitmap image, wherein the first value is contained within the attachment, and extracting the first value is at least partially based on the detection. The system according to claim 1, further comprising:

3. The aforementioned operation, Extracting the third value of the second field from the first web or app page, In response to extracting the third value, the first hash is calculated by combining the first subhash representing the first value and the second subhash representing the third value. Extracting a fourth value from the second web or app page, and In response to extracting the fourth value, the second hash is calculated by combining the third subhash representing the second value and the fourth subhash representing the fourth value. The system according to claim 1, further comprising:

4. The system according to claim 1, wherein the first hash represents either the concatenation of the values ​​of the multiple fields of the first web or app page, or the first value alone.

5. The system according to claim 1, wherein extracting the first value of a first field from the first web or app page is based on training a machine learning model to learn that each value of a plurality of values ​​is a component of an individual field of a plurality of fields.

6. The system according to claim 1, wherein the first field includes one of the following: an email address, a postal code, a postal address, a credit card number, a telephone number, a username, or a natural language name.

7. The system according to claim 1, wherein generating the second score is based on training a machine learning model to learn whether different computer sessions were engaged by the same user.

8. The system according to claim 1, wherein the first web or app page is part of a mobile application, the second web or app page is part of a web application, the user accesses the mobile application using a mobile device, and accesses the web application using a browser on a computing device different from the mobile device.

9. The aforementioned operation, In response to receiving the indication that the user has accessed the first web or app page of the first computer application, the computer user activity is recorded in a data record for the first computer session, wherein the data record excludes cookies. The system according to claim 1, further comprising:

10. The system according to claim 9, wherein recording the computer user activity includes capturing a document object model (DOM) of the first web or app page and serializing the DOM in the order in which the computer user activity is performed, and capturing and serializing the DOM corresponds to storing a visual representation of each input of the computer user activity performed by the user on the first web or app page in computer storage.

11. The aforementioned operation, Based on the acquisition of the record and the generation of the second score, at least one of the following occurs: targeted advertisements are transmitted to user devices associated with the user; fraud detection reports are generated; and user preferences are stored in computer storage. The system according to claim 9, further comprising:

12. The system according to claim 1, wherein each of the first hash and the second hash is a one-way hash, and each of the one-way hashes is calculated in a user device associated with the user.

13. Receiving an indication that a user has accessed a first page of a first computer application, wherein the access to the first page occurs during a first computer session via a first user device associated with the user. In response to receiving the indication that the user has accessed the first page of the first computer application, generate a first score indicating whether a first field or first value in the first page corresponds to a predetermined field. Detecting a first unique identifier associated with the user on the first page, at least in part, based on the first value or the first score indicating that the first field corresponds to the predetermined field. The method involves at least partially responding to the detection and calculating a first anonymized value, wherein the first anonymized value at least partially represents the first unique identifier, and the anonymized value protects the user's privacy. Comparing the first anonymized value with the second anonymized value, wherein the second anonymized value represents at least partially the second unique identifier derived during access to the second page, and Based at least in part on the aforementioned comparison, a second score is generated that at least in part indicates the probability that the user accessed the second page. Computer implementation methods, including those mentioned above.

14. To detect a third unique identifier on the first page mentioned above, In response to detecting the third unique identifier, the first anonymized value is calculated by combining a first subhash representing the first unique identifier and a second subhash representing the third unique identifier. To detect the fourth unique identifier on the second page, and In response to detecting the fourth unique identifier, the second anonymized value is calculated by combining the third subhash representing the second unique identifier and the fourth subhash representing the fourth unique identifier. The computer implementation method according to claim 13, further comprising:

15. The computer implementation method according to claim 13, wherein the first anonymized value represents either the concatenation of the values ​​of the multiple unique identifiers of the first page, or the first unique identifier alone.

16. The computer implementation method according to claim 13, wherein the detection of the first unique identifier indicates that the first unique identifier is a component of a particular field among a plurality of fields, the method comprising training a machine learning model to learn this.

17. The computer implementation method according to claim 13, wherein the first unique identifier includes one of an email address, a postal code, a postal address, a credit card number, a telephone number, a username, or a natural language name.

18. The computer implementation method according to claim 13, wherein generating the second score is based on training a machine learning model to learn whether different computer sessions were engaged by the same user.

19. When executed by one or more processors, Receiving an indication that a first page of a first computer application has been accessed, wherein the first page contains a plurality of values, and the access to the first page is within a first computer session. In response to receiving the indication that the first page of the first computer application has been accessed, generate a first score indicating whether a first field or first value in the first page corresponds to a predetermined field. Determining that the first value among the plurality of values ​​contains a unique identifier related to the user, at least in part on the first value or the first score indicating that the first field corresponds to the predetermined field, This causes the anonymization of the unique identifier, and the anonymization of the unique identifier protects the user's privacy. Comparing the anonymized unique identifier with a second anonymized unique identifier, wherein the second anonymized unique identifier represents a value entered on a second page during a second computer session, and Based at least in part on the aforementioned comparison, a second score is generated indicating the probability that the user accessed the content in both the first and second computer sessions. One or more computer storage media having computer executable instructions embodied on one or more computer storage media that cause one or more processors to perform operations including the above.

Images