Processing structured data for real-time segmentation

Abstract

A method for performing real-time segmentation by updating a wide record based on receipt of real-time data, where an item of real-time data represents a transaction, detecting that the updated wide record meets criteria for performing real-time segmentation, and performing real-time segmentation on the updated wide record, wherein the real time relates to when the transaction represented in the updated wide record occurs.

Description

[Technical Field]

[0001] (Priority Claim) This application claims priority under 35 U.S.C. §119(e) to U.S. patent application Ser. No. 63 / 443,295, filed February 3, 2023, and U.S. patent application Ser. No. 18 / 486,300, filed October 13, 2023, the entire contents of each of which are incorporated herein by reference. [Background technology]

[0002] The present disclosure relates to techniques for efficiently operating a data processing system having many data sets that may be stored in any of a number of data stores.

[0003] Modern data processing systems manage vast amounts of data within an enterprise. For example, a large institution may have millions of datasets. These datasets can support multiple aspects of the enterprise's operations. Complex data processing systems typically process data in multiple stages, with results produced by one stage feeding the next. The overall flow of information through such a system can be described in terms of a directed data flow graph, where nodes or vertices in the graph represent components (either data files or processes) and links or "edges" in the graph indicate the flow of data between components. A system for performing such graph-based computations is described in U.S. Pat. No. 5,966,072, entitled "Executing Computations Expressed as Graphs," which is incorporated herein by reference.

[0004] Graphs can also be used to invoke computations directly. Graphs created in accordance with this system provide methods for introducing information into and retrieving information from individual processes represented by graph components, methods for moving information between processes, and methods for defining an execution order for the processes. Systems for invoking these graphs include algorithms for selecting inter-process communication methods and algorithms for scheduling process execution and providing oversight of the graph's execution.

[0005] To support a wide range of functions, data processing systems may run applications, whether implementing routine processes or extracting insights from data sets. Applications may be programmed to access data stores to read and write data. Summary of the Invention

[0006] In a general aspect, a process is performed by a data processing system for performing real-time segmentation by updating a wide record based on receipt of real-time data, where the items of real-time data represent transactions. The process includes detecting that the updated wide record meets criteria for performing real-time segmentation, and performing real-time segmentation on the updated wide record, where real time relates to when the transaction represented in the updated wide record occurs.

[0007] A segment is a collection of data associated with one or more attributes that define the collection of data as distinct from one or more other collections of data. For example, a segment may specify that one or more records included in the segment are associated with one or more attributes that define the segment. The one or more attributes may include data that describes aspects of the data in the collection associated with the segment. For example, a segment may describe the entity associated with the key value of each record in the collection associated with the segment. In this example, a segment may include one or more attributes that describe a subscriber or customer, a type of data, or any other aspect of the data that can be described by one or more attributes. Attributes may include metadata that describes other data with which the attribute is associated. For example, attributes may include an age or time period associated with the data, subscriber demographics, a range of values ​​associated with the data, the type of data, the format of the data, the time the data was sent or received, for any such examples of descriptive values, phrases, or keywords that describe other data.

[0008] A real-time segment includes an item of data that specifies a segment associated with a key value at the time the real-time segment is generated. A real-time segment effectively represents a segment with which a key value is associated within a threshold period of time at which the real-time segment is determined. Generally, real-time segments are not stored in persistent memory because they may become stale as additional real-time data is generated for the key value, changing the segment to which the key value belongs. For example, a real-time segment may no longer represent the actual segment with which a key value is currently associated once additional transaction data is generated for that key value. Rather, a key value may be associated with a different segment not yet calculated by the data processing system. Real-time segments are used in real time, or near real time, relative to the time the transaction data for the key value is generated. As a result, the determined segment represents the true current segment associated with the key value, and not a stale segment that has become stale.

[0009] The data processing system is configured to determine a real-time segment within a threshold period after transaction data for a key value is received at a real-time data source (e.g., a message bus). The data processing system is configured to act on updates to data associated with a key value within the threshold period (in real time or near real time) before other data (e.g., additional transactions) are generated for the key value. The real-time segment allows the data processing system to determine an action to take based on an updated segment for a key value within the threshold period after transaction data for the key value is generated. Thus, the data processing system can operate when a key value is associated with a particular segment, even if the key value is only associated with the particular segment for a short period of time (e.g., after a single transaction intraday). The data processing system can also operate immediately when a user enters a particular segment. For example, the data processing system can generate a message, alert, discount, or other data associated with the key value immediately in response to receiving transaction data associated with the key value. This allows a user associated with the key value to know that the user has entered a segment, such as a preferred customer segment, in response to a particular transaction.

[0010] Generally, a wide record includes data from multiple different data sources within a computing network. For example, a wide record can include one or more fields combined from multiple different data records. The data records can be obtained from the same or different storage devices, the same or different remote devices, and the same or different data processing systems. A wide record generally includes data obtained from multiple data records associated with a particular processing workflow. In some implementations, a wide record is generated in response to a request for data in a data processing workflow. In some implementations, a wide record is generated in real time in response to a request. As real-time data is generated for a key value, a real-time segment for the key value is determined using data from each of the batch storage and the real-time data storage. The wide record includes both real-time and batch data stored for a given key value, enabling the data processing system to determine the real-time segment. Thus, the data processing system can determine the current segment associated with a key value after a particular transaction occurs in response to the transaction occurring. The data processing system uses the wide record to determine the real-time segment associated with the key value within a threshold time period in which the transaction occurs, such as in real time or near real time.

[0011] Real-time or near-real-time processing refers to a scenario in which received data is processed so that it is made available to systems and devices requesting the data immediately (e.g., within milliseconds, tens of milliseconds, or hundreds of milliseconds) after processing of the data is completed, without introducing data persistence or store-then-forward actions. In this context, a real-time system is configured to process data streams as they arrive and output results as quickly as possible (potentially with processing latency). In a pipelined architecture, data may be buffered between module interfaces, but individual modules operate on the most recent data available to them. The overall result, in a real-time context, is a workflow that receives a data stream and outputs processed data based on that data stream in a first-in, first-out manner. However, non-real-time contexts are also possible in which data is stored (either in memory or persistently) for later processing. In this context, modules of a data processing system do not necessarily operate on the most recent data available.

[0012] In one aspect, a process includes receiving, by a data processing system, one or more data items associated with a given key. The process includes detecting the occurrence of a transaction in the one or more received data items. In response to the detecting, the process includes accessing a volatile memory that stores data records for a plurality of keys. The process includes retrieving the data record for the given key from the volatile memory. The process includes specifying the occurrence of the transaction and updating the data record with data according to the received one or more data items associated with the given key. The process includes executing one or more rules on the updated data record, the rules being associated with a segment and specifying one or more conditions, the rules further specifying that if the data item satisfies the one or more conditions, the data item is associated with the segment. The process includes determining, based on the executing, one or more segments associated with the updated data record, the determination of the one or more segments being in near real time relative to the occurrence of the event. The process includes outputting, by the data processing system, instructions specifying one or more actions associated with the determined one or more segments.

[0013] In one aspect, a process implemented by a data processing system for performing real-time segmentation based on receipt of transaction data includes accessing from a hardware storage device one or more segmentation rules, where the segmentation rules specify segments representing one or more predetermined attributes describing an entity and one or more conditions for associating the entity with the segment; receiving, by the data processing system, one or more data items associated with the entity; detecting transaction data in the one or more received data items indicating the occurrence of a transaction; in response to the detecting, accessing a memory that stores data records for a plurality of entities; retrieving the data records for the entities from the memory; and updating the retrieved data record; executing one or more segmentation rules on the updated data record; determining that the updated data record satisfies one or more conditions of at least one of the one or more segmentation rules; and based on a result of the determining, associating an entity for the updated data record with one or more segments specified by at least one of the one or more segmentation rules, wherein the associating is in near real time with respect to an occurrence of the transaction; and outputting, by the data processing system, a notification of the one or more segments associated with the entity, wherein the notification results in performance of one or more actions for the entity in near real time with respect to the occurrence of the transaction, the one or more actions according to one or more predetermined attributes of the one or more segments.

[0014] Other aspects include a computer system and a computer program product.

[0015] One or more of the above aspects may include one or more of the features described herein or one or more of the following features.

[0016] In some implementations, including any of the above or below described implementations, the process includes generating a push notification that instructs the transaction processing system to update a data record associated with a given key when at least one segment associated with the given key changes from a previous segment.

[0017] In some implementations, including any of the above or below described implementations, the process includes receiving a request for a real-time segment associated with a given key as a service.

[0018] In some implementations, including any of the above or below described implementations, the process includes, in response to receiving the request, transmitting one or more segments associated with the updated data record in real time to the system that sent the request.

[0019] In some implementations, including any of the above or below implementations, the process includes providing a user interface for defining one or more rules prior to determining one or more segments.

[0020] In some implementations, including any of the above or below described implementations, the instructions specifying one or more actions associated with the determined one or more segments include an exclusive offer for one of the one or more segments.

[0021] In some implementations, including any of the above or below described implementations, the process includes determining that a given key is associated with at least two segments.

[0022] In some implementations, including any of the above or below described implementations, the process includes generating, by the remote computing system, instructions for performing at least two actions based on the determination.

[0023] In some implementations, including any of the above or below described implementations, the process includes retrieving batch data records associated with a given key from a data warehouse, and updating the data records with data according to the received one or more data items that specify the occurrence of a transaction.

[0024] In some implementations, including any of the above or below described implementations, the process includes generating a virtual record that includes at least a portion of the received one or more data items specifying the occurrence of a transaction and a batch data record associated with a given key.

[0025] In some implementations, including any of the above or below described implementations, the process includes determining, based on the virtual record, one or more segments associated with the updated data record.

[0026] One or more of the above aspects may provide one or more of the following advantages.

[0027] The data processing system is configured to calculate real-time segments for entities associated with a given key. The real-time segments are instantaneous segments of the entities. The real-time segments represent the segments to which each entity or portion of data associated with the entity belongs at the time the real-time segments are calculated. Computing systems such as logistics systems, transaction processing systems (e.g., automated teller machines (ATMs) or point-of-sale (POS) machines), internet brokers, online storefronts, and rewards engines can use the real-time segments immediately (in real time) to determine real-time actions for execution by computing systems (e.g., mobile devices) associated with the entities. The real-time actions respond to the status of the entities at the time the entity's segments are requested. The real-time segments are generated / updated by the data processing system between instances of batch workflow execution. For example, the real-time segments respond intraday to transaction data such that a new segment is determined for each transaction associated with the entity as the transaction occurs. In some implementations, the real-time segments for an entity can differ for each subsequent item of the entity's transaction data processed by the data processing system. The entity may be a user or a technological process or device. When referring to a "user" herein, the same description applies to other types or entities as well.

[0028] The real-time segment for an entity can be provided as a service by a data processing system. For example, a computing device (e.g., an ATM, a POS, etc.) may request a real-time segment for an entity while processing a transaction associated with the entity. When the computing system needs to know the current segment for the entity, the data processing system is invoked to provide the current segment in real time or near real time in response to the request.

[0029] The data processing system can push real-time segments to a computing device to indicate that an entity's current real-time segment has changed and to indicate a different status of the entity. For example, aggregate spending for the entity's account may exceed a threshold during a transaction represented in an item of transaction data. In this case, the entity belongs to a new segment of users or entities whose aggregate spending is greater than the threshold. The data processing system determines an updated segment for the entity in real time when the entity's status changes. The entity's updated real-time segment can be sent to another computing system to provide a real-time notification to that computing device. The computing device can take an action or generate an offer in real time in response to a change in the segment associated with the entity. The computing device does not need to wait for the entity's segment to be updated during batch processing. Thus, actions can respond to the entity's activity as it occurs during the day or during periodic batch processing of the entity's segments.

[0030] Real-time segments do not need to be stored as they become outdated over time. When an entity's real-time segment is needed again, the data processing system recalculates the current real-time segment for the new request.

[0031] Processing that incorporates the use of an entity's segment attributes is more accurate. Real-time segments represent the true segment status of an entity, not the entity's status at the time the batch process workflow was executed. In a batch-based segmentation system, an entity's segment attributes become increasingly inaccurate during the batch process. Batch segments are static until the batch process updates the segment. Transactions that are processed over a longer period after the batch workflow is executed (e.g., later in the day) do not incorporate the intraday results of previous transactions. An entity's batch segment becomes increasingly inaccurate as more transactions are processed for the entity and the batch segment is not updated.

[0032] The real-time segment is generated in an environment that includes batch and real-time processing. As described herein, combining a batch module and a real-time module results in efficient use of memory resources because batch searches can be performed once a day via the batch module and made available throughout the day via volatile memory, and the results of the batch searches can then be supplemented with real-time data. All incoming transactional data items are immediately incorporated into the segmentation to keep the latency associated with data searches low, enabling an accurate and real-time segmentation process. The real-time data is stored in memory before being committed to disk. Thus, rather than storing all batch and real-time data in memory, the memory only needs to store, for example, the previous 24 hours' worth of data, and the remainder of the transactional data for the entities needed for the real-time segment can be retrieved from disk, thereby reducing memory consumption.

[0033] The real-time segmentation described herein allows an entity to simultaneously define multiple different segments for the entity. The entity defines segments based on the selection of different values ​​in a provided graphical user interface. A user can simultaneously define multiple segments. A user can simultaneously define aggregations using many different data sources from a data catalog. Defining (and running) multiple segments simultaneously improves the efficiency of both defining real-time segments and the generated logic that calculates the real-time segments in a data processing system, compared to the efficiency of defining and running multiple segments serially.

[0034] Many systems can easily process a small number of data sets. However, as the number of data sets being processed increases, up to millions of data sets, the complexity of processing these data sets also increases. This is because efficient processing of large numbers of data sets requires a scalable system, which in turn requires optimization, logical access of data, and system feedback, each of which is addressed below.

[0035] To achieve efficiency in data processing, a dataflow graph is optimized. The dataflow graph includes data processing components, some of which have specified functionality, such as being pre-configured to perform partition and sort operations.

[0036] An example of an optimization is when an entity specifies that a particular component performs partitioning and sorting, but if a component preceding the particular component is configured for partitioning and sorting, the computer program is optimized to avoid performing the partitioning and sorting operations twice.

[0037] During optimization, memory resources are also efficiently allocated to increase efficiency in data processing by removing data items from memory that are no longer utilized by the dataflow graph. In particular, data being processed by the dataflow graph may also be stored, for example, in volatile or virtual memory. However, not all components of the dataflow graph need access to or use every item of data. For example, an upstream component may utilize an item of data, but a downstream component may not. In this example, the data processing system is configured to identify when an item of data is not utilized by a downstream component and delete or otherwise remove that data item from memory, thereby reducing memory usage, for example, relative to the memory usage of storing all data items until the completion of the dataflow graph.

[0038] The data processing system described herein is configured to process data items in a real-time workflow, independent of data preprocessing. Rather, the data processing system is configured to process data processed in a batch or real-time workflow to generate data records containing relevant data that are useful real-time calculations, such as segmentation. By streaming these data items to memory, the data processing system can provide live responses (using segmentation) to transactions (events) as they occur. The memory includes volatile memory in a real-time workflow that stores these data items. For example, data not relevant to performing real-time segmentation is removed from the transaction data and attribute data of a keyed data record. Instead, a record with the data necessary for processing by the segmentation engine is generated. The data processing system is configured to enable near-instantaneous or live-time segmentation in response to the content of a transaction (event) of a data record associated with an identifier when it is received, and to segment data of the same identifier, which also provides near-instantaneous or live-time visibility of the application results for this identifier.

[0039] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims. [Brief explanation of the drawings]

[0040] [Figure 1] FIG. 1 is a block diagram of a system. [Figure 2] FIG. 1 is a block diagram of a system. [Figure 3A] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation definition stage. [Figure 3B]FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation definition stage. [Figure 3C] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation definition stage. [Figure 3D] 1 shows an example of a virtual record for generating a real-time segment. [Figure 4A] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation stage where real-time segments are generated. [Figure 4B] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation stage where real-time segments are generated. [Figure 4C] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation stage where real-time segments are generated. [Figure 5A] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation definition stage, including controls for real-time or batch segmentation. [Figure 5B] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation definition stage, including controls for real-time or batch segmentation. [Figure 5C] FIG. 3 is a diagram of the system of FIG. 2 at the just-in-time segmentation definition stage, including controls for real-time or batch segmentation. [Figure 6] FIG. 1 is a flow diagram illustrating just-in-time segmentation in which real-time segments are generated. [Figure 7] FIG. 1 is a flow diagram illustrating just-in-time segmentation in which real-time segments are generated. [Figure 8A] FIG. 2 illustrates a user interface. [Figure 8B] FIG. 2 illustrates a user interface. [Figure 8C] FIG. 2 illustrates a user interface. [Figure 8D] FIG. 2 illustrates a user interface. [Figure 9] FIG. 1 illustrates details of a computer system, such as a data processing system. DETAILED DESCRIPTION OF THE INVENTION

[0041] 1 illustrates a system 100 that includes a transaction processing system 106 configured to process transaction data 104a-104c (collectively, transaction data 104) and determine segments 108 associated with the processed transaction data 104. The transaction processing system 106 receives a stream of items of transaction data 104a-104c from a real-time data source 102 in a real-time workflow 101. The real-time workflow 101 operates continuously or near-continuously (e.g., intraday) as transactions are performed. The transaction data 104 represents data processing transactions as they occur. Transaction data 104 is generated (e.g., by another computing system, not shown) to represent data processing transactions (also called events) as they occur.

[0042] A transaction represented by transaction data 104 may require some action to be performed in real time or near real time in response to the transaction. For example, when an entity performs an action using a computing device, such as making a payment, crediting an account, sending a text message, making a phone call, or any other event, a computing system associated with the event (e.g., mobile device 126) may need to determine the segment to which the user belongs. Based on the segment, computing system 126 can present a desired offer or action to the user as part of processing the transaction. Computing system 126 determines the action to perform based on determining the segment 108 to which the user belongs, in real time or near real time with respect to the occurrence of the transaction, or as part of processing the transaction.

[0043] The transaction processing system 106 determines which offer or action 110 to take with respect to the item of transaction data 104a-104c being processed based on the determined segment 108 for the item of transaction data 104a-104c. For example, the transaction processing system 106 can specify an action to be taken for a customer, subscriber, user, etc. associated with the item of transaction data 104a-104c. The transaction processing system 106 transmits data including the specified action or offer 110 to a remote device, such as a mobile device 126, associated with the owner of the item of transaction data 104a-104c. For example, the mobile device 126 can perform a first action or offer based on users belonging to a first segment, or can perform a second action or offer based on users belonging to a second segment that is different from the first segment.

[0044] Real-time or near-real-time processing refers to a scenario in which received data is processed so that it is made available to systems and devices requesting the data immediately (e.g., within milliseconds, tens of milliseconds, or hundreds of milliseconds) after processing of the data is completed, without introducing data persistence or store-then-forward actions. In this context, a real-time system is configured to process data streams as they arrive and output results as quickly as possible (potentially with processing latency). In a pipelined architecture, data may be buffered between module interfaces, but individual modules operate on the most recent data available to them. The overall result, in a real-time context, is a workflow that receives a data stream and outputs processed data based on that data stream in a first-in, first-out manner. However, non-real-time contexts are also possible in which data is stored (either in memory or persistently) for later processing. In this context, modules of a data processing system do not necessarily operate on the most recent data available.

[0045] To process the transaction data 104, the transaction processing system 106 retrieves segments 108 from the batch process workflow 103. A segmentation system 118 in the batch process workflow 103 processes the batch transaction data 104 periodically (e.g., once a day) to determine batch segment definitions 124, such as users, computing systems, customers, subscribers, etc. A batch search engine 116 in the segmentation system 118 searches the transaction data 104. The transaction data 104 is retrieved in bulk so that the segmentation engine 120 can define segments for a set of users based on all transaction data 104 (and other data, if applicable) that have been processed so far.

[0046] The segmentation engine 120 defines segments 124 (including segments 108) for a set of indexes (e.g., representing users, subscribers, computing systems, etc.). The segmentation engine 120 defines the batch segments 124 based on a set of segmentation rules 122. The segmentation rules include logic that associates users with segments based on one or more attributes associated with the users.

[0047] Although the real-time workflow 101 is configured to process the transaction data 104 in real time, the segment 108 is an old segment determined based on the batch data of the batch workflow 103. Therefore, the segment 108 is not a real-time segment. The segment 108 is updated only when the batch workflow 103 executes (e.g., once a day). The transaction processing system 106 cannot access updated segments between running instances of the batch workflow 103 when processing items of the transaction data 104a-104c. The segment 108 is static between updates by the batch workflow 103. For example, if several items of the transaction data 104 are associated with a particular user who is assigned a given segment 108 during the day, each item of the transaction data 104 is processed based on that segment for that user. The user's segment cannot be updated during the day, even if the user's segment changes based on previous items of the transaction data 104a-104c processed during the day. Rather, items of transaction data 104a-104c are sent to a load module 112. The load module 112 sends items of transaction data 104 processed during the day in the real-time workflow 101 to data storage, such as a data warehouse 114. The items of transaction data 104 are then processed in bulk by a batch workflow 103 to update the segments periodically (e.g., once a day).

[0048] The system 100 does not process items of transaction data 104a-104c based on real-time segments. The segments 108 are updated only when the batch workflow 103 executes. The offers 110 and actions generated by the transaction processing system 106 are not real-time offers or actions. The offers or actions 110 do not respond to values ​​in the items of transaction data 104a-104c. Furthermore, the generated segments 124 do not respond to cross-day processing by the real-time workflow 101. The generated offers or actions 110 may become stale, repetitive, outdated, or inaccurate as time passes between instances of execution of the batch workflow 103. For example, if the batch workflow 103 executes once a day, actions or offers 110 generated earlier in the day will be more accurate than offers or actions generated later in the day.

[0049] FIG. 2 is a block diagram of a system 200 that performs just-in-time segmentation. Just-in-time segmentation involves generating segments for entities in real time in response to a request for a segment of the entity. A data processing system 210 receives items of transaction data 104 in a stream from a real-time data source 204. The real-time data source 204 may include a message bus 204 that receives real-time data, such as transaction data 302, as a real-time stream as the real-time data is generated by one or more remote devices. For example, as transactions occur at a point-of-sale system, the generated transaction data may be streamed to the real-time data source 204. The real-time data is available from the real-time data source 204 over a period of time (e.g., two days). A collection module 206 or other system can subscribe to one or more fields of real-time data from the real-time data source 204. By subscribing to one or more fields of the real-time data source 204, the collection module 206 or other system receives corresponding real-time data for the one or more fields as a stream in real time as the data is available on the message bus.

[0050] Additionally, data processing system 210 receives previous batch transaction data 105 from data warehouse 114, for example, once a day. The previous batch data 105 is a portion of the batch data stored in persistent memory (such as data warehouse 114) and is therefore available for a longer period of time than the period of real-time data in real-time data source 204.

[0051] In the prior art of Figure 1, the batch search engine 116 performs batch access to transaction data 104 in the warehouse 114 once at the start of each workday, and the segments calculated at the start of the day determine what actions are indicated. Other transaction data 104a-104c coming in for the remainder of the day are stored in the warehouse 114 by the load module 112 and do not trigger segment adjustments and / or new actions during the same day. Only the next day is the previous day's transaction data again batch retrieved from the warehouse 114, and the indicated segments and / or new actions are adjusted.

[0052] In contrast, in the system of FIG. 2 , the data processing system 210 is configured to calculate real-time segments 222a-222c (collectively, real-time segments 222) for a user at essentially any time within a given period, such as a day. More typically, the segments are calculated in real time or near real time, such that an entity's segments are updated immediately in response to transactions. The real-time segments are instantaneous segments of the entity. The real-time segments represent the segments to which each user belongs at the time the real-time segments 222 are calculated. A computing system, such as the transaction processing system 230, can immediately (in real time) use the real-time segments 222 to determine real-time actions or offers 224 for implementation by a computing system associated with the user, such as the mobile device 126. The real-time offers 224 are therefore responsive to the user's status at the time the user's segments are requested. Thus, the real-time segments 222 can be generated / updated by the data processing system 210 during instances of batch workflow execution. For example, the real-time segments 222 are responsive to transaction data 104 during the day. In some implementations, the real-time segments 222a-222c for an entity may vary (can be updated) for each item of user transaction data 104a-104c processed by the data processing system 210.

[0053] A request for a user's real-time segments can occur in a variety of scenarios. The user's real-time segments 222 can be provided as a service by the data processing system 210. For example, a computing system (such as the mobile device 126) may request an entity's segments while processing a transaction associated with the user. When the computing system needs to know the user's current segment, the data processing system 210 is invoked to provide the current segment in real time or near real time in response to the request. The segments are not only accessed in real time by the data processing system 210. Rather, the data processing system 210 calculates the user's segments in real time based on the user's status, and the user's segments incorporate any transaction data 104 or other attributes associated with the user that are updated in real time. The real-time segments 222a-222c represent instantaneous segments associated with the user. The real-time segments 222a-222c are used in real time by the computing system that requested the real-time segments 222a-222c, such as the mobile device 126. The real-time segments 222 do not need to be stored because they become outdated over time. If the user's real-time segments are needed again, the data processing system 210 recalculates the current real-time segments for the new request.

[0054] The data processing system 210 can generate real-time segments 222 as transaction data 104 is processed. The data processing system 210 can generate real-time segments 222a-222c without a request from the transaction processing system 230. Rather, as an entity's segments change, the data processing system 210 can push updated real-time segments 222a-222c to the transaction processing system 230, causing offer or action 224 instructions to be generated by the transaction processing system 230. In this example, the transaction processing system 230 receives the real-time segments 222a-222c. In response to receiving the real-time segments 222a-222c, the transaction processing system 230 executes a process to determine whether to generate an offer or action instruction 224 based on the new segments.

[0055] An example of pushing real-time segments is described below. The aggregate spending for an entity's account may exceed a threshold during a transaction represented in an item of transaction data 104a-104c. In this case, the user belongs to a new segment of users whose aggregate spending is greater than the threshold. In the system 100 of FIG. 1, the user's segment is not updated until the batch workflow 103 executes and updates the user's batch segment 124. Only after the batch workflow 103 executes is the user's new segment accurate for subsequent transactions. In contrast, the system 200 calculates a segment for the user in real time based on already received items of transaction data 104a-104c for the user during that day. The segmentation engine 120 of the data processing system 210 calculates the user's real-time segments 222a-222c, which accurately represent the user's new segment 222a. For example, the new segment 222a specifies that the user is in a segment of users whose aggregate spending exceeds the threshold. The data processing system 210 sends the real-time segment 222a to the transaction processing system 230, which detects that the user's real-time segment has changed. The transaction processing system 230 generates an offer 224 for the user, and the user qualifies for the offer by being in the user's segment with the required aggregate spending. For example, if the user spends a sufficient amount with a rewards credit card, the transaction processing system 230 can generate an offer immediately when the user's aggregate spending exceeds a specified 30-day threshold (or meets any other metric defined in the segmentation rules 122). The offer is generated in real time in response to the transaction that changed the user's segment, rather than after a period of time has passed. This can enable a computing system, such as a point of sale system, to generate real-time discounts or rewards for an entity based on which segment the user is in at that given time.

[0056] The data processing system 210 receives items of transaction data 104 from the real-time data source 204. The data processing system 210 receives prior transaction data 105 from the data warehouse 114. The prior transaction data 105 represents transaction data stored in batches. The batch search engine 116 accesses the data warehouse 114 to retrieve relevant prior transaction data 105 (e.g., transaction data 105 associated with a user identified in an item of transaction data 104a-104c). The data processing system 210 loads the prior transaction data 105 into memory so that the transaction data 105 is available for real-time processing in the real-time segmentation invoker 212.

[0057] The collection module 206 collects items of transaction data 104a-104c for loading by the load module 112 into the data warehouse 114. These items of transaction data 104 are combined with previous transaction data 105 in a batch update process (e.g., once a day). The collection module 206 collects transaction data 104 generated on real-time data sources 204 and prepares the transaction data for batch loading into the data warehouse 114. The collection module 206 also forwards the transaction data to the real-time segmentation invoker 212 for processing in a real-time workflow. Thus, the transaction data 104 is available as previous transaction data 105 for the next period after batch loading (e.g., the next day), while the transaction data 104 is processed by the real-time segmentation invoker 212 and the segmentation engine 120 to update the real-time segments in real time. If desired, the data processing system 200 can process the previous transaction data 105 to generate batch segments (e.g., for a collection of entities) in addition to the real-time segments generated for each entity.

[0058] The real-time segmentation invoker 212 is configured to access the real-time transaction data 104 and the previous transaction data 105 to determine whether a real-time segment should be generated. The real-time segmentation invoker 212 accesses the items of transaction data 104a-104c using a real-time module 214 that receives the items of transaction data 104. The real-time segmentation invoker 212 accesses a volatile memory 220 that stores the previous transaction data 105 and associated user attributes. The real-time segmentation invoker 212 identifies an entity associated with the item of transaction data 104a-104c being processed. The real-time segmentation invoker 212 accesses attributes associated with the user, as described below.

[0059] The real-time segmentation invoker 212 generates, via a record generator module 216, records associated with the items of transaction data 104a-104c being processed. The records include all relevant data for a user for determining the user's real-time segment. For example, the records generated by the module 216 include user attributes, the user's previous transaction data 105, and the items of current transaction data 104a-104c received for the user. A user can be identified based on data included in the items of transaction data 104a-104c being processed. For example, each user can be represented by an index value, such as a key value. In some implementations, records are not actually generated, but are virtual records of data available in the real-time processing workflow associated with the user's key value.

[0060] The detection module 218 is configured to detect whether a virtual record has changed for an entity based on receiving an item of transaction data 104a-104c. When an item of transaction data 104a-104c changes the entity's virtual record, the detection module generates a notification (described in more detail below). The notification is output by the real-time segmentation invoker 212. The notification invokes the segmentation engine 120 to generate real-time segments for the user based on the virtual records generated by the record generator module 216.

[0061] The segmentation engine 120 is configured to generate real-time segments 222a-222c based on the virtual records and notifications associated with the user's key value received from the real-time segmentation invoker 212. When invoked by a notification specifying the key value, the segmentation engine 120 applies the segmentation rules 122 to the virtual records for the user's key. The segmentation engine 120 generates real-time segments 222a-222c for the user, as described above. The segmentation engine 120 outputs the real-time segments 222a-222c to a downstream system configured to consume the real-time segments, such as the transaction processing system 230 (described above).

[0062] 3A-3C are diagrams of the system 200 of FIG. 2 at a just-in-time segmentation definition stage. In FIG. 3A, the data processing system 210 is in an initial stage 300a for just-in-time segmentation definition. In stage 300a, the data processing system 210 is configured to receive segmentation rules 122 from the client device 202. The segmentation rules 122 include definitions of each segment for a user. For example, the segmentation rules 122 specify what values ​​are required for various attributes of the user for an entity to be in a given segment. The segmentation rules can include any desired number of segments. The segmentation rules 122 include logic for determining which segments are associated with the user. The logic specifies a priority for applying the rules of the segmentation rules. A user interface on the client device 202 is provided to allow the entity to define the segments as desired. The user interface is described in connection with FIGS. 8A-8D.

[0063] Segmentation rules 122 include rules for associating key values ​​(of an entity) with each segment defined in the segmentation rule. A rule may express conditions that must be met to assign a segment to a key value. Rules may be executed in an ordered sequence to apply the logic of the rule to records generated by the record generator module. If each criterion is met by the record's data, the user is within each segment defined by those criteria, and the user's key value is associated with that segment.

[0064] The data processing system 210 can associate one or more real-time segments with an entity's key value. A key value is not limited to a single real-time segment if the real-time segment is defined to allow an entity to exist in multiple segments simultaneously. The segmentation engine 120 identifies all segments that qualify for a record of the key value. When real-time segment data for a key value is needed by the transaction processing system 230, each real-time segment (along with the key value) is sent to the transaction processing system 230.

[0065] 3B illustrates a state 300b of the data processing system 210 for generating a virtual record 306 for an entity based on a user's key value. The virtual record 306 is a wide record that includes both real-time data from the real-time data sources 204 (e.g., transaction data 302) and batch data from the data warehouse 114 (e.g., previous transaction data 107). The virtual record 306 may also include other data associated with the key value 301 associated with the particular transaction data 302 being processed by the data processing system 210. The virtual record 306 combines both the batch data and the real-time data into a common record, allowing the segmentation engine 120 to determine the real-time segments associated with the key value 301 in real time or near real time as the transaction data is processed in a workflow.

[0066] Referring now to Figure 3D, a scaled-down version of Figure 3B is shown with the real-time segmentation invoker 212 highlighted. Refer to Figure 3B as needed to identify where the real-time segmentation invoker 212 is located relative to the other elements of Figure 3B. Also shown in Figure 3D is a wide record 306 generated by the real-time segmentation invoker 212. The wide record 306 includes a number of fields, including, for example, fields 340, 342, 344, 346...348. Some of the fields 340, 342...348 may be vectors of fields and / or vectors of vectors. In one example, the wide record includes a data structure structured with fields, where each field stores or is associated with a data value.

[0067] Wide record 306 shows 2146 fields, including a field 340 that stores a four-character key (e.g., key 301), customer information (batch field) 342 that is represented by multiple fields, vector field (batch field) 344 that is represented by multiple fields, aggregate spend (real-time field) 346, and transaction field (real-time field) 348. Real-time segmentation invoker 212 generates wide record 306 as follows: 7342;34...2,...,

[23] ...

[0431] ;201.21,13, Key;Batch field;Real-time field Here, wide record 306 includes, in the batch fields for key 7342, age "34," SMS data 431 for messages 23, aggregate spend 201.21, and transaction count 13. Segmentation engine 120, described in more detail below, is configured to execute the logic of segmentation rules 122 that specify a particular segment associated with key value 7342 based on one or more of the values ​​of fields 342, 344, 346, and 348. Examples of segmentation rules 120 are described below in connection with FIGS. 3D, 3C, 4A, 4B, and 4C. Based on the execution of the logic of the segmentation rules, data processing system 202 is configured to determine a real-time segment associated with key 7342. Transaction processing system 230, described below, can determine whether an entity (e.g., subscriber, customer, company, enterprise, etc.) associated with key value 7342 is eligible for a real-time offer based on the real-time segment determined by segmentation engine 120. The transaction processing system 230 generates real-time offers 312 that are sent to and received by devices such as the mobile device 126 .

[0068] As described above, some of the information in the wide record includes batch data, and some of the information includes real-time information. To begin the process of receiving the wide record, the segmentation engine 120 can send a request for virtual records 306 to the real-time segmentation invoker 212. In one example, the real-time segmentation invoker 212 sends the virtual records 306 to the segmentation engine 120 in response to an update of the virtual record, without a request from the segmentation engine. The real-time segmentation invoker 212 sends a notification 304 with the updated virtual record 306, as described below. Each of the notification 304 and the virtual record 306 is associated with a key value of the key 301. The segmentation engine 120 is triggered to execute the segmentation rules 122 in response to receiving the concepts 304 and the virtual records 306 within a threshold period, e.g., in real time.

[0069] Wide record 306 includes batch fields 342...344, which, as mentioned above, are four generations older than the real-time segments. Wide record 306 also includes real-time fields 346...348, which are current and up-to-date in real time relative to when transaction data 302 was generated. As shown in this example, by specifying in advance which fields 346...348 are real-time fields and which fields 342...344 are batch fields, data processing system 210 conserves system resources (e.g., processing and memory resources) by not having to continually generate up-to-date data records and by not having to retrieve values ​​for all fields from persistent memory in real time. This, if necessary, would introduce latency that would prevent the generation of real-time segments in real time relative to the generation of transaction data 302. Rather, at the time of generation of transaction data 302, data processing system 210 retrieves only the values ​​of a subset of fields used by segmentation engine 120 for the execution of the logic of segmentation rules 122. These subset of fields are real-time fields 346...348.

[0070] This technique is very effective when real-time data records are not needed frequently, but when real-time data records are needed, it is important that some fields are up to date in real time relative to when the request is received. Thus, these techniques provide improved computational efficiency for computing real-time records in response to requests because the records do not need to be continually updated, but rather are updated only when needed.

[0071] Furthermore, the system described herein generates wide records 306 when a certain amount of latency (approximately 1 ms) is acceptable in providing the records in response to detecting new transaction data 302 available in the real-time data sources 204. In other instances, latency is not acceptable in generating the records 306. In these instances where latency is not acceptable, the wide records are continually updated even when the records 306 are not in use and the tradeoff (for continually updating the records) is increased consumption of processing power and memory resources. However, when this certain amount of latency is acceptable, the system described herein takes advantage of the ability to generate wide records 306 on-demand using real-time data retrieved from the real-time data sources 204, thereby improving memory consumption and computational resources (compared to those required to constantly update the records) due to the fact that only the information actually requested is retrieved from the operational system and used in updating the data records. Furthermore, classifying data fields as either real-time or batch fields reduces the amount of latency in generating records in response to a request (compared to the amount of latency in retrieving all data from the operational system in real time). When configuring a system, this classification is done so that data of relatively high importance (e.g., to rule execution) or that changes frequently is classified as "real-time," and data that changes less frequently or is less important to real-time decisions (through rule execution) is classified as batch. Thus, the techniques described herein reduce the latency and memory resources in generating these wide records, while improving computational efficiency in doing so.

[0072] 3B, the real-time segmentation invoker 212 uses the real-time module 214, the record generator module 216, and the detection module 218 to process items of transaction data 104a-104c and determine whether a real-time segment should be generated by the segmentation engine 120. If the invoker 212 determines that a real-time segment should be computed by the segmentation engine 120 for a key value, the invoker generates a notification 304 to invoke the computation of the real-time segment and sends the associated virtual record 306 to the segmentation engine 120.

[0073] To begin processing an item of transaction data 104a-104c, the data processing system 210 determines a key value 301 specified in or included in the item of transaction data 104a-104c. The key value 301 generally indicates an entity, subscriber, computing system, or other entity associated with the item of transaction data 104a-104c. The data processing system 210 requests related prior transaction data 105 from the data warehouse 114. The prior transaction data 105 is associated with the same key value 301. The data from the data warehouse may also include any attributes or other metadata that describe the user, and such attributes or metadata may be used to determine real-time segments of the user.

[0074] A portion of the prior transaction data 105 from the wafer house 114 can be loaded into the volatile memory 220 (real-time workflow) of the data processing system 210, for example, by the batch search engine 116, before receiving any items of transaction data 104a-104c, such as at the start of each work period, such as a day. The portion of the prior transaction data 105 loaded into the memory 220 in the real-time workflow is referred to as subscribed prior transaction data 107. The subscribed prior transaction data 107 is loaded in bulk (e.g., when a batch processing workflow updates the prior transaction data 105). The selection fields (or specific transactions) of the subscribed prior transaction data 107 are based on subscriptions of transactions from the real-time data source 204 for the real-time segmentation invoker. The subscription specifies for which transactions real-time segments can be generated. The subscription is defined based on which real-time segments are defined in the segmentation rules 122 and based on user input (e.g., before processing the data in the real-time workflow of the data processing system 210). The availability of transaction data 105 in volatile memory 220 can reduce the latency of determining the real-time segments of an entity. These data 107 can be stored and updated in memory 220 each time a batch process is executed.

[0075] The subscribed prior transaction data 107 includes specific data selected based on which real-time segments are defined in the segmentation rules 122. For example, different real-time segments may use different fields in the prior transaction data 105. When the batch search engine 116 generates the subscribed batch aggregates for storage in the memory 220 in the real-time segmentation invoker, the prior transaction data 105 includes any batch data needed to generate the real-time segments 310 in real time from the transaction data 302. The specific data in the prior transaction data 105 may vary from period to period (e.g., from day to day) between updates to the prior transaction data 105 in the data warehouse 112. For example, a user may define different real-time segments for a given period and specify the set of transactions for the subscription from which the real-time segments are determined via the interface of the device 202.

[0076] The real-time module 214 receives the items of transaction data 104a-104c. The real-time module 214 extracts data from the items of transaction data 104a-104c relevant to processing the transaction data 104 for real-time segment determination. Other data from the real-time data stream, such as headers, addressing information, etc., can be removed by the real-time module 214.

[0077] The record generator module 216 receives items of transaction data 104a-104c and combines the real-time transaction data with previous transaction data 105 accessed from volatile memory 220. In contrast to the prior art of FIG. 1, batch retrieval from the warehouse 114 is not required when or after items of transaction data 104a-104c are received within a current period (e.g., a day), thus allowing items to be processed quickly, e.g., in real time, and still have all relevant transaction data available for an accurate, real-time segmentation process. The record generator module 216 generates comprehensive records for key values, which can be referred to as virtual records. The virtual records 306 include any data used to determine real-time segments of users associated with the key values. The virtual records 306 do not need to be generated as actual data items. Rather, the virtual records 306 allow the segmentation engine 120 to access any data needed for a given key associated with the transaction data 104a-104c.

[0078] For a given key, the detection module 218 determines whether a record 306 represents a change to any of the associated fields that may affect the user's real-time segment. The detection module 218 detects when the data in the associated fields has changed. If this occurs, the detection module 218 generates a notification 304 that invokes the segmentation engine 120 to determine the real-time segment for the key value. Generating the notification 304 enables the segmentation engine 120 to generate a real-time segment only when transaction data 104a-104c potentially causes a change to the real-time segment for the key value. The segmentation engine 120 does not need to calculate an updated real-time segment for each key value in the received transaction data 104a-104c. This saves computational bandwidth. However, if additional real-time transaction data is received for additional items of transaction data 104a-104c, this data can be combined with already received real-time transaction data for another received item of transaction data 104a-104c. This is because this transaction data is still available from the real-time data sources 204 - 206 and is combined with the previous transaction data 105 accessed from memory 220 .

[0079] The segmentation engine 120 determines real-time segments for a key value using segmentation rules 122 defined on the client device 202. The segmentation engine 120 calculates real-time segments for a key value by processing records 306 associated with the key value when provided with a notification 304 that wakes the segmentation engine 120 to proceed.

[0080] The segmentation engine 120 applies the segmentation rules 122 by applying the conditions specified in the segmentation rules. The segmentation engine 122 identifies one or more segments 308a-308b that satisfy the segmentation rules for the key values.

[0081] The segmentation engine 120 outputs a real-time segment 310 for the key value 301 representing the user. The real-time segment 310 can include any one of the available segments 308a-308b when those segments satisfy the segmentation rules 122.

[0082] The transaction processing system 230 receives the real-time segment 310 and determines one or more actions that can be performed on a remote computing device associated with the user represented by the key value. The transaction processing system 230 generates instructions for execution of the action. For example, the action may include presenting an offer 312 (e.g., a discount, a voucher, an upgrade, etc.) to an entity based on the new real-time segment 310 into which the user falls. Data including the offer 312 is transmitted to the remote computing device (e.g., the mobile device 126) for execution of the instructions specified in the offer data 312.

[0083] FIG. 3C shows a state 300c representing the processing of additional transaction data 322 to update the real-time segment 330 associated with the key value 301. The process in FIG. 3C is similar to the process in FIG. 3B. The additional transaction data 322 is associated with the same key value 301 as the transaction data 104a-104c. The real-time segmentation invoker processes the transaction data 322 and generates an updated virtual record 326 that includes the updated transaction data 322. The invoker 212 determines that the updated virtual record 326 can potentially change the real-time segment for the key value 301. Therefore, the invoker generates an updated notification 324 that indicates to the segmentation engine 120 that an updated segment for the key value 301 should be calculated. The additional transaction data 322 may be transaction data received on the same day or subsequent to the item of transaction data 104a-104c.

[0084] The segmentation engine 120 processes the updated record 326. The segmentation engine determines that the updated record 326 does not satisfy the first segment 308a (which record 306 satisfied), but rather satisfies the second segment 308b. The segmentation engine 120 generates data representing an updated real-time segment 330 for the key value 301. The updated real-time segment is sent by the data processing system 210 to the transaction processing system 230, which generates instructions for the presentation of an updated real-time offer 332. These instructions are sent to the mobile device 126 for execution and presentation of the offer.

[0085] 4A-4C are diagrams of the system of FIG. 2 at the just-in-time segmentation stage, where real-time segments are generated. In stage 400a of FIG. 4A, data processing system 210 receives transaction data 402. Transaction data 402 includes a key value ID=2324. Transaction data 402 also includes other data, such as a current segment "Level=Silver." Transaction data 402 includes attribute data, such as a total expenditure of $101.51 and a date of birth "DOB=4 / 5 / 1988."

[0086] The data processing system 210 accesses previous transaction data 105 for key value ID=2324 from the data warehouse. These data are batch transaction data 412. The batch transaction data 412 specifies that the aggregate spend for key value ID=2324 is $56.31 and that the total number of SMS messages associated with the key value is 17. The batch search engine 116 extracts fields of the data 412 that are related to the subscribed transactions. In this example, the data for the SMS aggregate may be discarded and not stored in memory 220. The batch search engine 116 generates a subscribed batch aggregate 414 that includes the relevant fields for key value 2324. The batch search engine sends the subscribed batch aggregate 414 to memory 220.

[0087] The fields of the subscribed batch aggregate data 414 are based on which real-time segments are defined in the segmentation rules 122. For example, different real-time segments may use different fields of the batch transaction data. When the batch search engine 116 generates the subscribed batch aggregates for storage in the memory 220 in the real-time segmentation invoker, the subscribed batch aggregate data 414 includes any batch data needed to generate the real-time segments 422 in real time from the transaction data 402.

[0088] The real-time module 214 removes data that is not necessary for the calculation of the real-time segments. For example, fields related to subscribed transactions 404 can be extracted from the transaction data 402. The real-time module 214 generates data 404 that includes ID, level, and spend fields.

[0089] The record generator module 216 combines the subscribed transaction data 404 with the subscribed batch aggregate 414 data to generate records 408. The records 408 may be virtual records representing all relevant data for a key value needed by the segmentation engine 120 to calculate the real-time segments associated with the key value 2324. The record generator module 216 generates records having fields including, for example, a detected spend field 416 containing incremental spend data 418 and aggregate spend data 420. The record generator module 216 combines data from the transactions 402 with the batch aggregate 414. For example, the incremental spend of $101.51 in the transaction data 402 is combined with the aggregate spend 420 in the batch aggregate 414 to generate a total aggregate spend of $157.82.

[0090] The detection module 218 determines that the relevant fields of the segment have been included in or changed in the record 408. The change can be detected based on checking a list of relevant fields for real-time segmentation (including the "expense" field 416) or by comparing the record to previous records associated with the key value. If the record 408 contains an updated value, the detection module 218 generates a notification 410 specifying that a "record has been updated" for key ID=2324. The notification 410 and the record 408 are sent to the segmentation engine 120.

[0091] The segmentation engine applies segmentation rules 122 to the generated records 408. The segmentation engine 120 can apply conditions for each segment to the data in the generated records 408. For example, for segment 1 422a, the segmentation engine checks whether Level=Silver, whether the aggregate spend is greater than 200, and whether the incremental spend value is greater than 0 (indicating that the user spent some money in the most recent transaction 402). Based on record 408, the segmentation engine 122 determines that the Level=Silver condition is met (indicated by a check mark). The segmentation engine determines that the condition "Aggregate spend > 200" is not met, as indicated by an "X" mark. The segmentation engine does not need to check the "Incremental spend > 0" condition because segment 1 does not apply.

[0092] The segmentation engine 120 then applies the conditions for segment 2 422b. The segmentation engine 120 determines that the key value ID=2324 is not in segment 1, therefore, the first condition is met. The segmentation engine 120 determines that the second condition, "Level=Silver," is met. The segmentation engine 120 determines that the third condition, "Aggregate Spend >= 150," is met because the aggregate spend is currently $157.82 for record 408, which is a combination of the incremental spend of $101.51 and the aggregate spend of $56.31. The segmentation engine determines that the fourth condition, "Incremental Spend >= 100," is met because the incremental data 418 from transaction data 402 is $101.51. Therefore, all four conditions are met for segment 2 422b. Key value ID=2324 is associated with real time segment 2. Real time segment 422 is sent to transaction processing system 230 as described above to generate real time offer 424 for "free car rental offer" as described above.

[0093] In stage 400b of Figure 4B, data processing system 210 receives additional transaction data 432. This additional transaction data 432 is processed by invoker 212 to determine whether updated real-time segments should be calculated by segmentation engine 120. Transaction data 432 includes the key value ID=2324 that was included in transaction data 402 of Figure 4A. Transaction data 432 also includes other data, such as the current segment "Level=Silver." Attribute data is included in updated transaction data 432, such as incremental spending total=$51.10 and date of birth "DOB=4 / 5 / 1988."

[0094] The data processing system 210 has already stored in memory 220 the previous transaction data 105 for key value ID=2324. The value of aggregate spend 420 for field spend 416 is already stored in memory 220. The batch transaction data specifies that the aggregate spend for key value ID=2324 is $157.31, which includes the batch aggregate spend of $56.31 in transaction data 402 and the incremental spend of $101.51.

[0095] The real-time module 214 removes data that is not necessary for the calculation of the real-time segments, as described above. For example, fields related to the subscribed transaction data 432 can be extracted from the transaction data 432. The real-time module 214 generates the subscribed transaction data 434, which includes ID, level, and spend fields.

[0096] The record generator module 216 combines the subscribed transaction data 434 with the subscribed batch aggregate data to generate updated records 436 that incorporate incremental spending data from the additional transaction data 432. The record generator module 216 combines data from the transaction data 434 with the batch aggregate 414. For example, the incremental spending of $101.51 in the transaction data 402 and the incremental spending of $51.10 in the additional transaction data 432 are combined with the aggregate spending 420 in the batch aggregate 414 to generate a total aggregate spending of $208.92.

[0097] The detection module 218 determines that the relevant field for the segment is included in or has changed in the record 436. The change can be detected based on checking a list of relevant fields for real-time segmentation (including the "expense" field 416) or by comparing the record to a previous record associated with the key value. If the record 436 contains an updated value for the expense field 416, the detection module 218 generates a notification 410 specifying that a "record has been updated" for key ID=2324. The notification 410 and the updated record 436 are sent to the segmentation engine 120.

[0098] The segmentation engine applies the segmentation rules 122 to the updated record 436. The segmentation engine 120 can apply the conditions for each segment to the data in the updated record 436. For example, for segment 1 422a, the segmentation engine checks whether level=silver, whether the aggregate spend is greater than 200, and whether the incremental spend value is greater than 0 (indicating that the user spent some money in the most recent transaction 402). Based on record 436, the segmentation engine 122 determines that the level=silver condition is met (indicated by a check mark). The segmentation engine determines that the condition "aggregate spend > 200" is met. The segmentation engine determines that the condition "incremental spend > 0" is met. Therefore, segment 1 is applied to key value ID=2324. This updated real-time segment 440 is sent to the transaction processing system 230 as described above. The transaction processing system generates updated real-time offers 442 based on the applicable segment 1. The offer specifies a "free flight."

[0099] The segmentation engine 120 then applies the condition for segment 2 422b. The segmentation engine 120 determines that the key value ID=2324 is in segment 1, and therefore the first condition is not met, as indicated by the "X" mark. Checking of segment 2 is complete.

[0100] In stage 400c of Figure 4C, data processing system 210 receives additional transaction data 450. This additional transaction data 450 is processed by invoker 212 to determine whether updated real-time segments should be calculated by segmentation engine 120. Transaction data 450 includes the key value ID=2324 that was included in transaction data 402 of Figure 4A. Transaction data 450 also includes other data, such as the current segment "Level=Silver." Attribute data is included in updated transaction data 450, such as incremental spending total=$101.10 and date of birth "DOB=4 / 5 / 1988."

[0101] Data processing system 210 has already stored previous transaction data 105 for key value ID=2324 in memory 220. The value of aggregate spend 420 for field spend 416 is already stored in memory 220. Batch transaction data 412, as previously described, specifies that the aggregate spend for key value ID=2324 is $208.92.

[0102] The real-time module 214 removes data that is not necessary for the calculation of the real-time segments, as described above. For example, fields related to subscribed transaction data 540 can be extracted from transaction data 450. The real-time module 214 generates subscribed transaction data 452, which includes ID, level, and spend fields.

[0103] The record generator module 216 combines the subscribed transaction data 452 with the subscribed batch summary 414 data to generate updated records 454 that incorporate the incremental spending data from the additional transaction data 450. The record generator module 216 combines the data from the transaction data 450 with the batch summary 414. For example, the incremental spending of $101.51 from the transaction data 402, the incremental spending of $51.10 from the additional transaction data, and the incremental spending from the additional transaction data 450 are combined with the aggregate spending of $157.41 from the batch summary 414 to generate a total aggregate spending of $310.02.

[0104] The detection module 218 determines that a relevant field for the segment has been included in or changed in the record 454. The change can be detected based on checking a list of relevant fields for real-time segmentation (including the "expense" field 416) or by comparing the record to a previous record associated with the key value. If the record 454 includes an updated value for the expense field 416, the detection module 218 generates a notification 410 specifying that a "record has been updated" for key ID=2324. The notification 410 and the updated record 454 are sent to the segmentation engine 120.

[0105] The segmentation engine applies the segmentation rules 122 to the updated record 454. The segmentation engine 120 can apply each segment's conditions to the data in the updated record 454. For example, for segment 1 422a, the segmentation engine checks whether level=Silver, whether the aggregate spend is greater than 200, and whether the incremental spend value is greater than 0 (indicating that the user spent some money in the most recent transaction 402). Based on record 454, the segmentation engine 122 determines that the level=Silver condition is met (indicated by a check mark). The segmentation engine determines that the condition "aggregate spend > 200" is met. The segmentation engine determines that the condition "incremental spend > 0" is met. Therefore, segment 1 is applied to key value ID=2324. This updated real-time segment 458 is sent to the transaction processing system 230 as described above. The transaction processing system generates an updated real-time offer 460 based on the applicability of segment 1. The offer specifies a "free flight."

[0106] The segmentation engine 120 then applies the conditions for segment 2 422b. The segmentation engine 120 determines that the first condition, "Level = Silver," is met. The segmentation engine 120 determines that the second condition, "Aggregate Spending >= 150," is met because the aggregate spending from record 454 is currently $310.02. The segmentation engine determines that the third condition, "Incremental Spending >= 100," is met because the incremental spending 418 from transaction data 450 is $101.10. Therefore, all three conditions are met for segment 2 422b. Key value ID=2324 is associated with real-time segment 2. Real-time segment 458 is sent to transaction processing system 230, as described above, to generate real-time offer 460 for "Free Car Rental Offer," as described above. In this example, both segment 1 and segment 2 of real-time segment 422 are included in real-time segment 458. The offers include offers for members of each of segment 1 and segment 2.

[0107] 5A-5C are diagrams of the system 200 of FIG. 2 at the just-in-time segmentation definition stage. In FIG. 5A, the system 500a is shown to include the data processing system 210 described above in connection with FIG. 3A. The data processing system 210 includes a real-time and batch control 502, also referred to as control 502 or RT / B control 502. The control 502 has a real-time switch 504 and a batch switch 506. When the real-time (RT) switch 504 is selected, the control allows real-time data (including real-time transaction data) to the segmentation engine to generate real-time segments, as described above. When the batch switch 506 is selected, the control allows batch data, including prior transaction data, from the data warehouse to the segmentation engine to generate batch segments. The control enables the segmentation engine 120 to generate both real-time and batch segments for an entity using the same segmentation rules 120. For example, via the user interface, a user can toggle control 502 to cause the segmentation engine to generate real-time segments with a left click or batch segments with a right click. In some implementations, as described below, the segmentation engine 120 can generate both real-time and batch segments simultaneously using the same segmentation rules 120.

[0108] 6 is a flow diagram illustrating a process 500 for just-in-time segmentation in which real-time segments are generated. The process 500 includes receiving (502) a plurality of data records, each including one or more fields, at least one of the one or more fields including a value. The process 500 includes receiving (504) a portion of a real-time data stream, the real-time data stream including at least one value associated with one or more fields of at least one record of the plurality of data records. The process 500 includes receiving (506) segmentation rules specifying one or more segments for the plurality of records, the segmentation rules being generated from a segmentation specification based on one or more parameters associated with the segmentation specification. The process 500 includes generating (508) a virtual record from the real-time data stream and the plurality of data records, the virtual record representing an updated value of at least one field of at least one record based on at least one value in the real-time data stream. The process 500 includes executing (510) a segmentation rule on the virtual record, the executing being based on the updated value of the at least one field. The process 500 includes determining (512) a segment associated with at least one record that includes the at least one field based on the executing the segmentation rule. The process 500 includes storing (514) data in memory that represents an association between the segment and the at least one record.

[0109] FIG. 7 is a flow diagram illustrating a process 600 for just-in-time segmentation in which real-time segments are generated. The process 600 includes receiving (602) a rule specification including one or more parameter values ​​specifying one or more segment criteria for associating one or more data records with at least one segment. The process 600 includes generating (604) one or more segmentation rules based on the one or more segment criteria, each including a logical rule for applying to multiple data records to associate fields of the one or more data records with at least one segment. The process 600 includes converting (606) the specification into an executable program. The process 600 includes storing (608) the executable program in memory. The process 600 includes executing (610) the executable program in response to receiving a portion of the real-time data stream to apply the logical rule to fields of at least one record included in the real-time data stream. The process 600 includes associating (612) a field of at least one record included in the real-time data stream with a given segment specified by the rule specification based on execution of the executable program.

[0110] 8A-8D illustrate a user interface for defining a real-time segment, including conditions for the segment, which can be used to configure the segmentation rules 122 described above.

[0111] 8A illustrates a user interface 700a in which segments 704a-704c are shown. An entity can define segments 704a-704c (collectively real-time segments 704) by using a selection tree. The selection tree includes nested logic in which subsegments, such as subsegments 706a-706c, are nested within larger segments, such as real-time segment 704b. Segment 704 includes a selection tree with three categories, including bronze, chocolate, and gold. Segment 704 includes derived segment 704a, selection tree segment 704b, and intake list 704c. Each of 704a, 704b, and 704c is a different type of segment definition.

[0112] Row 702 includes a header row for each column 708a gas-enriched liquid that describes segment 704. The first column 708a shows the trade name for each segment or subsegment. Column 708b shows the technical name for each segment or subsegment. Column 708c includes a description for each segment or subsegment. Column 708d includes an indicator of whether the segment is defined. Column 708e indicates when the segment was last evaluated. Column 708f indicates the status of the segment. Column 708g provides options for editing, removing, or adding subsegments to the segment.

[0113] FIG. 8B illustrates a user interface 700b in which a segment tree 704b is defined by an entity. The entity can define segment 704b by specifying logical conditions for segment 704b, or in this case, for each subsegment 706a-706c. The subsegments include subsegment gold 706c, subsegment silver 706d, subsegment bronze 706a, and subsegment blue 706e. Each of subsegments 706a-706e includes conditional logic 710a-710d. Subsegment gold 706c also includes second-level subsegment 712, which is a subsegment of subsegment 706c. Second-level subsegment 712 includes conditional logic 714 that defines inclusion in the second-level subsegment.

[0114] 8C shows a user interface 700c in which a new segment 720 is defined by an entity. Segment 720 includes three levels: Animals, Tigers, and Tony. These are nested such that Tony is a second-level subsegment of Animals. The second-level subsegment 720 can be defined by its properties 716, including its technical name, trade name, and its tree path (if it is a subsegment). A description can also be provided by the user. Segments 720 can be dynamically evaluated and saved as needed according to logical flags 718.

[0115] The segment 722 Tony defined in interface 700c is shown in interface 700d of Figure 7D. Interface 700d is the same as interface 700a, except that the new segment tree animal 722 is shown with subsegment Tony 720.

[0116] Generally, an "entity" includes a portion of a computer program (e.g., a predefined portion of a computer program for inclusion in another computer program) or one or more dataflow graph components (e.g., encapsulated together in a predefined module). Throughout this specification, an "entity" may also be referred to as a "module" for convenience, without limitation.

[0117] A dataflow graph component includes a data processing component and / or a data set. A dataflow graph can be represented by a directed graph including nodes or vertices, which represent dataflow graph components and are connected by directed links or dataflow connections, representing the flow of work elements (i.e., data) between the dataflow graph components. A data processing component includes code for processing data from at least one data input, e.g., a data source, and providing data to at least one data output, e.g., a data sink of system 200. Thus, a dataflow graph can implement a graph-based computation performed on data flowing from one or more input datasets, through graph components, to one or more output datasets.

[0118] The system 200 also includes a data processing system 210 for executing one or more computer programs (e.g., data flow graphs) generated by converting the specifications into computer programs using the techniques described herein. A transformation generator can convert the rule specifications into computer programs that implement the segmentation logic. In this example, selections made by entities via a user interface described herein form a specification that specifies which fields and datasets are used in compound aggregations. Based on the specifications, the transformations described herein are generated.

[0119] Data processing system 210 may be hosted on one or more general-purpose computers under the control of a suitable operating system, such as the UNIX operating system. For example, data processing system 210 may include a multi-node parallel computing environment, including a configuration of computer systems using multiple central processing units (CPUs), either local (e.g., a microprocessor system such as an SMP computer), locally distributed (e.g., multiple processors coupled as a cluster or MPP), or remotely distributed (e.g., multiple processors linked via a LAN or WAN network), or any combination thereof.

[0120] In some examples, the entities include data flow components corresponding to nodes joined by data flows corresponding to links. In this example, the computer program is a data flow graph including entities corresponding to nodes joined by data flows corresponding to links. In this example, the memory includes volatile or non-volatile memory. Furthermore, in some examples, the entities include one or more other entities.

[0121] The graph construction approach described above can be implemented using software for execution on a computer. For example, the software forms procedures in one or more computer programs executing on one or more data processing systems 210, e.g., computer-programmed or computer-programmable systems (which may be of various architectures, such as distributed, client / server, or grid), each of which includes at least one processor, at least one data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device or port, and at least one output device or port. The software may form one or more modules of a larger program that, for example, provides other services related to the design and construction of data flow graphs. The nodes and elements of the graph can be implemented as data structures stored on a computer-readable medium or other organized data conforming to a data model stored in a data repository.

[0122] The software may be provided on a storage medium, such as a CD-ROM, readable by a general-purpose or special-purpose programmable computer, or may be distributed (encoded in a propagated signal) via a communication medium of a network to a computer on which the software is executed. All functions may be performed on a dedicated computer or using dedicated hardware, such as a coprocessor. The software may also be implemented in a distributed manner, with different parts of the data flow specified by the software being performed by different computers. Each such computer program is preferably stored on or downloaded to a storage medium or device (e.g., solid-state memory or media, or magnetic or optical media) readable by a general-purpose or special-purpose programmable computer, and when the storage medium or device is read by the data processing system 210, it configures and operates the computer to perform the procedures described herein. The data processing system 210 may also be implemented as a computer-readable storage medium configured with a computer program, which causes the data processing system 210 to operate in a specific, predefined manner to perform the functions described herein.

[0123] 9, an exemplary operating environment for practicing embodiments of the present invention is shown, generally designated as computing device 800. The essential elements of computing device 800, or computer or data processing system or client or server, are one or more programmable processors 822 for performing actions in accordance with instructions and one or more memory devices 824 for storing instructions and data. Generally, the computer also includes or is operatively coupled to (via bus 821, fabric, network, etc.) I / O components 826, such as a display device, a network / communication subsystem, etc. (not shown), and one or more mass storage devices 828 for storing data, instructions, etc., and a network communication subsystem 830 powered by a power source (not shown). Residing on memory device 824 is an operating system 832 and an application 834 for application programming.

[0124] Suitable devices for storing computer program instructions and data include, by way of example, all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0125] To provide for interaction with a user, embodiments of the subject matter described herein are implemented on a computer having a display device (monitor) for displaying information to the user, a keyboard that allows the user to provide input to the computer, and a pointing device (e.g., a mouse or trackball). Additionally, the computer can interact with the user by sending documents to and receiving documents from devices used by the user (e.g., by sending web pages to a web browser on the user's user device in response to a request received from the web browser).

[0126] Embodiments of the subject matter described herein can be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a user computer having a graphical user interface or web browser through which a user can interact with an implementation of the subject matter described herein), or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0127] A computing system may include users and servers. Users and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server may arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., HTML pages) to a user device (e.g., for the purpose of displaying data to and receiving user input from a user interacting with the user device). Data generated at the user device (e.g., results of user interaction) may be received from the user device at the server.

[0128] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features unique to particular inventive embodiments.

[0129] Similarly, although operations are depicted in the figures in a particular order, this should not be understood as requiring that such operations be performed in the order shown or in any sequential order, or that all illustrated operations be performed, to achieve desirable results. Furthermore, the separation of various system components in the above embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems may generally be integrated together in a single software product or packaged in multiple software products.

[0130] Several embodiments have been described. Nevertheless, it will be understood that various modifications can be made without departing from the scope of the techniques described herein. For example, some of the steps described above may be order-independent and, therefore, can be performed in an order different from that described. Furthermore, any of the foregoing techniques described with respect to dataflow graphs can also be implemented and performed with respect to programs. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. 1. A method implemented by a data processing system for performing real-time segmentation based on receipt of transaction data, the method comprising: One or more segmentation rules, the segmentation rules comprising: a segment representing one or more predetermined attributes that describe the entity; accessing from a hardware storage device one or more segmentation rules specifying one or more conditions for associating entities with said segments; receiving, by the data processing system, one or more data items associated with an entity; detecting transaction data indicative of an occurrence of a transaction in one or more of the received data items; In response to said detecting, accessing a memory storing data records for a plurality of entities; retrieving from the memory a data record for the entity; updating the retrieved data record according to the transaction data; executing the one or more segmentation rules on the updated data records; determining that the updated data record satisfies one or more conditions of at least one of the one or more segmentation rules; associating the entity for the updated data record with one or more segments specified by the at least one of the one or more segmentation rules based on a result of the determining, wherein the associating is in near real time with respect to the occurrence of the transaction; outputting, by the data processing system, a notification of the one or more segments associated with the entity; The method, wherein the notification results in the performance of one or more actions for the entity in near real time relative to the occurrence of the transaction, the one or more actions being in accordance with one or more predetermined attributes of the one or more segments.

2. 10. The method of claim 1, further comprising: generating a push notification instructing a transaction processing system to update the data record associated with the updated data record when at least one segment associated with the updated data record has changed from a previous segment.

3. receiving a request for a real-time segment associated with the updated data record as a service; 3. The method of claim 1, further comprising: in response to receiving the request, transmitting the one or more segments associated with the updated data record in real time to a system that sent the request.

4. The method of any one of claims 1 to 3, further comprising providing a user interface for defining the one or more rules prior to determining the one or more segments.

5. The method of any one of claims 1 to 4, wherein the notification includes an offer that is exclusive to one of the one or more segments.

6. determining that the updated data record is associated with at least two segments; The method of any one of claims 1 to 5, further comprising: generating, by the remote computing system, a notification for performing at least two actions based on said determining.

7. updating the data record with data specifying the occurrence of the transaction according to the received one or more data items; accessing from volatile memory a batch data record associated with the updated data record, the batch data record including historical batch data; generating a virtual record including at least a portion of the received one or more data items specifying the occurrence of the transaction and at least a portion of the historical batch data of the batch data record associated with the updated data record; and determining the one or more segments associated with the updated data record based on the virtual record.

8. The method of claim 7 , further comprising retrieving the historical batch data from a data warehouse and loading the retrieved historical batch data into the volatile memory.

9. 9. The method of claim 1, wherein determining one or more segments associated with the updated data record is performed in response to determining that the updated record is likely to change a segment for a given key in the updated data record.

10. 1. A data processing system for performing real time segmentation by updating a wide record based on receipt of real time data, wherein items of real time data represent transactions, detecting that the updated wide record satisfies criteria for performing real time segmentation, and performing real time segmentation on the updated wide record, wherein real time relates to when a transaction represented in the updated wide record occurs, the data processing system comprising: One or more segmentation rules, the segmentation rules comprising: a segment representing one or more predetermined attributes that describe the entity; accessing from a hardware storage device one or more segmentation rules specifying one or more conditions for associating entities with said segments; receiving, by the data processing system, one or more data items associated with an entity; detecting transaction data indicative of an occurrence of a transaction in one or more of the received data items; In response to said detecting, accessing a memory storing data records for a plurality of entities; retrieving from the memory a data record for the entity; updating the retrieved data record according to the transaction data; executing the one or more segmentation rules on the updated data records; determining that the updated data record satisfies one or more conditions of at least one of the one or more segmentation rules; associating the entity for the updated data record with one or more segments specified by the at least one of the one or more segmentation rules based on a result of the determining, wherein the associating is in near real time with respect to the occurrence of the transaction; outputting, by the data processing system, a notification of the one or more segments associated with the entity; 10. A data processing system, comprising: a data processing system configured to process a transaction for a user of the data processing system; a data processing system configured to process a transaction for a user of the data processing system;

11. 11. The data processing system of claim 10, further comprising: generating a push notification instructing a transaction processing system to update the data record associated with the updated data record when at least one segment associated with the updated data record has changed from a previous segment.

12. receiving a request for a real-time segment associated with the updated data record as a service; 12. The data processing system of claim 10, further comprising: in response to receiving the request, transmitting the one or more segments associated with the updated data record in real time to a system that sent the request.

13. The data processing system of any one of claims 10 to 12, further comprising providing a user interface for defining the one or more rules prior to determining the one or more segments.

14. The data processing system of any one of claims 10 to 13, wherein the notification includes an offer that is exclusive to one of the one or more segments.

15. determining that the updated data record is associated with at least two segments; 15. The data processing system of claim 10, further comprising: generating, by the remote computing system, a notification for performing at least two actions based on said determining.

16. updating the data record with data specifying the occurrence of the transaction according to the received one or more data items; retrieving batch data records associated with the updated data records from a data warehouse; generating a virtual record including the received one or more data items specifying the occurrence of the transaction and at least a portion of the batch data record associated with the updated data record; and determining the one or more segments associated with the updated data record based on the virtual record.

17. performing real-time segmentation by updating a wide record based on receipt of real-time data, wherein an item of real-time data represents a transaction; detecting that the updated wide record satisfies criteria for performing real-time segmentation; and one or more computer-readable hardware storage devices for performing real-time segmentation on the updated wide record, wherein real time relates to when a transaction represented in the updated wide record occurs, the one or more computer-readable hardware storage devices comprising: One or more segmentation rules, the segmentation rules comprising: a segment representing one or more predetermined attributes that describe the entity; accessing from a hardware storage device one or more segmentation rules specifying one or more conditions for associating entities with said segments; receiving, by the data processing system, one or more data items associated with an entity; detecting transaction data indicative of an occurrence of a transaction in one or more of the received data items; In response to said detecting, accessing a memory storing data records for a plurality of entities; retrieving from the memory a data record for the entity; updating the retrieved data record according to the transaction data; executing the one or more segmentation rules on the updated data records; determining that the updated data record satisfies one or more conditions of at least one of the one or more segmentation rules; associating the entity for the updated data record with one or more segments specified by the at least one of the one or more segmentation rules based on a result of the determining, wherein the associating is in near real time with respect to the occurrence of the transaction; outputting, by the data processing system, a notification of the one or more segments associated with the entity; one or more computer-readable hardware storage devices, wherein the notification results in the performance of one or more actions for the entity in near real time relative to the occurrence of the transaction, the one or more actions according to one or more predetermined attributes of the one or more segments.

18. 20. The one or more computer-readable hardware storage devices of claim 17, further comprising: generating a push notification instructing a transaction processing system to update the data record associated with the updated data record when at least one segment associated with the updated data record has changed from a previous segment.

19. receiving a request for a real-time segment associated with the updated data record as a service; 19. The one or more computer-readable hardware storage devices of claim 17 or 18, further comprising: in response to receiving the request, transmitting the one or more segments associated with the updated data record in real time to a system that sent the request.

20. 20. The one or more computer-readable hardware storage devices of any one of claims 17 to 19, further comprising providing a user interface for defining the one or more rules prior to determining the one or more segments.

21. The one or more computer-readable hardware storage devices of any one of claims 17 to 20, wherein the notification includes an offer that is exclusive to one of the one or more segments.

22. determining that the updated data record is associated with at least two segments; 22. The one or more computer-readable hardware storage devices of claim 17, further comprising: generating, by a remote computing system, a notification for performing at least two actions based on said determining.

23. updating the data record with data specifying the occurrence of the transaction according to the received one or more data items; retrieving batch data records associated with the updated data records from a data warehouse; generating a virtual record including the received one or more data items specifying the occurrence of the transaction and at least a portion of the batch data record associated with the updated data record; and determining the one or more segments associated with the updated data record based on the virtual record.

24. 1. A method implemented by a data processing system for performing real-time segmentation by updating data records based on receipt of real-time data, where items of real-time data represent transactions, detecting that the updated data records meet criteria for performing real-time segmentation, and performing real-time segmentation on the updated data records, wherein real time relates to when a transaction represented in the updated data records occurs, the method comprising: receiving, by a data processing system, one or more data items associated with a given key; detecting the occurrence of a transaction in one or more of the received data items; accessing a volatile memory storing data records for a plurality of keys in response to said detecting; Retrieving a data record for the given key from the volatile memory; updating the data record with data according to the received one or more data items specifying the occurrence of the transaction and associated with the given key; executing one or more rules on the updated data record, the rules being associated with a segment, the rules specifying one or more conditions, the rules further specifying that if the one or more conditions are satisfied by data in the updated data record, then associate the data record for the given key with the segment; determining, based on the performing, one or more segments associated with the updated data record, wherein the determination of the one or more segments is in near real time with respect to the occurrence of the transaction; outputting, by the data processing system, instructions specifying one or more actions associated with the determined one or more segments.

Images