Predictive segmentation of energy customers

Abstract

To provide a computer algorithm for analyzing energy consumers.SOLUTION: A computer system receives customer records listing customer attributes and an adoption status of a customer, such as whether the customer has enrolled in a particular energy efficiency program. An initial set of patterns is identified among the customer records according to a decision tree or the like. The initial set is pruned to obtain a set of patterns each meeting minimum support and effectiveness and maximum overlap requirements. The patterns are assigned to segments according to an optimization algorithm that seeks to maximize the minimum effectiveness of each segment, in which the effectiveness indicates the number of customers matching the pattern of each segment that have positive adoption status. The optimization algorithm may be a bisection search algorithm that evaluates a linear-fractional integer program (LFIP-F) to iteratively approach an optimal distribution of patterns.SELECTED DRAWING: None

Description

[Technical field]

[0001] FIELD OF THEINVENTION This application claims the benefit of U.S. Provisional Application No. 62 / 269,793, filed December 18, 2015, and entitled “Predictive Segmentation of Energy Consumers,” the entire contents of which are incorporated herein by reference.

[0002] The present invention relates to a computer algorithm for analysing energy consumers. [Background technology]

[0003] In recent years, energy utility companies have been increasingly focused on improving their relationships with a customer base that has not traditionally engaged with their electricity supplier. Historically, both energy companies and their consumers have understood the utility's role as "keeping the lights on." However, current technology trends and shifts in customer thinking (spurred in particular by the rise of consumer-facing Internet companies that are adept at understanding and predicting their customer preferences) have led to an increased interest in utilities in engaging with their customers.

[0004] These trends are compounded by an increase in data availability (both granular consumption data collected through sensing infrastructure such as smart meters and other "metadata" about the consumers themselves) and computational methods for processing this data (e.g., Li and Yang (2015); Liu and Nielsen (2015)). Energy utilities are therefore increasingly relying on analytical techniques that may provide ways to increase their customer satisfaction and engagement and participation in environmentally friendly programs within their customer base. Customer segmentation is a cornerstone of the marketing toolbox of organizations large and small, as a technique for understanding customers and identifying ways to act upon that understanding. It is primarily used in marketing (a comprehensive review can be found in the Association (2014)), online advertising (e.g., Yan et al. (2009)), or e-retailing (e.g., Bhatnagar and Ghose (2004)), to name a few applications.

[0005] As utilities seek to develop more personal and modern relationships with their customers, they are embracing segmentation as a means to rethink their communications about efficiency measures and other programs to increase participation and engagement. Most market segmentation techniques employed are in fact focused on the application of a fixed set of rules. For example, consumers who live in large homes and have children are assigned to a "high consumer" category, while those who subscribe to environmental magazines are attributed to an "environmental advocate" group. Typically, these rules are derived from counterfactual or anecdotal experiences, behavioral studies, or small-scale psychology experiments, and are in fact considered "public facts." Being the result of extracted expertise, such segmentation strategies are certainly useful and should inform theory and practice.

[0006] The approach described herein provides an improved approach to segmenting energy consumers. Summary of the Invention [Means for solving the problem]

[0007] This application introduces a predictive segmentation technique for identifying subgroups within a large population that are both homogeneous with respect to specific patterns in customer attributes and predictive for a desired outcome. The motivating setting is to create a very straightforward and intuitive, and in some sense even optimal, segmentation and targeting process for the customers of an energy utility company. In this setting, the energy utility wants to design a small number of message types to be sent to well-selected customers who are most likely to respond to different types of communications. The proposed method uses consumption, demographic, and program enrollment data and extracts basic predictive patterns using standard machine learning techniques. The method then defines feasible potential allocations of patterns to a small number of segments described by expert guidelines and hypotheses about consumer characteristics available from previous behavioral studies. The algorithm then identifies an optimal allocation of patterns to both segments that is both feasible and maximizes predictive power. The method is implemented on a large dataset from a major US energy utility, obtaining segments of customers whose enrollment probability is twice that of the average population and described by a small number of simple intuitive rules. The present invention provides, for example, the following: (Item 1) 1. A method comprising: receiving, by a computer system, a plurality of customer records, each of which includes attributes and an adoption status of the customer; identifying, by the computer system, a plurality of patterns, each pattern defining a grouping of customer records of the plurality of customer records according to an attribute of the plurality of customer records; allocating, by the computer system, the plurality of patterns among a plurality of segments such that each pattern of the plurality of patterns is assigned to one segment of the plurality of segments, the allocating of the plurality of patterns among the plurality of segments being performed according to an algorithm that iteratively increases a minimum effectiveness for the plurality of segments, the effectiveness of each segment being a function of the adoption status of customer records of the plurality of customer records that match a pattern of the plurality of patterns distributed among the respective segments; A method comprising: (Item 2) 2. The method of claim 1, wherein the algorithm includes solving an equation LFIP-F. (Item 3) 3. The method according to claim 2, wherein the algorithm includes a step of solving a linear integer programming feasibility problem according to a binary search algorithm 1. (Item 4) Item 10. The method of item 1, wherein each pattern of the plurality of patterns comprises, for each attribute of the attributes of the each pattern, a threshold value for the each attribute. (Item 5) 2. The method of claim 1, wherein identifying the plurality of patterns comprises processing the plurality of customer records according to a decision tree. (Item 6) The step of identifying a plurality of patterns comprises: processing the plurality of customer records according to a decision tree to obtain an initial pattern set; culling the initial pattern set to obtain the plurality of patterns; The method according to item 1, comprising: (Item 7) The step of selecting the initial pattern set to obtain the plurality of patterns includes: removing patterns having a support below a minimum support, the support of each pattern in the set of patterns indicating the number of customer records that match each pattern; The method according to item 6, comprising: (Item 8) The step of selecting the initial pattern set to obtain the plurality of patterns includes: removing patterns having a validity below a minimum validity, the validity of each pattern in the set of patterns indicating a percentage of customer records matching each pattern having a positive acceptance status; 8. The method according to item 7, comprising: (Item 9) 9. The method of claim 8, wherein the minimum validity is a multiple greater than one of the percentage of all customer records of the plurality of customer records that have a positive adoption status. (Item 10) 10. The method of claim 9, wherein the multiple is at least two. (Item 11) 7. The method of claim 6, wherein sifting the initial pattern set to obtain the plurality of patterns includes removing each pattern of the initial pattern set that has a set of customer records of the plurality of customer records that match each pattern, such that more than a maximum percentage of the set of customer records also match another pattern of the initial pattern set. (Item 12) Item 12. The method according to item 11, wherein the maximum percentage is 60 to 75 percent. (Item 13) 2. The method of claim 1, wherein the effectiveness of each segment is further a function of a coverage matrix of each pattern assigned to the each segment, the coverage matrix of each pattern having a value of 1 / n for each customer record of the plurality of customer records that matches the each pattern, where n is the number of patterns of the plurality of patterns that are matched by the each customer record. (Item 14) 1. A system comprising one or more processing devices and one or more memory devices operatively coupled to the one or more processing devices, the one or more memory devices storing executable code, the executable code causing the one or more processors to: receiving a plurality of customer records, each of which includes attributes and an adoption status of said customer; identifying a plurality of patterns, each pattern defining a grouping of customer records of the plurality of customer records according to an attribute of the plurality of customer records; allocating the plurality of patterns among a plurality of segments such that each pattern of the plurality of patterns is assigned to a segment of the plurality of segments, the plurality of patterns being distributed among the plurality of segments according to an algorithm that seeks to maximize a minimum effectiveness of the plurality of segments, the effectiveness of each segment being a function of the adoption status of customer records of the plurality of customer records that match a pattern of the plurality of patterns distributed among the respective segments; A system that is effective to perform the above. (Item 15) Item 15. The system of item 14, wherein the algorithm includes using a binary search algorithm 1 to solve a linear integer programming feasibility problem according to equation LFIP-F. (Item 16) 15. The system of claim 14, wherein each pattern of the plurality of patterns comprises, for each attribute of the attributes of the each pattern, a threshold value for the each attribute. (Item 17) The executable code further causes the one or more processors to: processing the plurality of customer records according to a decision tree to obtain an initial pattern set; culling the initial pattern set to obtain the plurality of patterns; 15. The system of claim 14, wherein the plurality of patterns are identified by: (Item 18) The executable code further causes the one or more processors to: removing patterns having support below a minimum support, the support of each pattern in the set of patterns indicating the number of customer records that match each pattern; removing patterns having a significance below a minimum significance, the significance of each pattern in the set of patterns indicating a proportion of customer records matching each pattern having a positive adoption status; removing each pattern of the initial pattern set that has a set of customer records of the plurality of customer records that match each pattern, such that more than a maximum percentage of the set of customer records also match another pattern of the initial pattern set; 20. The system of claim 17, wherein the system is effective to sift through the initial pattern set to obtain the plurality of patterns by: (Item 19) 20. The system of claim 18, wherein the multiple is at least two. (Item 20) 15. The system of claim 14, wherein the executable code is further effective to cause the one or more processors to calculate an effectiveness of each segment as a function of a coverage matrix of each pattern assigned to the each segment, the coverage matrix of each pattern having a value of 1 / n for each customer record of the plurality of customer records that matches the each pattern, where n is the number of patterns of the plurality of patterns that are matched by the each customer record. [Brief description of the drawings]

[0008] So that the advantages of the present invention may be readily understood, a more particular description of the invention briefly described above will be had by reference to specific embodiments which are illustrated in the accompanying drawings, in which: The present invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, with the understanding that these drawings depict only exemplary embodiments of the invention and therefore should not be considered as limiting its scope.

[0009] [Figure 1] FIG. 1 is a schematic block diagram of components for implementing predictive customer segmentation in accordance with one embodiment of the present invention.

[0010] [Diagram 2] FIG. 2 is a schematic block diagram of a computing device.

[0011] [Diagram 3] FIG. 3 is a process flow diagram of a method for performing predictive segmentation according to one embodiment of the present invention.

[0012] [Figure 4] FIG. 4 is a diagram illustrating a decision tree extracted from customer data.

[0013] [Diagram 5] FIG. 5 is a plot of the distribution of pattern effectiveness versus the number of rules.

[0014] [Figure 6] FIG. 6 is a graph showing predictor variables for describing energy customer registration.

[0015] [Figure 7] FIG. 7 is a feasibility matrix for patterns assigned to customer segments.

[0016] [Figure 8] FIG. 8 is a plot showing the distribution of overlap between patterns.

[0017] [Figure 9] FIG. 9 is an example of a pattern associated with two segments.

[0018] [Figure 10] FIG. 10 is a plot of segment effectiveness versus lower and upper bounds on the number of iterations.

[0019] [Figure 11] FIG. 11 illustrates the pattern / segment allocation matrix.

[0020] [Figure 12] FIG. 12 illustrates segment overlap.

[0021] [Figure 13] FIG. 13 is another diagram illustrating segment overlap.

[0022] [Figure 14] Figure 14 shows a list of segments and the corresponding patterns that follow the segmentation algorithm.

[0023] [Figure 15] Figure 15 shows [ka] 1 is a plot showing a sensitivity analysis of the segmentation algorithm with respect to .

[0024] [Figure 16] Figure 16 shows [ka] 1 is a plot of segment effectiveness with respect to

[0025] [Figure 17] FIG. 17 is a scatter plot of segment effectiveness as a function of segmentation complexity. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] 1. Operating environment and overview Referring to FIG. 1, the methods disclosed herein may be implemented by the illustrated operating environment 100. A server system 102 or other type of computer system may host or access a database 104. The server system 102 may also be replaced with a desktop or laptop computer, or even a mobile device with sufficient computing power. The database 104 may include customer records 106 for multiple customers. The methods disclosed herein are described with respect to energy customers. Thus, each customer record 106 may include data for a single household or customer account, which may therefore include data for multiple individuals living together.

[0027] The customer record 106 may include such information as one or more customer identifiers 108a in the form of a name, account number, or other unique identifier. The customer record 106 may also include the customer's address 108b and demographic information 108c about one or more individuals associated with the customer record 106, such as age, income, gender, occupation, education level, and any other information that may characterize the customer.

[0028] When the methods disclosed herein are applied to energy customers, the customer record 106 may further include usage data 108d, such as the number of kilowatts per hour used per year, month, or day. The usage data 108d may include daily, monthly, or seasonal usage patterns derived from analysis of the electricity consumption data. In other applications, the usage data 108d may include usage of different services or purchases of specific items or necessities.

[0029] The customer record 106 may also include any other data 108e available about the customer that may be useful in identifying patterns that describe the type of customer and customer behavior.

[0030] The methods disclosed herein are used to analyze data and determine patterns in customer data (demographics, usage, and other) that predict customers taking certain actions. For energy customers, this may include enrolling in an energy efficiency program or taking other actions to reduce consumption or otherwise reduce the customer's environmental impact. Thus, the customer record 106 may further include one or more adoption status indicators 108f that indicate whether the customer has chosen to participate in a particular program. For example, the status 108f may be 1 if the customer has chosen to participate and 0 otherwise. In other embodiments, the adoption status may be one of a range of values ​​that indicate the degree of compliance with program guidelines or the amount of money spent for a particular purpose.

[0031] The database 104 may further store segments 110 having multiple patterns 112 assigned to them. Each segment 110 has a validity 114, which is a measure of the number of customer records that have a positive adoption status 108f and match one of the patterns 112 assigned to the segment 110.

[0032] The segments 110 may be defined by an analysis module 116, which implements the methods disclosed herein below. In particular, the analysis module 116 may include a pattern generation module 118a, which identifies a set of attributes that co-occur in customer records. As described below, the patterns may be described in terms of thresholds for the values ​​of various attributes within each customer record. Also, as described below, the patterns may be generated using decision trees or other pattern identification algorithms.

[0033] The analysis module 116 may further include a pattern pruning module 118b. As described below, the pattern pruning module 118b may prune patterns that do not meet minimum support, validity, or non-overlapping criteria. The analysis module may include a segmentation module 118c. The segmentation module 118c assigns patterns 112 that survive pruning 118b to segments 110 such that a set of segments 110 is obtained in which the minimum validity 114 of the segments is increased through an algorithm that distributes patterns among the segments.

[0034] 2 is a block diagram illustrating an example computing device 200. The computing device 200 may be used to perform various procedures, such as those discussed herein. The server system 102 may have some or all of the attributes of the computing device 200.

[0035] Computing device 200 includes one or more processors 202, one or more memory devices 204, one or more interfaces 206, one or more mass storage devices 208, one or more input / output (I / O) devices 210, and a display device 230, all coupled to a bus 212. Processor 202 includes one or more processors or controllers that execute instructions stored in memory device 204 and / or mass storage device 208. Processor 202 may also include various types of computer-readable media, such as cache memory.

[0036] The memory device 204 includes a variety of computer-readable media, such as volatile memory (e.g., random access memory (RAM) 214) and / or non-volatile memory (e.g., read only memory (ROM) 216). The memory device 204 may also include a writeable ROM, such as flash memory.

[0037] The mass storage device 208 includes a variety of computer readable media, such as magnetic tape, magnetic disks, optical disks, solid state memory (e.g., flash memory), etc. As shown in Figure 2, a particular mass storage device is a hard disk drive 224. Various drives may also be included within the mass storage device 208 to enable reading from and / or writing to the various computer readable media. The mass storage device 208 includes removable media 226 and / or non-removable media.

[0038] The I / O devices 210 include a variety of devices that allow data and / or other information to be entered into or read from the computing device 200. Exemplary I / O devices 210 include cursor control devices, a keyboard, a keypad, a microphone, a monitor or other display device, a speaker, a network interface card, a modem, a lens, a CCD or other image capture device, and the like.

[0039] Display device 230 includes any type of device capable of displaying information to one or more users of computing device 200. Examples of display device 230 include monitors, display terminals, video projection devices, and the like.

[0040] The interface 206 includes various interfaces that allow the computing device 200 to interact with other systems, devices, or computing environments. An exemplary interface 206 includes any number of different network interfaces 220, such as interfaces to a local area network (LAN), a wide area network (WAN), a wireless network, and the Internet. Other interfaces include a user interface 218 and a peripheral device interface 222. The interface 206 may also include one or more peripheral interfaces, such as interfaces for pointing devices (mouse, trackpad, etc.), keyboards, and the like.

[0041] The bus 212 allows the processor 202, the memory device 204, the interface 206, the mass storage device 208, the I / O devices 210, and the display device 230 to communicate with each other and with other devices or components coupled to the bus 212. The bus 212 represents one or more of several types of bus structures, such as a system bus, a PCI bus, an IEEE 1394 bus, a USB bus, etc.

[0042] For purposes of illustration, programs and other executable program components are shown herein as discrete blocks, however, it should be understood that such programs and components may in some cases reside in different storage components of computing device 200 and are executed by processor 202. Alternatively, the systems and procedures described herein can be implemented in hardware or in a combination of hardware, software, and / or firmware. For example, one or more application specific integrated circuits (ASICs) can be programmed to implement one or more of the systems and procedures described herein.

[0043] 3, the server system 102 may execute the method 300 shown. The method 300 may include receiving 302 customer data. This may include receiving data over a period of time as data is collected about the customer. The received data may include some or all of the data described above as being included within the customer record 106.

[0044] Method 300 may further include a step 304 of determining a customer adoption status. The adoption status 108f may be included in the customer record when received or may be received as part of a subsequent program that extends the offer to the customer and receives a response. In either case, data is provided manually or automatically to server system 102 indicating the adoption status for each customer. In some embodiments, method 300 may be performed only for customers that received the offer.

[0045] Method 300 may further include generating 306 an initial pattern set. For example, generating 306 an initial pattern set may include traversing a decision tree as known in the art, where each node of the decision tree is an attribute value or range of attribute values ​​corresponding to attributes 108b-108e of customer record 106. An exemplary decision tree is shown in Figure 5, and the generation of the initial pattern set is described in more detail below in Sections 3.2, "Extracting Predictive Patterns from Data," and 5.2, "Predictive Patterns Extracted from Data."

[0046] The method 300 may further include a step 308 of pruning the initial pattern set. This may include removing patterns that do not have enough support (i.e., an insufficient number of customer records 106 that match the pattern), patterns that do not have enough validity (i.e., an insufficient number of customer records 106 that match the pattern and have a positive acceptance status), and patterns that have a higher than a threshold percentage of matching customer records that also match another pattern. A more detailed description of the pruning process is described below in Section 3.2, "Extracting Predictive Patterns from Data," and Section 5.2, "Extracting Predictive Patterns from Data."

[0047] Method 300 may further include a step 310 of assigning patterns to segments according to an algorithm that iteratively approaches a maximum value for a segment's minimum validity, where validity is a measure of how many of the customer records that match the patterns assigned to each segment have a positive acceptance status. This may include running an optimization algorithm, as described in Section 3.1 "Increasing Minimum Validity" below.

[0048] The segments may then be further processed 312. In particular, segments may be used for targeted marketing, i.e., advertisements may be organized and transmitted only to customers who match the patterns of one segment to increase their effectiveness. Segments may also be used to visualize customer behavior or for any other business objective.

[0049] The algorithms implemented by method 300 are described in more detail below in Sections 2-3. Section 4 contains an overview of previous approaches, and Section 5 illustrates experimental results using real customer data.

[0050] It should be noted that the following description is of an optimization algorithm that seeks to maximize the minimum effectiveness of the segments. Thus, references to "maximum," "optimum," "optimized," "minimum," and "minimum" are to be understood as not referring to absolute or actual maximum, optimum, or minimum values, but rather maximum, optimum, or minimum values ​​determined subject to the limitations of the disclosed algorithm and following performance of a finite number of iterations of the disclosed algorithm.

[0051] Specifically, "maximizing" a value, "maximizing" a value, and the "maximum" of a value shall be understood to refer to an increase in the value compared to a previous iteration of the disclosed algorithm or compared to not performing the disclosed algorithm, except where a closed set of values ​​is considered and the maximum value within the closed set can be determined with certainty.

[0052] "Minimizing" a value, "minimizing" a value, and "minimum" a value shall be understood to refer to a decrease in the value compared to a previous iteration of the disclosed algorithm or without the disclosed algorithm, except where a closed set of values ​​is considered and the minimum value within the closed set can be determined with certainty.

[0053] "Optimize" shall be understood to mean finding a value closer to the absolute optimum than would be the case without performing the disclosed algorithm, and not to actually find the absolute optimum. Similarly, an "optimum" value shall be understood to be near the optimum, with "near" referring to limitations in expressing the accuracy of expressing numbers and performing mathematical operations thereon, limitations of what the disclosed algorithm can theoretically achieve, and limitations of the number of iterations that can be performed in practice.

[0054] 2. Predictive segmentation A transparent and useful segmentation strategy should achieve the following: 1. Capture existing and useful expertise and best practices so that practitioners can easily relate to and adopt it. 2. Be understandable and intuitive to non-technical program managers at energy utilities and be useful for generating marketing communications. 3. Provide some optimality guarantee in terms of validity, i.e. be highly discriminatory with respect to its purpose of identifying a subgroup of members who are more likely to take action than consumers drawn randomly from the entire population.

[0055] First, a great deal of expertise and practical experience exists among energy utilities, allowing them to make assumptions about certain high-level types of customers they wish to identify from their base. For example, most experienced program managers will agree that "environmental advocate" consumers will respond to other types of communications (emphasizing environmental impact) than more "cost-conscious" consumers (who may respond to arguments about saving money).

[0056] Second, the method may start from existing expert knowledge associating certain variables with each given segment (e.g., "green advocates" may be defined by their income, household type, and level of education) and identify simple logical rules with those variables that lead to the most effective segmentation strategies. Such intuitive segments should make it possible to generate appropriate messaging strategies. For example, consumers in the "green advocates" group would receive messages that emphasize the environmental aspects of energy conservation, while those consumers in the "high spender" category would be informed about ways in which they can reduce their high bills.

[0057] The challenge is then to develop an algorithmic segmentation methodology that incorporates the requirements of points 1 and 2, while ensuring the useful nature of the resulting segments (as presented in the third point above) and ensuring that the best possible segmentation that satisfies the imposed structure is achieved. The desired outcome is to maximize the impact of marketing communications regarding energy efficiency program enrollment, i.e., targeting those customers who are more likely to enroll. Both tailoring communications and managing campaigns are costly, so the focus is on crafting messages to a small number of segments and including those consumers who are likely to take action.

[0058] 2.1. Problem setting A population X of N consumers is served by an operator (an energy utility company). For each consumer, the utility obtains M number of characteristics, consisting of both consumption and customer characteristics (such as socio-demographic and physical building attributes). [ka] Therefore, the feature data across all consumers is expressed as the matrix [ka] The utility also observes for each consumer i whether or not he or she has enrolled in any program within the past year, which is stored in the binary variable [ka] is encoded as

[0059] A utility wishes to use data (X,y) to identify K segments within a population that are "homogeneous" with respect to attribute X, with the goal of informing, simplifying, and increasing the effectiveness of targeted communications for demand-side efficiency program enrollment. Based on prior marketing research, the utility may have certain hypotheses regarding the "types" of customers it serves. This prior knowledge is assumed to be of the following form: "Environmental Advocates" have relatively high incomes or at least college degrees. "Home Improvement Seekers" are homeowners or breadwinners.

[0060] The data (X,y) is then stored in the set [ka] The pattern, [ka] These hypotheses can be used to make concrete by extracting patterns that are descriptive, in that they refer to the characteristics of consumers who exhibit these patterns, and predictive, in that consumers who fall into a pattern are more likely to enroll than consumers selected randomly from the entire population. A pattern may thus be defined to be a logical expression of the form: [ka] During the ceremony, [ka] is a base rule (logical statement). Thus, a pattern is defined as a set of conjunctions. Interchangeably, a pattern can be referred to as a set of consumers that conform to the logical definition of the pattern. Let a base rule be of the form: [ka]

[0061] So the base rule is a variable x that points in a direction (either "≧" or "≦"). j (the jth variable in x) and a threshold t learned from the data j Rule P is defined by j A rule (·) is said to be consistent with a hypothesis if both the variables and the directions that define that rule match the hypothesis. Similarly, we define a pattern P to be δ-consistent with a hypothesis if it contains at least δ ≥ 1 rules that are consistent with the hypothesis.

[0062] It is useful to define a coverage matrix C that summarizes the extent to which item i is covered by pattern m. [ka]

[0063] The effectiveness of a pattern P may be calculated as the (empirical) enrollment probability of the consumers covered by that pattern. [ka]

[0064] Using the above setting, we define the K segments as a set of patterns such that every pattern in each segment is δ-consistent with the hypothesis that defines that segment. [ka] Define it as: [ka] Let define a (well-known) consistency matrix that describes the admissible relationships between segments and patterns. [ka]

[0065] Finally, the segmentation results in the following set of individual segments: [ka]

[0066] 2.2. Effective Segmentation We now consider a segmentation strategy that is effective if consumer segments are distinguishable in terms of their rate of enrollment. That is, a good strategy (for K segments) is one that has a probability q that is significantly different (either smaller than or greater than) the overall rate q observed in the entire population. k , k = 1,...,K. For example, if the segmentation consists of groups A and B with K = 2, then if all consumers in A have enrolled but none in B have enrolled (hence, q A =1 and q B =0), is perfectly valid. A perfectly ineffective segmentation is one in which consumers in A enroll at the same rate as consumers in B (so q A =q B ). Of course, one could group consumers into two segments by putting everyone who enrolls in the efficiency program at any one time into one of the segments. The challenge, however, is to identify patterns in consumer characteristic X that lead to a clear, intuitive definition of a segment that also predicts enrollment.

[0067] The effectiveness of each segment may be calculated in a similar manner to the effectiveness of a pattern as the (empirical) enrollment probability of consumers in that segment. [ka]

[0068] The segments are therefore [ka] is a good value measure number for a registration if [ka] is the rate of enrollment in the overall population. The problem to be solved is to determine whether the resulting segments have at least the following desirable efficacy properties, e.g. [ka] and at most [ka] The aim is to allocate the pattern of the number of segments to each segment. [ka] Ensure proper balance of effectiveness across segments [ka] where θ is a given weight vector.

[0069] In this regard, the decision variables are as follows: [ka] Define. [ka]

[0070] Therefore, segment k is defined as follows: [ka]

[0071] The problem is then solved such that one of the objectives (8-9) is maximized, subject to the following feasibility constraints (F 0 ) is satisfied, z mk This means finding the value of [ka]

[0072] There are many possible patterns for a given segment, namely: [ka] Moreover, patterns may overlap (i.e., the sets of consumers they define are not disjoint). [ka] ). Segments in S may then overlap if they happen to contain overlapping patterns in the customers they describe. This imposes additional complications on the proper formulation of the optimization problem that addresses (8-9) and satisfies the constraints (F0).

[0073] If the patterns do not overlap, the segment effectiveness can be written as: [ka]

[0074] However, pattern overlap can be substantial, so the above formula [ka] One simplification we employ to address this issue is to relax the definition of the coverage matrix C, noting that a consumer covered by n distinct patterns can be considered to have a fractional coverage of 1 / n on each pattern. This is the modified coverage matrix [ka] , that is, it is converted to [ka]

[0075] Thus, the modified coverage matrix assigns a weight to each consumer i that represents the fractional coverage of a single pattern (giving equal importance to each pattern). For convenience, we still refer to this modified matrix as C.

[0076] 3. Calculating predictive segments The design of the algorithm for computing the predictive segmentation will be determined by the specific form that the objective function takes (all constraints are simple linear forms). Here, we focus on the situation where the objective is to allocate admissible patterns to segments so as to maximize the minimum effectiveness across K segments (see equation (8)). This is a natural requirement for program managers who want to have a guarantee on the minimum effectiveness of their targeted communication strategies.

[0077] 3.1.Increasing the minimum validity The above expression in equation (12) encodes the decision variables for each segment into K vectors z k To express the objective and constraints in a more convenient affine form using a single decision variable vector, the following notation can be adopted: [ka] During the ceremony, [ka] The effectiveness can then be expressed as: [ka]

[0078] F 0 The feasibility conditions for are: [ka]

[0079] In the max-min objective case (8), the optimization tries to increase the lower bound of the efficiency across the segments as much as possible. This is because k This results in a relatively homogeneous distribution of s. This situation may be desirable, for example, when action will be taken on each of the segments. In this case, the optimization problem may be expressed as: [ka]

[0080] The problem (LFIP) is a generalized (max-min) linear-fractional integer program with linear constraints. This class of problems has been extensively studied in the literature (see, e.g., Horst and Pardalos (1995), Feng et al. (2011), and Schaible and Shi (2004) for reviews). Following Boyd and Vandenberghe (2004), we propose the following equivalent formulation of (LFIP) as the linear integer programming feasibility problem (LFIP-F): [ka] where, for k=1,…,K, A is the row [ka] where D is a matrix with rows [ka] For a given value of λ, the above feasibility problem (LFIP-F) can be solved using a standard mixed integer programming package. The initial customer characteristic data can be very large (here, [ka] 100 customers), the number of patterns is expected to be much smaller (M~1,000), as is the number of segments (here K=5). Then the standard package can perform well out of the box. Then the maximum with the corresponding optimal z [ka] can be found efficiently using an iterative binary search algorithm 1 (see Table 1 below), which solves a feasibility problem (LFIP-F) at each step. * is guaranteed to fall within the large interval [l 0 ,u 0 Starting from λ (here, [0,1]), the algorithm successively narrows the interval [l,u] at every step, * Ensure that ∈[u,b]. This is outlined in Lemma 1, which was developed in Patel et al. (2013). [ka]

[0081] Lemma 1. The output of Algorithm 1 is within tolerance ε and log 2 (ε 0 / ε) λ, the maximum value of λ within an iteration * The optimal z corresponding to * It is.

[0082] To prove Lemma 1, we use Algorithm 1 to find a unique value λ, which is the maximum feasible value that λ can take. * In this regard, we define the following feasible set: [ka]

[0083] Using this notation, we have: [ka] Segment z * The optimal pattern allocation to λ * By definition, the optimal λ * is the feasible set Λ and the infeasible set [ka] Therefore, the following must be true for the tolerance parameter ε>0 (small): [ka]

[0084] Algorithm 1 is optimal λ * We need to show that the above conditions are satisfied to prove that we will find

[0085] To prove the first condition, take λ∈Λ and the following needs to be proven: [ka]

[0086] The fact that λ∈Λ implies the following: [ka]

[0087] Then, λ+ε and the same z λ With respect to [ka]

[0088] The second term above is for ε>0 and D and z λ Both ,are positive since they have only non-zero inputs.

[0089] To prove the second condition, the value [ka] is fixed, and then [ka] Or it is desired to show that ε>0. [ka] The fact is that [ka] and so for a given value of λ [ka] z λ teeth, [ka] Let be the decision variable vector corresponding to λ that yields the maximum value of λ and satisfies all other conditions that define the feasible set Λ. Then [ka]

[0090] From the infeasibility of λ, we further see that [ka] and then for λ+ε, [ka] The decision vector z λ+ε However, as mentioned above, z λ+ε Including, [ka] Then, z λ+ε With respect to [ka]

[0091] Next, [ka] Therefore, [ka] Therefore, Algorithm 1 always obtains the optimal allocation vector z * The maximally feasible λ corresponding to * Furthermore, with each step, the algorithm halves the search interval [l,u], so that [ka] To reach the completion condition, it takes at most the following steps: [ka] As is readily apparent, the optimization algorithm approximates the optimal solution such that it lies within a search window [l,u]<ε, which becomes smaller with each iteration, as explained above.

[0092] 3.2. Extracting predictive patterns from data Given a set of observations encoded as a feature matrix X and a binary response (training) vector y, we have a very valid (q>>q 0 ) patterns P. In this regard, the following approach is adopted: 1. Use ensemble methods such as Random Forest or AdaBoost Hastie et al. (2009) with classification trees as base learners to generate many decision trees of variable depth (here we have generated trees with up to 5 levels). This step begins with a list of initial patterns P obtained by traversing the decision trees down to each leaf. 0 Depending on the level of the tree used as the base classifier in the reinforcement ensemble method, these rules can be of variable complexity, from a single statement (a tree of depth 1 or a decision tree) to the conjunction of several base rules. 2. Pattern List P 0 , and eliminate rules that do not correspond to some set criteria of "quality". For this purpose, a pattern is called a rule if it meets both of the following criteria: [ka] shall be considered "valid". ·Minimum support: |P|>η, i.e. the number of customers matching the patterns must exceed η, such that η+1 is the minimum population matching each pattern. Here, η=500 is used. ·Minimum validity: q(P)>ζq 0 Here, ζ=2 is used. 3. Further remove patterns that overlap with other patterns by more than v% (here, v=70%, however, values ​​of v between 60 and 75% may also be used) and have a lower validity q. For example, matching customer C 1 and effectiveness q 1 A pattern P having 1 and matching customer C 2 and q 1 Effectiveness less than q 2 A pattern P having2 Regarding this, customer C exceeds v% 2 But, C 1 If it is contained within, then pattern P 2 would be selected out as it has a lower efficacy.

[0093] This procedure is a selection of patterns [ka] This results in a set of

[0094] 4. Literature review Customer targeting for energy programs has recently attracted attention from seemingly disparate literatures in engineering and computer science, operations management, and marketing. This study contributes to the larger discussion in those fields by providing a simple and transparent methodology that results in straightforward segment building on top of existing expertise within operations and marketing departments at energy utilities. Engineering research on demand side management has recently been motivated by the availability of detailed customer data, including detailed consumption readings and socio-demographic information. Typically, there has been a focus on several main areas: i) using whole-household data (either from smart meters or custom fixtures) to describe the consumption patterns of a population of users with the goal of informing programs such as time-of-need pricing or smart thermostat controls (Kwac et al. (2013) , Albert and Rajagopal (2015)); ii) collecting both whole-household and individual appliance experiment data and reconstructing distinct end uses from an aggregate signal (Carrie Armel et al. (2013) , Kolter and Jaakkola (2012) , and iii) studying the average impact of different external factors (in particular, weather) on energy use (Houde et al. (2012) , Kavousian et al. (2013) , Kavousian et al. (2015) ).

[0095] More recent literature on energy analytics concerns the characterization of consumption patterns (load profiling) in an extension of traditional demand management practices in utilities, using aggregate demand profiles to inform programs. A strategy of segmenting consumers according to the cost their consumption behavior brings to the grid is proposed in Albert and Rajagopal (2014) as a way to target groups of consumers that contribute most to volatility in demand. A popular topic of research is the heterogeneity in typical daily load profiles (typically involving clustering of daily user consumption load shapes using off-the-shelf unsupervised algorithms such as K-means), which can then be used for interventions such as differential pricing or encouraging energy reduction. This approach has been taken, for example, in Flath et al. (2012), Rasanen and Kolehmainen (2009), Figueiredo et al. (2005), Smith et al. (2012), Tsekouras et al. (2007), Espinoza et al. (2005). Other variations, which segment load profiles based on first learning generative models of consumption and then cluster the resulting models, have been discussed, for example, in Albert and Rajagopal (2013), Alzate et al. (2009). However, this line of research is largely descriptive in nature, typically no clear use cases for the identified load patterns are provided, and few programs at utilities currently exist that could incorporate such information.

[0096] Meanwhile, the operations management and marketing literature has seen increasing interest in energy applications over the past few years, which may be influenced by the fact that in many utility companies, the departments involved in allocating, enrolling, and targeting consumers with efficiency programs have traditionally been either operations or marketing.

[0097] 5. Experimental Setup 5.1. Customer Profile Data The data used in this application were obtained from a large energy company in the northeastern United States and consisted of approximately 100 socio-demographic and building characteristics and monthly energy consumption readings over a two-year period for N=957,150 consumers. After standard data cleaning procedures, 43 variables of interest were selected that had at least 80% valid entries across the entire population. Of these, 19 variables were categorical and 24 were numerical. A final data set of P=304 variables was obtained by converting the categorical variables to binary dummy variables. q 0 A total of 48,310 consumers, corresponding to a proportion of =4.9%, had enrolled in a voluntary energy efficiency program within the two years prior to data collection. [Table 1] [Table 2]

[0098] Table 1 describes some of the categorical variables of attention. The majority of consumers (about 80%) own their homes, while only about 16% rent. Education levels, as a whole, reflect society in general, with a quarter of consumers having bachelor's and master's degrees, respectively, and half of consumers having a high school diploma or less. The "environmental consciousness" variable summarizes the results of a third-party analysis that considers factors such as magazine subscriptions, community involvement, political studies, and membership in different organizations, resulting in an estimated level of attention to environmental issues.

[0099] Table 2 summarizes some of the more quantitative variables of interest. The average year of birth is 1957, suggesting a Baby Boom demographic. The average family in the sample lives in a large home (6 rooms) with a residence period of over 12 years.

[0100] 5.2. Predictive patterns extracted from data Prediction rules were extracted from the data as described in Section 3 above. After selection, a list of prediction patterns (whose validity was at least [ka] and had support of at least η = 500), respectively, for up to five base rules M 0 The data set contained =2,965 patterns (1,852 patterns with 5 base rules, 963 patterns with 4 base rules, 143 patterns with 3 base rules, and 7 patterns with 2 base rules). Figure 4 illustrates an example decision tree of height 3 extracted from the data. The highlighted patterns are paths in the decision tree starting from the root with an efficacy (proportion of positive samples) of 8%. Figure 5 illustrates the M 0 Figure 1 illustrates the distribution of pattern effectiveness q(R) for patterns of different complexity (2-5 base rules) for =2,965 patterns. As expected, the distribution exhibits an exponential behavior, with more patterns of lower effectiveness and fewer patterns of higher effectiveness.

[0101] The top 20 most important variables for predicting enrollment are listed in Figure 6. These include, among others, homeownership (loan-to-value ratio, available equity), size of the home and the family living there, and family income. This suggests that enrollment depends on the perception of financial responsibility and ability to make improvements to the home. This analysis considered only enrollment in voluntary energy efficiency programs. An analysis of specific programs targeted at more specific types of consumers would likely yield more refined distinctions in important variables (such as discounts for insulation as opposed to efficient appliances).

[0102] 5.3. Associating patterns with segments The segments were defined using the results of previous behavioral studies and extensive interactions with the energy utility that provided the data. The utility wanted to identify consumers who fell into a small number of segments that were already defined based on its own internal expertise and research and independent third-party behavioral and marketing studies such as Frankel et al. (2013). As explained in Section 2 above, the purpose of the segments was two-fold: i) to generate a small number of marketing communications, such as standardized emails with appropriate information and planning for each segment, and ii) to identify consumers corresponding to each segment who are likely to enroll in an energy efficiency program.

[0103] Based on this prior art, the utility believed that consumers fell into K=5 segments: "Environmentalists," "High Spenders," "Home Improvement Wanters," "Cost Priorities," and "Cultural Motivators." The segment semantics that encode this hypothesis are summarized in Table 3. Given these segment definitions, P 0 Potential patterns P from are associated with distinct segments by ensuring that each pattern P is δ-congruent with a hypothesis about the meaning of the individual segments (see Section 2). That is, for a given segment S, [ka] is the set of at least δ base rules that match both the variable j and the direction (either above or below the threshold learned from the data). [ka] The resulting set of patterns P contained M=2 patterns. [ka] Reduced patterns with 614,830 (64% of the original samples) [ka] Although not all consumers were covered by the reduced set, 89% of the registered consumers were included within the reduced set. [Table 3]

[0104] The number of patterns obtained for each segment and its coverage rate (the number of consumers in the pattern) are also listed in Table 3. Although the number of patterns is smaller than the initial approximately 3,000, choosing a number small enough to approach maximum effectiveness is still a non-trivial task. The association matrix B that encodes the pattern / segment assignment feasibility is illustrated in FIG. 7. Some patterns may belong to multiple segments, as illustrated in FIG. 8, where the distribution of the number of patterns that cover a user is plotted. Most users are covered by a small number of patterns. However, at the same time, there are also a small number of users who fall into more than 50 patterns.

[0105] Two examples of rules extracted from the data and assigned to the segments "high spender" and "cost-conscious" are displayed in Figure 9. The pattern assigned to "high spender" contains at least δ=1 base rules with the condition that consumption is above a given threshold.

[0106] 6.Results Algorithm 1 is [ka] was used to obtain a near-optimal feasible allocation of patterns to the segment Z when 0 width) [ka] We narrow the search space until convergence in 14 iterations when ε = 10. Therefore, the allocation of patterns to segment Z is -14 The binary search process is illustrated in FIG. [ka] We show that we can find a maximal lower bound λ on the segment efficiency by iteratively solving a feasibility problem (LFIP-F) using [ka] The resulting optimal allocation matrix Z for F is displayed in FIG. 11, where the horizontal axis orders the patterns by arbitrary ID numbers in the same format as that used in FIG. 10 to represent the allowable allocation matrix B. The algorithm has the best efficiency property, (F 0 ) we selected a small number of patterns that satisfied the constraints in

[0107] The optimal solution contains 10 patterns spread across 5 segments. Table 4 summarizes the effectiveness and size of the resulting segments. All final effectiveness counts are 2×q 0 , and consumers assigned to one segment ("Cultural Incentives") enrolled at nearly three times the rate within the overall population. [Table 4]

[0108] FIG. 12 shows an example of overlap between segments. This overlap is induced because the patterns that make up the segments themselves can and do overlap within the customers they cover. However, segment overlap is a natural concept in reality because consumers may have certain characteristics that may qualify them as belonging to one segment (e.g., "cost prioritizers"), while other characteristics are shared with consumers in a different segment (e.g., "home improvement seekers"). Segmentation techniques transparently take this situation into account. A more comprehensive view of segment overlap is presented in FIG. 13 as a network plot, where each segment is represented as a node with a size proportional to the number of customers in that segment. The weights of the links between segments represent the pair-wise overlap of the segments. The constraints are [ka] from [ka] As the number of patterns is changed, the structure of the segmentation changes since more patterns are used to build some of the segments.

[0109] Note that "segments" are a construct defined by program administrators to aid in creating and managing communications that differentiate to some degree among consumers while keeping operational costs and complexity low. They expose some degree of heterogeneity, but at the same time do not allow for perfectly tailoring interventions to individual needs. Imposing that all consumers belong to only one segment would impose an unrealistic assumption, which the present approach avoids.

[0110] Figure 14 shows [ka] The hypotheses in Table 3 about the meaning of each segment are based on the threshold t j The patterns are then augmented with specific information such as the number of users (precisely defining what "high" and "low" mean) and additional base rules. For example, one type of "home improver" who enrolls in energy efficiency programs at a high rate is South Asian, earning more than $75,000 per year and owning more than $306,870 in equity in their home. Similarly, one type of "environmental advocate" is a family who earns more than $75,000 per year, earns at least 2.5 times the median income level for their state, has children, and does not live in an apartment building. The patterns within each segment may then be used to design marketing communications specific to that segment to include elements to which consumers in that segment are perceived to respond. Furthermore, the specificity of the patterns (in terms of thresholds learned from the data) allows for targeting those consumers who are most likely to enroll.

[0111] From the above discussion, the resulting segmentation structure depends on the nature of the constraints, in particular: [ka] It becomes extremely clear that the strong dependence of the θ on the value of θ is very clear. To study this dependence, Algorithm 1 is [ka] This was carried out with respect to [ka] Purpose [ka] The optimal values ​​of and the number of patterns selected for segmentation are illustrated in FIG. 15. The best results are [ka] (So ​​the algorithm does not extract more patterns than necessary into segments.) [ka] However, the medium to large [ka] and low [ka] These maps therefore provide guidelines on how to trade off model complexity and segmentation effectiveness.

[0112] [ka] For a given value of [ka] Using the above, the target λ * and individual segment effectiveness values ​​q k , k=1,...,K variations have been observed. This can then serve as a tuning parameter for the resulting segmentation complexity, which can be designed to adapt to the desired effectiveness value of the individual segments of interest. [ka] For example, if emphasis is placed on "cultural incentives", [ka] For all values ​​of k, a segmentation with q k is clearly λ * Note that this exceeds

[0113] Finally, the dependence of individual segment effectiveness on segmentation complexity (total number of patterns selected across a segment) is illustrated in Figure 17. This highlights the best possible effectiveness value that can be achieved for a fixed given value of segmentation complexity. For example, if an efficiency program manager desires to select a total number of patterns between 20 and 25, the optimal effectiveness of the "Cultural Incentives" segment can be expected to consistently exceed that of the "Cost Prioritizers" segment. [ka] For that range, "home improvement advocates," "environmental advocates," and "cultural incentives" all have effectiveness values ​​of about 11%.

[0114] 7. Conclusion This application introduced a method for programmatically building a comprehensible predictive segmentation of energy consumers. The predictive segmentation problem is formulated based on first extracting predictive patterns (conjunctions) from the data, then optimally allocating the patterns to segments. The segments were defined using prior behavior and marketing studies at energy utilities. The optimal allocation is formulated to solve a generalized (max-min) linear fractional integer program with linear constraints. An efficient binary search algorithm was used to solve the program. The method was used to identify optimal predictive segments within a population of approximately one million electricity consumers of a large U.S. energy utility. An optimal subset of consumers, whose characteristics matched the utility's general hypotheses about the types of consumers it serves and who enrolled at least twice the enrollment rate of approximately 5% in the overall population, was identified. These segments represent consumers for whom the utility can generate appropriate messages and who are more effective and economical to target.

[0115] In the above disclosure, reference has been made to the accompanying drawings, which form a part of this specification, and in which specific implementations in which the disclosure may be practiced are shown by way of illustration. It should be understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. References in the specification to "one embodiment," "an embodiment," "an exemplary embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but not all embodiments may necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Moreover, when a particular feature, structure, or characteristic is described in relation to an embodiment, it is contemplated that it is within the knowledge of one of ordinary skill in the art to affect such feature, structure, or characteristic in relation to other embodiments, whether or not explicitly described.

[0116] Implementations of the systems, devices, and methods disclosed herein may comprise or utilize special-purpose or general-purpose computers, including computer hardware, such as one or more processors and system memory, as discussed herein. Implementations within the scope of the present disclosure may also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media can be any available media that can be accessed by a general-purpose or special-purpose computer system. Computer-readable media that store computer-executable instructions are computer storage media (devices). Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not by way of limitation, implementations of the present disclosure may comprise at least two separately distinct types of computer-readable media, namely computer storage media (devices) and transmission media.

[0117] Computer storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid state drives ("SSD") (e.g., based on RAM), flash memory, phase change memory ("PCM"), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special purpose computer.

[0118] Implementations of the devices, systems, and methods disclosed herein may communicate via a computer network. A "network" is defined as one or more data links capable of transporting electronic data between computer systems and / or modules and / or other electronic devices. When information is transferred or provided to a computer via a network or another communications connection (either wired, wireless, or a combination of wired or wireless), the computer properly regards the connection as a transmission medium. A transmission medium may include a network and / or data links that may be used to carry desired program code means in the form of computer-executable instructions or data structures and that may be accessed by a general-purpose or special-purpose computer. Combinations of the above should also be included within the scope of computer-readable media.

[0119] Computer-executable instructions comprise, for example, instructions and data that, when executed on a processor, cause a general-purpose computer, special-purpose computer, or special-purpose processing device to perform a certain function or group of functions. Computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the present subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts. Rather, the described features and acts are disclosed as example forms for implementing the claims.

[0120] Those skilled in the art will appreciate that the present disclosure may be practiced in a network computing environment with many types of computer system configurations, including in-dashboard vehicle computers, personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, cell phones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The present disclosure may also be practiced in a distributed system environment, where both local and remote computer systems linked through a network (either by wired data links, wireless data links, or a combination of wired and wireless data links) perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[0121] Furthermore, where appropriate, the functions described herein may be performed in one or more of hardware, software, firmware, digital components, or analog components. For example, one or more application specific integrated circuits (ASICs) may be programmed to implement one or more of the systems and procedures described herein. Certain terms are used throughout the description and claims to refer to particular system components. As one skilled in the art will appreciate, components may be referred to by different names. This document does not intend to distinguish between components that differ in name but not function.

[0122] It should be noted that the sensor embodiments discussed above may comprise computer hardware, software, firmware, or any combination thereof to perform at least some of its functions. For example, the sensor may include computer code configured to run in one or more processors, or may include hardware logic / circuitry controlled by computer code. These exemplary devices are provided herein for illustrative purposes and are not intended to be limiting. The embodiments of the present disclosure may be implemented in additional types of devices that will be known to those skilled in the art.

[0123] At least some embodiments of the present disclosure are directed to computer program products comprising such logic (e.g., in the form of software) stored on any computer-usable medium, such software, when executed within one or more data processing devices, causing the devices to operate as described herein.

[0124] Although various embodiments of the present disclosure have been described above, it should be understood that they are presented by way of example only and not limitation. It will be apparent to those skilled in the art that various changes in form and detail can be made therein without departing from the spirit and scope of the present disclosure. Thus, the scope and scope of the present disclosure should not be limited by any of the above exemplary embodiments, but should be defined only according to the following claims and their equivalents. The above description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. Furthermore, it should be noted that any or all of the above alternative implementations may be used in any combination as desired to form additional hybrid implementations of the present disclosure.

[0125] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are considered in all respects only as illustrative and not limiting. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are intended to be embraced within their scope.

[0126] The following references are incorporated herein by reference in their entireties: [Table 5-1] [Table 5-2] [Table 5-3] [Table 5-4]

Claims

[Claim 1] A method or system.

Images