Method and system for segmentation and uses thereof

Abstract

A method and system for determining segmentation features for a population of users is described. A processor captures detectable interactions by users to issued communication variants. Next, gain for the user attributes of profiles for which a detectable interaction with a communication variant exists is determined. One or more tree structure graphs are generated and user profiles allocated to branches according to the values of respective user attributes. Best communication variant for each group of user profiles allocated to each branch is determined and branches of the tree structure for user profiles with the same best communication variant aggregated and expected gain and confidence value determined. Segmentation features for said population of users are returned; and further steps performed by the method and system to personalise issuance of one or more communications to a specified user of a population of users.

Description

[0001] METHOD AND SYSTEM FOR SEGMENTATION AND USES THEREOF FIELD OF THE DISCLOSURE The present disclosure relates to a system and method for segmentation of customers and for using the same in customising communications provided over a network. BACKGROUND OF THE DISCLOSURE A typical approach to determining user engagement or satisfaction with a communication about a service or product is the comparative analysis of user responses. Such comparative analysis of user responses typically has the goal of incrementally improving choices by capturing and analysing successive simple user interactions. A very common and simple example of comparative analysis of user responses in internet or application development is determining the choice of the colour of a button on a website page in order to increase the click rate of that button. An initial test population of visitors will be randomly presented with a web page with the button having one colour (among of choice of two). Based on average click rate for each colour, the best performing colour will eventually be chosen for all future visitors. When comparative analysis of user responses is coupled with clustering or segmentation of customers into distinct groups of individuals with similar characteristics based on similar behaviours this provides a powerful tool for merchants to target specific customer groups with optimised mass communications. However, as the complexity of potential variations in a communication to a customer increases (e.g. including a range of colours on a website button rather than two, or including numerical continuous values such as prices in the communication), it becomes extremely difficult for companies to determine optimal configuration options of the communication for presentation to specific clusters or segments of customers. Furthermore, customer behaviours may change in non-periodic and non-linear ways of a duration of customer interaction with a system based on range of external stimuli. This difficulty is further compounded by difficulty in determining how to actually determine features by which customers may be segmented or clustered for further analysis. Often such segmentation is performed based on an ad-hoc determination by marketing professionals relying on observations and professed experience of the customer identified and fixed prior to performing a particular communication campaign or cycle.

[0002] 1 424954 456477\41405803_1.docx Although there are some approaches to comparative analysis of user responses which employ machine learning techniques, such approaches have been restricted due to the complexity of variants possible and variation in customer interaction options. In addition, inappropriate segmentation may mean that customers are treated as having the same preference when in fact appropriate segmentation would have provided a much more nuanced insight. Over segmentation of customers according to features that provide little predictive insight by iteratively segmenting customers into smaller segments causes increased computational load on processors conducting the analysis; and wastes analytical time. Furthermore, in some implementations the segmentation is process unable to be explained; and hence referred to as a “black box”. Accordingly, segmentation difficulties and limitations associated with more complex comparative analysis of user responses has therefore compromised deployment of such approaches. Accordingly, it is an object of the present disclosure to develop a system and method which addresses at least some of the above difficulties or provides a useful choice. SUMMARY OF THE DISCLOSURE Features and advantages of the disclosure will be set forth in the description which follows, and in part will be obvious from the description, or can be learned by practice of the herein disclosed principles. The features and advantages of the disclosure can be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. In accordance with a first aspect of the present disclosure, there is provided a computer implemented method for determining segmentation features for a population of users, each of said users having a user profile comprising values associated with a plurality of user attributes; the method comprising: determining, by a processor, one or more detectable interactions by users from said population to a plurality of communication variants issued thereto; determining, by a processor, gain for the user attributes of profiles for which a detectable interaction with a communication variant exists; said gain being determined relative to all detectable interactions by users of said population; generating one or more tree structure graphs based upon the determined gain for the attributes; said tree structure graphs having branches comprising nodes representative of said attributes and edges representative of the possible values thereof, and allocating user profiles to branches of said tree structure according to the values of the respective user attributes,

[0003] 2 424954 456477\41405803_1.docx determining by a processor the best communication variant for each group of user profiles allocated to each branch, aggregating branches of the tree structure for user profiles with the same best communication variant; determining for all remaining branches of the tree structure graphs the expected gain and confidence value; returning attributes of said user profiles from said tree structure which exceed a predetermined threshold for gain and confidence value as segmentation features for said population of users. Advantageously, the method comprises deleting branches of the tree structure for attributes of user profiles which lack detectable interactions and / or which are below predetermined thresholds for gain and / or confidence value. The best communication variant for a user profile may be selected from the plurality of communication variants by a method selected from the group comprising determining the upper bound confidence interval for each profile; determining the highest mean amongst communications variants provided to each user profile or by determining which communication variant for a profile exceeds a predetermined performance threshold and falls below a predetermined standard deviation. Preferably, the generating of the one or more tree structure graphs comprises generating a first of the one or more tree structure graphs which has a root node having the highest gain for detectable interactions relative to all detectable interactions for each of the other user attributes. Generating of the one or more tree structure graphs may comprise generating up to n-1 additional tree structure graphs, where n is the number of attributes in said user profiles. The root node of each additional tree structure graph may be selected as the next highest gain for detectable interactions of the communications relative to all detectable interactions amongst attributes of said user profiles. The expected gain and confidence value for the remaining structures of the tree structure may be determined using a technique selected from a group of techniques comprising performing a Monte Carlo simulation; a quasi Monte Carlo simulation, closed form equation solving or prediction via a machine learning model.

[0004] 3 424954 456477\41405803_1.docx In a further aspect, there is provided a computer implemented method for personalising the issuance of one or more communications over a network to a specified user of a population of users; wherein said specified user has a plurality of user attributes and said one or more issued communications are selected from a plurality of communication variants, wherein the method comprises: identifying by a processor segmentation features from amongst said user attributes of users in said population of users using the method described herein; generating by a processor a communication for issuance to said specified user by selecting the communication variant most likely to be associated with a detectable action being performed by said user; wherein said selection is performed by identifying said variant either by: (a) using Thompson sampling on said segmentation features for said user if there exists at least one detectable action recorded by the network for said variant for other users sharing the same values of the identified segmentation features; or else by (b) randomly choosing a communication variant from said plurality of communication variants where there does not exist at least one detectable action for said variant for other users having the same values of said segmentation features and confirming the presence or absence of a subsequent detectable action. Preferably, the communication variant is transmitted across the network as an email message or text message or other electronic communication. The detectable action by said first user may comprise detecting an interaction by said user with the one or more issued communications. The detectable action may comprise opening the one or more issued communications. Each communication variant may have identical elements contained therein but differ in the time of despatch. The detectable action by said first user may comprise detecting an interaction with one or more elements of the issued communication.

[0005] 4 424954 456477\41405803_1.docx The detectable interaction by the said first user may comprise clicking on a button element contained in the communication. Advantageously, the button element has parameters selected from the group comprising typeface, image, size, colour and shape. The detectable action performed by said user may be an interaction recorded on a data store in communication with the processor independent of any detectable interactions by the user with the communication. The method may further comprise generating a further one or more communication variants for at least one successive other user from the population of users; wherein said further generated communication variant is influenced by the presence or absence of detectable action responsive to transmission of an earlier communication variant to said other preceding users having one or more similar attributes; said action determined for within a predetermined time frame. The method may further comprise repeating the steps thereof for a plurality of communication variants to a plurality of the population of users over a predetermined time period for determining the distribution of detectable actions for user profiles thereof in the population. The method may comprise generating a report on the detectable actions performed in response to said transmitted communication variants issued to at least some of said population of users. The number and attributes of at least some of the population of users may be specified and said simulated user profiles used for determining a communication variant strategy. In a further aspect, there is provided a computer system for determining segmentation features for a population of users, each of said users having a user profile comprising values associated with a plurality of user attributes; the system comprising: a database configured to store one or more detectable interactions by users from said population of users to a plurality of communication variants issued thereto; a segmentation module comprising one or more processors configured to perform the method described above. The system may further configured to personalise the issuance of one or more communications over a network to a specified user of a population of users; said system further comprising an analysis module having one or more processors configured to perform the method described above.

[0006] 5 424954 456477\41405803_1.docx There is also provided a non-transitory computer-readable storage medium, storing program instructions computer-executable by a processor of a computer to perform the computer implemented method according to claims. BRIEF DESCRIPTION OF THE FIGURES In order to describe the manner in which the above-recited and other advantages and features of the disclosure can be obtained, a more particular description of the principles briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended Figures. Understanding that these Figures depict only exemplary embodiments of the disclosure and are not therefore to be considered to be limiting of its scope, the principles herein are described and explained with additional specificity and detail. Preferred embodiments of the present disclosure will be explained in further detail below by way of examples and with reference to the accompanying Figures, in which:- Fig 1A depicts a diagram of an exemplary system for conducting the personalised issuance of communication variants over a network. Fig 1B depicts a schematic diagram of an exemplary system architecture for conducting the personalised issuance of communication variants over a network. Fig 2A depicts a flowchart of an exemplary process for analysing the issuance of communications to a population of users in a software system. Fig 2B depicts a flowchart outlining in more detail how the segmentation process of Fig 2A for the population of users is performed. Fig 2C depicts an exemplary beta distribution for a user profile with specific user attributes and values of various communication variants provided thereto. Fig 3 depicts an overview of an exemplary segmentation discovery process according to an aspect of the present disclosure. Fig 4A (i) depicts an exemplary process in an optional implementation of choosing a root node from amongst the possible segmentation values for customer attributes. Fig 4A (ii) depicts an abstracted exemplary set of initialisation graphs with a depth of 1 in an optional implementation with a single root node; demonstrating all possible values N segments can have.

[0007] 6 424954 456477\41405803_1.docx Fig 4B depicts an exemplary set of possible initialisation graphs for a specified example of specified segment values. Fig 5A depicts an abstracted exemplary abstracted set of initialisation graphs derived by expanding the set of graphs of Fig 4A (ii) with all permutations having the root node of Segment 2. Fig 5B depicts a further graph for the specified example of Fig 4B in an implementation where no root node is selected. Fig 5C depicts a final graph for the specified example of Fig 4B in an implementation where no root node is selected. Fig 5D depicts an exemplary graph (one level) that may be created for the remaining country root node in the example of Fig 4B. Fig 6A depicts an abstracted exemplary set of reduced tree structure graphs formed from the graph of Fig 5A formed either by removing permutations that are not discriminant (a profile allocated to that segment will go to the same winner bandit regardless of value for that segment) or by removing permutations which are redundant. Fig 6B depicts an exemplary set of reduced graphs for the example depicted in Fig 5B. Fig 7 depicts the steps in conducting a Monte Carlo simulation, an exemplary approach for computing the expected gain and confidence of each branch in the exemplary tree. Fig 8A depicts the cumulative email open rates for an exemplary experiment in a first county. Fig 8B depicts the cumulative email sending rates and preferential weighting for the experimental results depicted in Fig 8A. Fig 9 depicts the results from another exemplary experiment in another country for a different campaign. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Various embodiments of the disclosure are discussed in detail below. While specific implementations are discussed, it should be understood that this is done for illustration purposes only. A person skilled in the relevant art will recognize that other components and configurations may be used without departing from the spirit and scope of the disclosure.

[0008] 7 424954 456477\41405803_1.docx The present disclosure addresses the need in the art for providing an adaptable method of providing dynamic multivariate comparative analysis of user responses for both continuous and categorical metrics with optimised dynamic weighting of variants as the experiment progresses. Advantageously, the present disclosure also teaches a way to select non-redundant and discriminatory attributes of user profiles in a population of user profiles in a segmentation process utilised in the dynamic multivariate comparative analysis of user responses disclosed. As disclosed further herein in embodiments of the present invention, there is disclosed a contextual multi-armed bandit engine which is configured to identify user segments and find relationships between the communication variants and different customer segments. The contextual multi-armed bandit engine can be updated over the course of the experiment which may be performed for a number of successive cycles, learning from historical user or customer data from previous cycles. Accordingly, the present disclosure teaches a system which addresses the problems of traditional comparative analysis of user responses testing outlined above including the complexity of variants possible and variation in customer interaction option; and over segmentation of customers according to features that provide little predictive insight by iteratively segmenting customers into smaller segments. Advantageously, the method and system of the present disclosure reduces computational load on processors conducting the analysis; and avoids wasted analytical time. Referring to Fig 1A, there is disclosed a diagram of an exemplary system for conducting the personalised issuance of communication variants over a network. As depicted, there are a plurality of users 16a, 16b, 16c also marked as P1, P2, P3 etc.; which are three users randomly selected from a population of users who may be in communication over a network with the system 10 of the present disclosure via an electronic device 18a, 18b, 18c. It would be appreciated that many more users would be possible other than those depicted which have been included for exemplary purposes only. The users of the plurality of users have certain user attributes 19 (e.g. age, gender, country, device operating system, etc.). It would be appreciated that some users may share one or more of the same attributes and that such attributes could be anything; and would not be limited to those depicted which are provided solely for context. The user attributes may be categorical variants which are binary (such as gender) or multivariate (such as country) or continuous (such as salary).

[0009] 8 424954 456477\41405803_1.docx A system administrator 14 controls the operation of the system and specifies the settings for the cycle. A plurality of different communication variants 12a, 12b, 12c also marked as V1, V2, V3 etc. with various attributes 21 are depicted. It would be appreciated that many more variants would be possible other than those depicted which are included for exemplary purposes only. Similarly, it would be appreciated that these communication variants may be binary - such as a red button, or a blue button in the communication. Alternatively, they may be multivariate (such as red, blue or green button) or may be continuous variants e.g. sent at different times or with different prices. Advantageously, the system 10 provides an approach which uses a machine learning algorithm to build a self-updating model which provide optimised communication variants to a population of users based upon detected responses to other users with similar profiles. More particularly the system 10 applies a bandit algorithm, such as a contextual bandit algorithm to solve the problem of which communication variant to send. In this sense it solves a problem where a fixed limited set of resources (in this case communication variants) are allocated between competing or alternative choices, i.e., multiple “arms” of bandits, in a way that maximizes long term expected gain for the resources. As is known in the art, a multi-armed bandit problem in which an agent analyzes a multi- dimensional feature vector, i.e., a context vector, associated with a current iteration before choosing between arms in each iteration. Context vectors along with rewards of the arms played in the past enable the choice of arm to play in the current iteration. Over time, the aim of the contextual multi-armed variant or bandit model of the present disclosure is to collect enough information about how user attributes and communication attributes (represented as vectors) and detectable actions relate to each other, so that choices can be made by analyzing the vectors. A contextual bandit algorithm is applied to communications issued by the system to the users, and it would be appreciated the performance of competing variants is unclear before testing. By sending variants and recording responses from user / customers, the contextual bandit algorithm can update the bandit model built contextual multi-armed algorithm to optimize the allocation of variants per campaign, i.e., each cycle of sending communications. Thus,

[0010] 9 424954 456477\41405803_1.docx system 10 may determine what communication variant is to be distributed to which segment of customers based on the updated model in successive campaigns. Accordingly, system 10 can promptly adjust and automatically re-allocate the distribution of communications and reduce time and cost. Each time the data has been collected from the responses of the customers and it is sent to the system 10, a new iteration begins (i.e. of both segmentation process and traffic allocation processes discussed below). A new iteration consists of sending the latest data to the system 10 to update the whole historical data with the new data received. Having the historical performance updated, the approach of the present disclosure can determine which are going to be the next segments based on the gain and confidence. Each potential segment will be simulated; and the potential segments chosen by the potential gain associated therewith. In one approach to determining the gain and confidence, a Monte Carlo simulation or similar may be run on all the different segments, and only the segments with a gain and confidence level over a certain threshold will be created for the next iteration. Duration can be set at start in the case of an experiment fixed duration (e.g. a 6-week marketing campaign). Alternatively, duration may not be initially set, to let the experiment run until best variants for given segments are found by the system with a high certainty, and then stop at any time. Optionally. an experiment may be stopped at any time with a pre-defined strategy upon cessation covering for example once stopped; which customer should receive which email. It would be appreciated that the specifics of handling of the experiment once stopped can be adjusted for each campaign. For example, in a marketing campaign that sends communications on a daily basis, the bandit model may be updated over each email cycle, and the distribution may be re-allocated day-to- day. Thus, the bandit model may be trained and refined within a relatively short period, such as one week (e.g., seven email cycle). Thus, comparing to general comparative analysis, system 10 is much faster and more flexible in response. Referring now to Fig 1B, there is depicted an exemplary logical technical architecture of the system 10 depicted in Fig 1A. As depicted one or more hardware processors 40 are communicatively coupled via a bus 39 with a storage device 34. The storage device 34 may

[0011] 10 424954 456477\41405803_1.docx include a memory, which includes computer-readable media in the form of volatile and / or non- volatile memory such as read only memory (ROM) or RAM 36. In some embodiments, storage device(s) 34 may include a main memory, which can be used for storing temporary variables or other intermediate information during execution of instructions by processor(s) 30. Such instructions, after being stored in non-transitory storage media accessible to processor(s) 30, render system 10 a special-purpose machine that is customized to perform operations specified in the instructions. The term “non-transitory media” as used herein refers to any non-transitory media storing data or instructions that cause a machine to operate in a specific fashion. Such non-transitory media can include non-volatile media and / or volatile media. The system may be coupled via a communication interface module 42 over a network which may be a local network, an internet service provider, internet, or any combination thereof. Advantageously, this module mediates communications with any 3rd-party system (e.g. email gateway) and with other components, typically using API or similar Advantageously, the processor 40 either singly or in conjunction with other processors and RAM / ROM in the same or different physical modules may provide the following logical modules – an auto-segmentation module 44, an analysis module 46 for determining and reporting on optimised communications. Also provided is a control or monitoring module 48 for detecting and updating a database which will be described further herein. In addition, advantageously, there is included a database or data store 32 which stores past communication with the 3rd-party system, and results returned by the algorithms. The auto- segmentation module 44 permits identification of the subset of possible user attributes for a population of a campaign which provide meaningful information gain. A Thompson sampling service analysis module 46 allows the updating of the bandit model built with a contextual multi- armed algorithm to optimize the allocation of communication variants per campaign, i.e., each cycle of sending communications. A control module 48 allows a system administrator or authorised personnel to specify the parameters of an experiment and visualize the progress and results. Advantageously, it may also return the next variant for an input segment in communication with the other modules as described further herein. Advantageously, due to the efficiency of the algorithms disclosed herein, the system can work on very light-weight hardware. In a typical implementation the simulations (handled by the Thompson sampling service and auto-segmentation service) for this disclosure can be run on cloud instances with 2 vCPU and 4 GB RAM.

[0012] 11 424954 456477\41405803_1.docx Advantageously, the database 32 used for the present disclosure may be a MongoDB or similar “key-value database” where the data stored is not tabular but in the form of key-value dictionaries. Advantageously, the backend module and frontend module can be also deployed on cloud instances with 2 vCPU and 4 GB RAM. Fig 2A depicts the first stages in an overview flow diagram of an exemplary process for providing dynamic multivariate comparative analysis of user responses of a population of users in a software system. As depicted at step 100, the experiment administrator decides that an experiment is to be conducted. If a new experiment is to be conducted; the communication variants to test in step 102 are specified; as well as the detectable interaction to be monitored in step 104. Advantageously, these experimental parameters may be stored in a database in step 106. If the experiment is a repeat of an earlier experiment (e.g. which is rerun at a later point in time), the parameters may be retrieved from the database; and a new instance of the experiment with the same parameters commenced. It would be appreciated that the communication variants may be electronic communications such as an email message or text message or other electronic communication issued over a network. Optionally, the communication variants may have identical elements contained therein but may only differ in the time of despatch. Alternatively, the communications may have elements which differ in elements having different values of typeface, image, size, colour and shape or similar. It would also be appreciated that the communication variants may contain references by which a user is able to access a customised sequence of web pages, with the sequence and / or web page content specifically customised to provide predetermined (and different) user experiences for different groups within the user population. The experiment administrator then commences the experiment in step 108 via the control module or similar. The system determines whether segmentation for attributes which comprise the user profiles of the population has already been performed with such segmentation stored; for example in a database or data store accessible over the network. If the segmentation for the population and communication variants to be investigated has not been performed or otherwise specified (or if it is outdated); the segmentation may be determined in step 112 as described further herein with reference to Fig 3 - Fig 7.

[0013] 12 424954 456477\41405803_1.docx The obtained segmentation which includes non-redundant and discriminatory segmentation features selected from amongst user attributes from user profiles in the population of users is stored and used in the experiment in step 114. Alternatively, if segmentation of the user attributes amongst the user profiles which comprise the population has already been performed and stored; or if otherwise specified by the experiment administrator this segmentation may be retrieved and applied in the experiment in step 116. The segmentation is applied in the experiment by introducing the plurality of user profiles to be evaluated for a detectable response based on segmentation features identified to determine the distribution of each communication variant for the user profile and hence the communication variant with the most likelihood of producing a detectable action in step 118. This is performed using Thompson sampling or random allocation as described further herein with reference to Fig 2B. It would be appreciated that the user profiles may be actual detected user profiles (or generated user profiles generated from an understanding of the population) without departing from the scope of the present disclosure. Advantageously, and optionally the system may then generate the identified communication variant upon receiving user attributes for a user profile; which can then transmit this communication variant to that user in step 119a. If the experiment is still ongoing once the user receives the communication variant, the presence or absence of a detectable action associated with that user profile within a time period specified by the experiment administrator may then be utilised to modify the communication variant distribution in step 119b; in accordance with established principles of Thompson sampling as described herein with reference to Fig 2B. As is known by persons skilled in the art, Thomson sampling relies on historical performance to put more weight on certain variants. At start, there is no history, so all variants have the same history (none), so the system will choose among the variants with the same likelihood (i.e. randomly). Then, information on historical performance of variants increases, more successful variants will be chosen. When historical performance records have a significant number of records, the most successful variants will be selected even more often; which is self-perpetuating as the iterations increase- the cumulatively accruing historical performance is fundamental. For example, it would be appreciated that 2 cycles of 100,000 customers could lead to selecting one variant almost always. Conversely, 1,000 cycles of 2 customers might not be sufficient to clearly favour one variant. After the system has more information, successful variants will be

[0014] 13 424954 456477\41405803_1.docx selected more often. The more information, the lesser the uncertainty, and the more certainty present. Referring now to Fig 2B, there is detected an exemplary flowchart of how the traffic (i.e. user profiles) are applied in step 118 to determine the variant distribution and hence to determine the best communication variant likely to result in the specified detectable action. In step 120, the running mode of the experiment is determined. If the experiment is running in real time (as indicated by step 122) each time a new variant is shown for a specific segment, the algorithm is executed, and based on the latest historical performance data - that is being updated continuously, the algorithm will determine which new variant has to be shown for this particular customer. Alternatively, if the experience is asynchronous (i.e. as indicated by step 124) then for the current iteration, each segment has pre-defined probabilities / traffic allocation to each of the different variants identified as the result of the segmentation analysis. In this mode, the historical performance data is updated; but does not affect the allocation of traffic for the current iteration. Once a new iteration begins, the algorithm will be executed again, new segment calculations will be performed, and new traffic allocation to the communication variants will be determined for each of the variants, for each of the segments determined. Next, in step 130, the nature of the communication variants determines the distribution which is utilised in the Thompson sampling. For categorical communication variants where there only a set of defined versions to choose from (e.g. red or blue buttons in a webpage or email) a beta distribution (step 132) may be used in the subsequent Thompson sampling 140. Alternatively, if the variant is a continuous variant (step 134) (e.g. prices in an electronic message or email) a normal gamma distribution is used in the Thomson sampling performed at step 140. As applied in the present disclosure, the outcome of the Thompson sampling step 140 is a suite of “winning” bandit (communication variant) choices expected to maximise the likelihood of a detectable action occurring based on segmentation features identified amongst the attributes of a specified user profile when provided with a communication over the network. In the asynchronous or batch scenario, Thompson Sampling is used to determine the traffic allocation amongst all the different bandits / communication variants for all the different segments.

[0015] 14 424954 456477\41405803_1.docx In particular, a random is obtained inside the area of each beta distribution for a given customer as depicted in step 146. The number of times that the variant has been chosen for each active segmentation is then determined in step 148. Next, in step 149, the probabilities of each variant for the active segmentation may then be assigned based on the result from step 148. In a real-time scenario, Thompson Sampling determine which is the best bandit / communication variant to be shown for a given customer that belongs to a certain segment. In particular, a random value is obtained inside the area of each beta distribution for a given customer as depicted in step 142. The communication variant with the highest value in the X axis is then shown to the customer. For each user profile, a winning bandit is selected using Thompson sampling on said segmentation features for said user profile if there exists at least one detectable action recorded by the network for said variant for other users with the segmentation features identified for that user. Alternatively, if there is no winning bandit for said segmentation features for said user profile the bandit is randomly chosen. This is typically the scenario where the experiment has just begun; and there is no information established yet as to the performance of the communication variants. The success rate for the detectable action for that user profile is advantageously based upon historical success rate which may be retrieved from a database or other data store; or otherwise associated with the user profile and provided for example by a cookie. As is known the art, Thompson sampling statistically samples based upon known performance of that sample. Using Thompson sampling, as the number of user profiles allocated to the communication variants increases, the overall reliability of the allocation increases, and amongst the communication variants for specific segmentation features of a user profile a relative winner emerges. The “winner” is the communication variant having the highest likelihood of leading to a detectable action occurring for the specific segmentation features of a specific user profile. This identified communication variant for the user profile can then be returned in step 142; and if appropriate an appropriate communication variant generated (see step 119A of Fig 2A). Fig 2C depicts an example of traffic allocation for a user profile with specific user attributes; in this case iOS, male, French. It would be appreciated that the data is merely exemplary and other values could (and indeed would) be observed in an actual experiment. As depicted, the “winning communication variant” or bandit is the bandit marked beta winner 5030, 1232 for this profile; this having variants text 2, image 1, button 2 in the communication. The values in parentheses

[0016] 15 424954 456477\41405803_1.docx represent the number of successes and failures for the action undertaken by that profile. This communication variant therefore has the highest probability of providing the desired action; with 5030 success and 1232 failures in the experiment undertaken. For ease of reference, 4 distributions only are depicted; but it would be appreciated an appropriate number of bandits for each variant issued would be created. Fig 3 depicts an overview of an exemplary segmentation discovery process 200 according to an aspect of the present disclosure. As depicted, the specified experimental setup is received from the experiment administrator in step 202; comprising the specified communication variants 102 and detectable interaction in step 104. Customer or user historical data is received including various user attributes such as demographic information specific to the user (e.g. age, gender, nationality, location, marital status etc.) or their means of receiving the message (e.g. device, operating system, model) etc. Other types of user attributes which may be received without limitation. In addition, the performance of any detectable action undertaken by the user in response to receiving a communication; and the communication variant which was proximal to that detected interaction may also be specified by the experiment controller. In an exemplary embodiment to test the performance of a particular communication variant sent to a specified user population looking to measure the click or interaction with the communication may use an exemplary specified experimental setup as represented in tabular form below. It would be appreciated that this is non- limiting; with other user attributes; user attribute values and communication variants possible. In order to identify the best communication variant strategy in accordance with business requirements, it is advantageous to determine which user attributes offer the best segmentation insights; and customize the current experiment based upon responses received (or future experiments depending on the duration of the experiment). User Attributes User Attribute values Communication Variants Gender Male -0 Button Female-1 Test A->0 Test B->1 Test C→2 Operating System of device Android-0 Image iOS-1 Test A->0

[0017] 16 424954 456477\41405803_1.docx Test B->1 Test C→2 Country of Residence Spain-0 Text Italy-1 Test A->0 France-2 Test B->1 Test C→2 Language English-0 Other-1 As is known in the art, the historical data set of records for individuals may be represented as a mapped matrix of values (e.g. male = 0, female=1); as depicted for the first four records shown below. Out[5]: target-clicked gender os variants_text variants_image variants_button country language 0 True 0 0 0 1 0 0 0 1 True 0 0 1 1 0 2 1 2 False 0 0 0 2 2 1 0 3 True 1 0 0 0 1 2 0 4 True 1 0 1 0 2 1 1 A set of trees may be generated for with the attribute represented as a node; and the values as edges or paths for all user profiles, which may be visualised as a set of tree structure graphs as depicted in Figs 5A-5C. It would be appreciated that FIG 5A is an abstracted form analogous to the graphs depicted in Fig 5B, Fig 5C. In this implementation, it would be appreciated that where there a n-possible attributes that may be used for segmentation, n-1 tree structure graphs would be created; and the gain for each of the edges or branches of these graphs as well as the confidence of that gain for that branch determined relative to all detectable interactions by the users of the population as is discussed further below. Each additional tree structure graph may be created such that the root node of each additional tree structure graph is the next highest gain for detectable interactions of the communications relative to all detectable interactions amongst attributes of said user profiles. Alternatively, in an optional implementation, trees may be not be created for all segment values of the user profiles. Instead, in this optional implementation, at step 204a a root node may be selected from amongst the plurality of user attributes based upon a determination of most gain provided by that root node relative to the others of the plurality of user attributes.

[0018] 17 424954 456477\41405803_1.docx This process is outlined in more detail with reference to Figs 4A (i) - (ii) ; and by reference to the above exemplary example as depicted in Fig 4B. If a root node (user attribute) which provides the highest score is determined; a decision tree where each attribute is a node and the values of that attribute are edges may be generated for use in determining probabilities for other user attributes in an exemplary graph in step 206. In the optional implementation this step may be conceptualized as an exemplary theoretical tree being created for all other user attributes having the identified maximum theoretical gaining root node. This tree therefore has all possible permutations starting from the identified root node with the highest gain, even if such permutations of user attribute values do not exist in the historical data set provided. Reference can be made to Fig 5A for a theoretical representation, and to Fig 5B where an exemplary tree has been created for the data provided and is discussed in more detail with the best performing root node being the Operating System (OS). If the implementation is NOT based on identifying a single root node; then additional trees such as that depicted in Fig 5C and Fig 5D may be created with different root nodes may be created. Next, in step 208, the best splits are selected from the relevant trees which have been created for each user profile. Typically using the UCB1 algorithm (upper confidence bound algorithm); each profile for each segment is assigned or allocated to a test in that segment attribute which provides the highest UCB1 score. In step 210, the branches of the theoretical tree(s) created which do not provide any gain are pruned or deleted. Importantly, this also includes aggregating branches of the tree structure for user profiles with the same best communication variant. This pruning takes place to remove from all possible permutations of user attributes the user attributes in the historical data records for which there exists no information on the performance of any of the particular communication variant(s); or for which the performance of the various communication variant are the same irrespective of the values of that user attribute. Pruning is discussed in more detail with reference to Fig 6A and Fig 6B. In step 212 the expected gain and confidence for each branch of the tree after pruning of the redundant or non-discriminatory communication bandits is determined in an exemplary embodiment using a Monte Carlo simulation. This is discussed further with reference to the steps outlined in Fig 7. Other alternative approaches to determination of the expected gain and

[0019] 18 424954 456477\41405803_1.docx confidence could be performed by persons skilled in the art using techniques including as quasi- monte carlo simulation, closed form equation solving and predication via a machine learning module. The attributes of said user profiles from said tree structure which exceed a predetermined threshold for gain and confidence value may be returned as segmentation features for said population of users. Accordingly, the user attributes of the user profiles which provide the best segmentation have been identified; with the expected gain and confidence of those branches determined. Referring now to Fig 4A (i), there is depicted the steps in the process of identifying the best root node (user attribute) (step 204 as depicted in Fig 3) by evaluating the entropy of the dataset before and after a transformation. As depicted in step 205a, all communication variants are treated as dependent variables. The assumption here is that the actions are dependent between each other. If they are not, that means that we can treat them independently, and therefore, each variant in Thompson sampling will correspond to one action, not to a set of actions. The mean for all bandits or communication variants for all the user profiles for that segment (i.e. root node) is calculated and stored. As depicted in step 205b, the best segment for allocation each user profile (specific combination of attributes) based on the highest mean for that communication variant. Next, in step 205c, the mean is multiplied by a weighting factor determined from the relative percentage of the user attribute occurring amongst the historical data. This ensures proportionality of that user attribute relative the user profiles in the sample; so that the mean is not unduly influenced by an unusual distribution of that attribute amongst the user profiles analyzed at that particular time. Finally, in step 205d, the segment (user attribute) which has the highest score is selected as a root node or starting node for the next stage of the analysis. This is represented pictorially for a theoretical set of data in the set shown below in Fig 4A (ii), with the trees as shown. In this case the abstracted data of user profiles in the matrix is represented by comprises: SEG1: Seg_1 Val_1; Seg_1 Val_2; Seg_1 Val_3;

[0020] 19 424954 456477\41405803_1.docx SEG 2: Seg_2 Val_1; Seg_2 Val_2; SEG 3: Seg_3 Val_1; Seg_3 Val_2. It would be appreciated that this could represent the user profiles in the example described above are a data level representation of the following user attributes and possible values of said attributes as depicted in Fig 4B. SEG 1: Country: Spain, France, Italy; SEG 3: Gender: Male: Female; SEG 3: Operating System: Android; iOS For the purposes of an example, where a single root node is selected and only one tree generated assume that the operating system attribute of the user profile was determined to provide the highest level of gain of all the single level trees generated for possible user attributes. Referring now to Fig 5A, there is depicted the outcome of the step 206 where all of the possibilities may be visualised for all values of the user attributes having a root node of Segment 2 is generated for the data set above. (If multiple graphs with different root nodes are generated then a similar visualisation would be appropriate as depicted in Figs 5B, 5C for a specific example, for each tree generated.) Referring now to Fig 6A, there is depicted an abstract representation of pruning redundant / non- discriminatory user attribute values from the tree representation of Fig 5A; while Fig 6B represents the same approach for the specific example discussed herein. If multiple trees have been created; it would be appreciated that the same pruning processes could be applied to the branches of such trees. Pruning may be performed for each communication variant or bandit based upon the highest mean obtained for a variant; or by using Upper Confidence Bound (UCB1) algorithm. Advantageously, if the higher mean approached is utilized; the other bandits / communication variants no longer need to be considered. In an application of the example being discussed, it is noted using the best mean approach is selected and with reference for the tabular representation shown below; it is found that for OS android, the user country attribute does not provide any useful insight as the performance is the same for all communication variants across the various countries.

[0021] 20 424954 456477\41405803_1.docx OS Country Variants_text Variants_image Variants_button iOS Spain 1 0 0 iOS Italy 2 1 1 iOS France 1 1 0 Android Spain 2 1 2 Android Italy 2 1 2 Android France 2 1 2 Hence this user attribute does not provide any useful segmentation and may be removed from possible allocation of user profiles in step 118 of Fig 2A to test later in the experiment. This may be conceptualised as deletion from the theoretical tree and depicted in Fig 6B in dotted outline for ease of reference; and as depicted in the table shown below. Hence, only profiles outlined below need to be allocated, with the other profiles being deleted and OS Country Variants_text Variants_image Variants_button iOS Spain 1 0 0 iOS Italy 2 1 1 iOS France 1 1 0 Furthermore, and alternatively, it may be discovered in another iteration that where the segment Operating System is ioS, the user profiles with attributes of country being Spain and Italy behave the same with the same “winner” communication variant; OS Country Variants_text Variants_image Variants_button iOS Spain 2 1 2 iOS Italy 2 1 2 iOS France 1 1 0 Android Spain 1 0 2 Android Italy 2 1 2 Android France 2 1 0 Therefore in tabular form this may be represented as below; and the segment attribute of that segment aggregated as shown in Fig 6B. OS Country Variants_text Variants_image Variants_button iOS Spain, Italy 2 1 2 iOS France 1 1 0 Android Spain 1 0 2 Android Italy 2 1 2 Android France 2 1 0 Alternatively, the brute force approach may be employed where all user profiles are simulated and only those profiles which can be allocated in a segment and provide a mean which exceeds

[0022] 21 424954 456477\41405803_1.docx a predetermined threshold and under an acceptable standard deviation threshold are retained. As would be appreciated; where there are multiple possible user attributes by which segmentation could be conducted; this brute force approach increases the computational load required in view of the data which must be processed. Optionally, the confidence interval of the gain may also be determined after normalising the increase in the expected mean obtained via such simulation. If the Upper Confidence Bound 1 (UCB1) algorithm is applied to score the performance of the variant for the data set being a analysed a score may be derived using an UCB1 representing the likelihood of the detectable action being performed by that user profile at that segment in respect of each communication variant issued to that segment. The highest score may then be selected to identify the best performing communication variant for that profile and hence segmentation effect of that user attribute. UCB1 may exploit the segments where the performance is known to have good performance and different segments may be explored where there is not sufficient data accumulated. The outcome of any of these approaches is to select the communication variant or winner bandit for user values which provides the most meaningful outcome; reducing the number of simulations that need to be performed as can be appreciated when comparing Fig 6A with Fig 5A; and Fig 6B with Fig 5B. Once the bandits have been identified for the historical data comprising the user profiles and behavior; in an exemplary embodiment a Monte Carlo simulation can be performed with the steps as described with reference to Fig 7. The Monte Carlo simulation enables an estimation of the mean of the gain for a given profile in the future by applying Thompson sampling N x M times. Other approaches could be employed in the determination of the gain and confidence; as known to a person skilled in the art including by using a quasi-monte carlo simulation, closed form equation solving or prediction via a machine learning model. Based on the pruned branches in the tree(s) remaining; t the expected gain and confidence in that determined gain for each branch of the tree(s) which are generated; according to the flowchart steps described 212 in Fig 7. This may be performed by conducting a Monte Carlo simulation or other techniques such as quasi Monte Carlo simulation, closed-form equation solving, prediction via a machine learning model.

[0023] 22 424954 456477\41405803_1.docx As depicted, in step 212a for each profile in the tree the alpha and beta values of each test is calculated; where alpha is the number of successes and beta is the number of failures which are used in the beta distributions. Next, in step 212b, a random beta is introduced amongst all the beta distributions. The number of times each test has the highest value relative to the x axis (probability) of the distribution is then counted in step 212c. The expected performance for each test can then be calculated in step 212d by multiplying the probability of success for the detectable event for a particular communication variant by the number of times each variant has been chosen. Finally, the mean and standard deviance is then calculated for each branch in the tree(s) is then calculated in step 212e. This means that the expected gain and confidence for each branch of each tree generated for the user attributes can then be returned in step 212f. Turning now to a first example, which illustrates how the issuance of one or more communications over a network to a specified user of a population of users can be personalised. The specified user has a plurality of user attributes and the one or more communications are selected from a plurality of variants thereof. Feedback or response to the communication may be logged by a third party gateway managing the issuance of the communications; with this response being provided to the system of the present disclosure for determination of the segmentation and of which communication variant to issue. This may be provided on a user-by-user basis; or a grouped basis for a particular iteration of the experiment. Steps in Experiment 1. A business professional wants to conduct an experiment to test the same email template (communication variants) sent at different times (continuous features) between 8 a.m. and 10 p.m. to a population of users in the company customer base over a predetermined time period such as a week, two weeks or a month. The detectable target interaction to optimise is the click rate (categorical metric). For ease of understanding the sole difference in this example is between template A and template B is simply what time the

[0024] 23 424954 456477\41405803_1.docx same email is sent in this experiment, but such templates could comprise other changes to elements such as button, image text used etc. 2. The experiment starts. There is no knowledge about which customer segment prefers which email communication variant - in this experiment; the first emails are sent randomly, i.e., any template can be sent at any time to any customer as there are no records of performance. 3. The system records and maps click / no click with variants of template, time, and feature of the customers receiving the email (e.g., gender, OS - iOS, Android, proximity to certain location in a city etc.) recorded. 4. The system begins to weight the variants according to their historical performance, e.g. sending more template variant A at 9.37 am and less template variant B at 1.45pm. 5. The system continues sending email until the number of records reaches a statistical threshold allowing to stop considering all customer as one unique segment and to create multiple segments which will have their respective weighting for each template / time combination. 6. The system identifies three new segments (e.g., male > 33 years old, male <= 33 years old, and female), provides the distinct weighting of the combinations of the communication variants suggested for each of the three new segments, and indicates the expected gain (i.e. the expected click-rate increase) with this new segmentation. 7. The new segmentation is applied, and emails are sent accordingly. Each time the algorithm detects new segments, the new segmentation is applied, and whenever there is more data that needs to be allocated to a certain variant, this new segmentation will be applied 8. More records are accumulated as more emails sent and clicked / not clicked, allowing to update further the segmentation and weighted. 9. The process continues until the business professional decides to stop the experiment. In a further exemplary example, an 8-week experiment was conducted in a first country. The segmentation applied to the customer base was simplified to either Android or iPhone users, and emails proposed renewal of a certain phone protection guarantee.

[0025] 24 424954 456477\41405803_1.docx The target detectable action was email open rate. Two different email templates could be sent at 8 different times (i.e., 2 x 8 = 16 communication variants in total). It would be appreciated that as described earlier herein these parameters are merely exemplary and many other possible variations of the detectable interaction and templates could be used. As depicted in Fig 8A, after the lapse of a predetermined amount of time, the end of the experiment was reached. It was noted that both customer segments evaluated after segmentation had already been performed converged to have the highest open rate of the email template A of 69% when sent at 8.30 am. The experiment led to an increase of the click rate of 10 points relative to template A opening when sent at other times and when compared with the performance before the experiment. As depicted in Fig 8B, the cumulative weighting of the templates increases to favour Template A at 8:30 am as the number of emails are sent increases; given the increased performance of this template over the course of the experiment to maximise open rate. It can be seen that as the total number of emails sent approaches 5,000, approximately 8X the number of the most likely successful template (Template A at 8:30 am) are sent as compared to Template A at 9:00 am. This experiment demonstrates the significant performance improvement offered by the generation of customised communication variants over the course of the experiment. In another experiment in another country, the customer base was segmented between high-value and low-value phone users. In an experiment short message service messages (SMS) were sent with the objective of reducing the churn rate (ratio of customers unsubscribing) for a certain phone protection guarantee. The SMS can be sent either 60 days, 85 days, at the end of the 2nd month, or at the end of the 3rd month after the subscription starts. There is also one choice consisting of not sending any message. The detectable action was measuring whether or not the customer went to a different provider or not. It was determined that after 4 months, 2,130 records; the messages sent at the end of the 3rd month after the subscription starts improved retention rate by 20 points when compared with not sending any message. Reference is made to Fig 9, in which these results are visualised. This example provides the insight that sending messages at the end of 3 months before expiry improves customer retention by 21.4 ppts relative to not sending a message and we are 99.9% sure of this result.

[0026] 25 424954 456477\41405803_1.docx Group Number of Number of Retention Rate Chance of subscriptions retained (%) beating “No subscriptions message” No message 894 406 45.4 - 60 days 210 107 51.0 80.9 2 months end 363 172 47.4 73.9 3 months end 385 239 66.8 99.6 subtotal 1852 924 It would be appreciated that many different scenarios could be investigated and specified by the experimenter; including testing the messages to determine external parameters of communication variants such as at what time of the day is it best to send the email for highest response rate, what day of the week is the best day etc. Alternatively, different communication variants which have different internal message parameters such as different coloured buttons, text inside the button, images, layout, length, call to action etc. may also be tested to determine which parameter or combination thereof has the highest response rate for particular populations. The present disclosure teaches a method of discovering optimal segmentation attributes from a plurality of user attributes associated with user profiles in a population; for which there is some information known including detectable interactions with an electronic communication which has been sent to them and as a result of which a detectable action occurs. The method and system of the present disclosure provide an efficient and easily operable system in which subsequent experiments can be set up to optimise the behaviour desired for a group of users with specific attributes. This provides rapid insight into the effectiveness of a campaign; and allows the personalisation of communication variants so that groups of users are likely to receive an appropriate variant most likely to result in a detectable interaction. The system and method of the present disclosure avoids inappropriate segmentation and over segmentation of customers according to features that provide little predictive insight. This in turn reduces computational load on processors conducting the analysis; and shortens analytical time. Furthermore, the segmentation process is able to be explained; and hence is no longer a “black box”. Methods according to the above-described examples can be implemented using computer- executable instructions that are stored or otherwise available from computer readable media. Such instructions can comprise, for example, instructions and data which cause or otherwise

[0027] 26 424954 456477\41405803_1.docx configure a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Portions of computer resources used can be accessible over a network. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and / or information created during methods according to described examples include magnetic or optical disks, flash memory, Universal Serial Bus (USB) devices provided with non-volatile memory, networked storage devices, and so on. Devices implementing methods according to these disclosures can comprise hardware, firmware and / or software, and can take any of a variety of form factors. Typical examples of such form factors include laptops, smart phones, small form factor personal computers, personal digital assistants, and so on. Functionality described herein also can be embodied in peripherals or add- in cards. Such functionality can also be implemented on a circuit board among different chips or different processes executing in a single device, by way of further example. The instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are means for providing the functions described in these disclosures. It will be appreciated that the embodiments relating to clients, servers, services, state machines and systems may be implemented in a combination of electronic hardware, firmware and software. The firmware and software may be implemented as a series of processes, applications and / or modules that provide the functionalities described herein. The algorithms and processes described herein may be executed in different order(s). Interrupt routines may be used. Data may be stored in volatile and non-volatile devices described herein and may be updated by the hardware, firmware and / or software. Although a variety of examples and other information was used to explain aspects within the scope of the appended claims, no limitation of the claims should be implied based on particular features or arrangements in such examples, as one of ordinary skill would be able to use these examples to derive a wide variety of implementations. Further and although some subject matter may have been described in language specific to examples of structural features and / or method steps, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or acts. For example, such functionality can be distributed differently or performed in components other than those identified herein. Rather, the described features and steps are disclosed as examples of components of systems and methods within the scope of the appended claims.

[0028] 27 424954 456477\41405803_1.docx

Claims

CLAIMS 1. A computer implemented method for determining segmentation features for a population of users, each of said users having a user profile comprising values associated with a plurality of user attributes; the method comprising: determining, by a processor, one or more detectable interactions by users from said population to a plurality of communication variants issued thereto; determining, by a processor, gain for the user attributes of profiles for which a detectable interaction with a communication variant exists; said gain being determined relative to all detectable interactions by users of said population; generating one or more tree structure graphs based upon the determined gain for the attributes; said tree structure graphs having branches comprising nodes representative of said attributes and edges representative of the possible values thereof, and allocating user profiles to branches of said tree structure according to the values of the respective user attributes, determining by a processor the best communication variant for each group of user profiles allocated to each branch, aggregating branches of the tree structure for user profiles with the same best communication variant; determining for all remaining branches of the tree structure graphs the expected gain and confidence value; returning attributes of said user profiles from said tree structure which exceed a predetermined threshold for gain and confidence value as segmentation features for said population of users.

2. The computer implemented method according to claim 1 further comprising deleting branches of the tree structure for attributes of user profiles which lack detectable interactions and / or which are below predetermined thresholds for gain and / or confidence value.

3. The computer implemented method according to any one of the preceding claims wherein the best communication variant for a user profile is selected from the plurality of communication variants by a method selected from the group comprising determining the upper bound confidence interval for each profile; determining the highest mean amongst communications variants28 424954 456477\41405803_1.docxprovided to each user profile or by determining which communication variant for a profile exceeds a predetermined performance threshold and falls below a predetermined standard deviation.

4. The computer implemented method according to any one of the preceding claims wherein the generating of the one or more tree structure graphs comprises generating a first of the one or more tree structure graphs which has a root node having the highest gain for detectable interactions relative to all detectable interactions for each of the other user attributes.

5. The computer implemented method according to claim 4 wherein the generating of the one or more tree structure graphs comprises generating up to n-1 additional tree structure graphs, where n is the number of attributes in said user profiles.

6. The computer implemented method according to claim 5 wherein the root node of each additional tree structure graph is the next highest gain for detectable interactions of the communications relative to all detectable interactions amongst attributes of said user profiles.

7. The computer implemented method according to claim 1 wherein the expected gain and confidence value for the remaining structures of the tree structure is determined using a technique selected from a group of techniques comprising performing a Monte Carlo simulation; a quasi Monte Carlo simulation, closed form equation solving or prediction via a machine learning model.

8. A computer implemented method for personalising the issuance of one or more communications over a network to a specified user of a population of users; wherein said specified user has a plurality of user attributes and said one or more issued communications are selected from a plurality of communication variants, wherein the method comprises: identifying by a processor segmentation features from amongst said user attributes of users in said population of users using the method of claim 1; generating by a processor a communication for issuance to said specified user by selecting the communication variant most likely to be associated with a detectable action being performed by said user; wherein said selection is performed by identifying said variant either by: (a) using Thompson sampling on said segmentation features for said user if there exists at least one detectable action recorded by the network for said variant for other users sharing the same values of the identified segmentation features; or else by29 424954 456477\41405803_1.docx(b) randomly choosing a communication variant from said plurality of communication variants where there does not exist at least one detectable action for said variant for other users having the same values of said segmentation features and confirming the presence or absence of a subsequent detectable action.

9. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 8 wherein the communication variant is transmitted across the network as an email message or text message or other electronic communication.

10. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 9 wherein the detectable action by said first user comprises detecting an interaction by said user with the one or more issued communications.

11. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 10 wherein the detectable action comprises opening the one or more issued communications.

12. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 8 wherein each communication variants have identical elements contained therein but differ in the time of despatch.

13. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 8 wherein the detectable action by said first user comprises detecting an interaction with one or more elements of the issued communication.

14. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 13 wherein the detectable interaction by the said first user is clicking on a button element contained in the communication.

15. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 14 wherein the element has parameters selected from the group comprising typeface, image, size, colour and shape.

16. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 8 wherein the detectable action performed by said user is an interaction recorded on a data store in communication with the processor independent of any detectable interactions by the user with the communication.30 424954 456477\41405803_1.docx17. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 8 further comprising generating a further one or more communication variants for at least one successive other user from the population of users; wherein said further generated communication variant is influenced by the presence or absence of detectable action responsive to transmission of an earlier communication variant to said other preceding users having one or more similar attributes; said action determined for within a predetermined time frame.

18. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 17 further comprising repeating the steps thereof for a plurality of communication variants to a plurality of the population of users over a predetermined time period for determining the distribution of detectable actions for user profiles thereof in the population.

19. The computer implemented method for personalising the issuance of one or more communications over a network according to claim 18 comprising generating a report on the detectable actions performed in response to said transmitted communication variants issued to at least some of said population of users.

20. The computer implemented method for personalising the issuance of one or more communications over a network according to any one of the preceding claims 8-19 further comprising specifying the number and attributes of at least some of the population of users and using said simulated users for determining an communication variant strategy.

21. A computer system for determining segmentation features for a population of users, each of said users having a user profile comprising values associated with a plurality of user attributes; the system comprising: a database configured to store one or more detectable interactions by users from said population of users to a plurality of communication variants issued thereto; a segmentation module comprising one or more processors configured to perform the method according to any one of claims 1-7.

22. The computer system of claim 21 further configured to personalise the issuance of one or more communications over a network to a specified user of a population of users; said system further comprising an analysis module having one or more processors configured to perform the method of any one of claims 8-20.31 424954 456477\41405803_1.docx23. A non-transitory computer-readable storage medium, storing program instructions computer-executable by a processor of a computer to perform the computer implemented method according to claims 1-20.32 424954 456477\41405803_1.docx

Images