Information processing device and information processing method

Abstract

An information processing device according to one embodiment of the present invention includes: an acquisition unit that acquires an effect of a measure on each individual user belonging to a first user group having used a first store in which the measure has been implemented, the effect being calculated from a comparison between records of the first user group and records of a control group in the measure; and an output unit that outputs a result of clustering the first user group into a measure-exposed group and a measure-non-exposed group on the basis of the effect on each individual user.

Description

Information processing device and information processing method

[0001] This invention relates to a technology for estimating the proportion of users who have been exposed to a measure among those who have used a store that has implemented a measure.

[0002] Technologies for evaluating the impact of advertising campaigns are known. For example, Patent Document 1 discloses an invention that estimates the impact of an advertisement by considering the manner in which a user interacts with an advertisement based on their location information for outdoor advertisements.

[0003] Patent No. 7485801

[0004] The invention described in Patent Document 1 merely employs a method for identifying advertisers based on the user's location information.

[0005] In contrast, the present invention provides a technology for appropriately and efficiently classifying individual users who have used stores implementing the policy into those who have been exposed to the policy and those who have not.

[0006] An information processing device according to one aspect of the present disclosure includes an acquisition unit that acquires the effect of the measure on each individual user belonging to the first user group, calculated from a comparison of the performance of the first user group that used the first store where the measure was implemented with the performance of the control group in the measure; and an output unit that outputs the result of clustering the first user group into a group that was exposed to the measure and a group that was not exposed to the measure, based on the effect of each individual user.

[0007] An information processing method relating to another aspect of this disclosure includes the steps of: a computer obtaining the effect of the measure on each individual user belonging to the first user group, calculated from a comparison of the performance of the first user group that used the first store where the measure was implemented with the performance of the control group in the measure; and outputting the result of clustering the first user group into a group that was exposed to the measure and a group that was not exposed to the measure, based on the effect of each individual user.

[0008] According to the present invention, individual users who have used stores implementing the measures can be appropriately and efficiently classified into those who have been exposed to the measures and those who have not.

[0009] A diagram illustrating the system configuration of an information processing system 1 according to one embodiment. A diagram illustrating the functional configuration of the information processing system 1. A diagram illustrating the hardware configuration of the information processing device 10. A sequence chart illustrating a method for estimating the number of people exposed to advertisements in the first user group. A diagram illustrating the management server database 1000. A flowchart illustrating a method for specific processing in the information processing system 1. A diagram illustrating a DID analysis model for ATT of the first user group. A diagram illustrating the ATT distribution 2000 for the first user group. A flowchart illustrating a clustering method for the first user group.

[0010] 1. Diagram 1 illustrates the system configuration of an information processing system 1 according to one embodiment. In this example, the information processing system 1 (or simply the system) is a system for estimating individuals who have been exposed to advertisements from among individual store users when evaluating the effectiveness of measures such as advertisements implemented in stores. In this example, stores include, for example, physical stores (retail stores, restaurants, tenants, or specialty stores). It also includes individual online stores, member stores of various point programs, or individual web stores operated by online businesses. In a broad sense, measures refer to activities introduced for the purpose of sales promotion, and among these, they particularly include advertisements, campaigns, coupons, or various promotions conducted for customers.

[0011] Here, the measures include advertising media where it is unclear whether customers actually viewed the advertisements, such as in-store advertising (e.g., POP advertising or digital signage advertising), outdoor advertising (e.g., billboards), or flyer distribution. In other words, because it is unknown which of the store users saw the advertisements, it is possible that the advertising effect cannot be properly evaluated (for example, the advertising effect may be underestimated or overestimated). Regardless of the form of the advertising media, in order to properly evaluate the measures, it is necessary to accurately estimate the proportion of customers who used the store where the measures were implemented who were exposed to the advertisements.

[0012] Conventional related technologies have employed methods to identify individuals exposed to advertisements, such as using customer location information or determining objects viewed based on actions like eye movements. However, these methods have drawbacks in terms of privacy concerns, high barriers to implementation, and cost (and effort). Therefore, there is a need for a novel approach to replace these related technologies. Accordingly, the present invention provides the following system for appropriately and efficiently classifying individual users into those exposed to the campaign and those not exposed.

[0013] The information processing system 1 comprises an information processing device 10, user terminals 20 (such as user terminal 211 in the figure), a management server 30, and store terminals 40 (such as store terminal 41 in the figure). In this example, each component of the system is connected via a network 9 as shown in Figure 1. In this example, the network 9 is a computer network such as the Internet or a mobile network.

[0014] The information processing device 10 is an information processing device or server device in the information processing system 1. In this example, users who used the stores where the campaign was implemented are classified into a group of people exposed to the campaign and a group of people not exposed to the campaign. The group of people exposed to the campaign (group G11 in the figure) is a collection of users who are thought to have been exposed to the advertisement in various ways, and includes, for example, a collection of store users who viewed the in-store advertisement. Conversely, the group of people not exposed to the campaign (groups G10 and G20 in the figure) includes, for example, a collection of users who did not view the in-store advertisement, or a collection of users who did not have the opportunity to physically come into contact with the advertisement. In the situation described above, it is unclear which users viewed the advertisement, so the information processing device 10 aims to classify users with high accuracy.

[0015] Furthermore, the information processing device 10 is a device for evaluating policies using a method called Fuzzy-DID (Fuzzy Differences-in-Differences). Fuzzy-DID is a method for evaluating the influence of specific elements on a subject. For example, DID analysis is known for comparing an intervention group (an example of the first user group) in which a specific element is introduced with a comparison group (an example of a control group) without the intervention. In Fuzzy-DID, in addition to comparisons using normal DID analysis, an "intervention rate" indicating the degree to which the policy has an impact on the intervention group can be considered, enabling a more accurate evaluation of policies.

[0016] The user terminal 20 is a terminal used by store users (so-called end users) in the information processing system 1. The user terminal 20 includes, for example, a smartphone, tablet, or personal computer. In this example, the user terminal 20 is owned by each user belonging to the intervention group (or control group), and various data operated by the user via the terminal are recorded as logs. The data recorded on the user terminal 20 is output to the management server 30 as appropriate.

[0017] The management server 30 is a server device that manages information about policies, results (e.g., payment amounts), and users. In this example, the management server 30 acquires data such as the user's location information, store usage information, or payment information from the user terminal 20. The management server 30 also acquires information about the store (e.g., sales, policy status, users, business type, category, or location) from the store terminal 40.

[0018] The store terminal 40 is a terminal corresponding to the POS (Point of Sales) system of an individual store, and includes, for example, a smartphone, tablet, or personal computer. In this example, the store terminal 40 can record performance data (e.g., sales performance) for each store or each user. The information collected from the store terminal 40 is used, for example, to calculate performance or estimate the effectiveness of measures.

[0019] In Figure 1, store M41 (an example of the first store) where the measures were implemented and store M42 (an example of the second store) where the measures were not implemented are shown as representative examples. In this example, if it is not possible to identify which users of store M41 actually viewed the advertisement (i.e., whether they belonged to group G11), the effectiveness of the advertisement cannot be accurately measured. Furthermore, the inability to efficiently allocate advertising costs makes it difficult to properly evaluate the measures. Information processing system 1 addresses this problem.

[0020] Figure 2 is a diagram illustrating the functional configuration of the information processing system 1. In this embodiment, the information processing device 10 has functional blocks (or components) consisting of an acquisition unit 11, an output unit 12, a calculation unit 13, a clustering unit 14, an estimation unit 15, an evaluation unit 16, a storage unit 191, and a control unit 192. In this example, the storage unit 191 stores various types of data, programs, and software, including a database. In this example, the control unit 192 performs various types of control.

[0021] The acquisition unit 11 acquires the effect of the measure (hereinafter referred to as "measure effect") based on a comparison between the performance of the first group of users who used the first store where the measure was implemented and the performance of the control group in the measure. In this example, the performance refers to the performance of users in commercial transactions, specifically information on each user's payment at the store, including, for example, the payment amount, number of payments, frequency of payments over a specific period, or the average payment price. The measure effect is the effect of the measure on each individual user (i.e., each person) belonging to the first group of users.

[0022] The output unit 12 outputs the results of clustering the first user group into a group of people who were exposed to the measures and a group of people who were not exposed to the measures, based on the effects of each individual user (hereinafter referred to as "clustering results").

[0023] The calculation unit 13 calculates the effect of the policy on the first user group by comparing the performance of one user belonging to the first user group at the first store with the performance of one user belonging to the control group. In this example, the comparison of the performance of each user belonging to the first user group is performed using, for example, DID analysis. In DID analysis, one user belonging to the first user group and one user belonging to the control group are compared on a one-to-one basis.

[0024] Furthermore, if the control group includes a second group of users who used the second store where the measures were not implemented, the calculation unit 13 will perform the calculation using the performance of one user belonging to the second group at the second store.

[0025] The clustering unit 14 clusters the first user group into a group of users who were exposed to the policy and a group of users who were not exposed to the policy, based on the distribution of ATT, which indicates the effect on individual users. In this example, ATT (average treatment effect on the treated) is an index calculated by the calculation unit 13 that represents the effect of the policy on individual (or multiple) users. ATT is calculated by a predetermined method. The distribution is a set representing the first user group and is represented using a graph such as a histogram.

[0026] It is thought that the ATT of customers who actually viewed the advertisement among those who visited the store where the advertising campaign was implemented (i.e., the first user group) will be calculated from a different distribution than the ATT of customers who did not actually view the advertisement. The clustering unit 14 utilizes these differences in the properties of ATT for each user to visualize the people who actually viewed the advertisement in the graph.

[0027] If the measure includes advertising related to the first store, the estimation unit 15 estimates the proportion of the first user group that were exposed to the measure and saw the advertisement, based on the clustering results.

[0028] The evaluation unit 16 evaluates the impact of the measures implemented at the first store based on the proportion of the group of people exposed to the measures estimated by the estimation unit 15. In this example, the evaluation unit 16 uses Fuzzy-DID to evaluate the impact of the measures.

[0029] Figure 3 illustrates the hardware configuration of the information processing device 10. Physically, the information processing device 10 is configured as a computer including a processor 101, memory 102, storage 103, communication device 104, input device (optional), display device (optional), and a bus connecting these. Each of these devices operates on power supplied from a battery (not shown). In the following description, the term "device" can be read as a circuit, device, unit, etc. The hardware configuration of the information processing device 10 may include one or more of the devices shown in Figure 3, or it may be configured without some of the devices. Alternatively, multiple devices with different enclosures may be connected via communication to constitute the information processing device 10.

[0030] Each function in the information processing device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 101 and memory 102, which allows the processor 101 to perform calculations, control communication by the communication device 104, and control at least one of the reading and writing of data in the memory 102 and storage 103.

[0031] The processor 101 controls the entire computer, for example, by running the operating system. The processor 101 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. Also, for example, a baseband signal processing unit or a call processing unit may be implemented by the processor 101.

[0032] The processor 101 reads programs (program code), software modules, data, etc., from at least one of the storage 103 and the communication device 104 into the memory 102 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described later. Functional blocks of the information processing device 10 may be stored in the memory 102 and implemented by control programs that run on the processor 101. Various processes may be executed by one processor 101, or they may be executed simultaneously or sequentially by two or more processors 101. The processor 101 may be implemented by one or more chips. The program may also be transmitted to the information processing device 10 via a telecommunications line.

[0033] Memory 102 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 102 may also be called a register, cache, main memory, etc. Memory 102 can store executable programs (program code), software modules, etc., for carrying out the method according to this embodiment.

[0034] The storage 103 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 103 may also be called an auxiliary storage device.

[0035] The communication device 104 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0036] Each device, such as the processor 101 and memory 102, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used for each device.

[0037] The information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by this hardware. For example, the processor 101 may be implemented using at least one of these hardware components.

[0038] In this example, the program stored in the storage 103 includes a program (hereinafter referred to as the "server program") that causes the computer to function as a server in the information processing system 1. When the processor 101 is executing the server program, the processor 101, memory 102, storage 103, and communication device 104 are examples of functional blocks for operating the information processing device 10. The processor 101 is an example of an output unit 12, a calculation unit 13, a clustering unit 14, an estimation unit 15, an evaluation unit 16, and a control unit 192. At least one of the memory 102 and the storage 103 is an example of a storage unit 191. The communication device 104 is an example of an acquisition unit 11.

[0039] Although detailed description is omitted, the user terminal 20 and the store terminal 40 are computers having a processor, a memory, a storage, a communication device, a display device, an input device, and an output device, specifically, for example, a smartphone, a tablet terminal, or a personal computer. In this example, the programs stored in the storages of the user terminal 20 and the store terminal 40 include a program (hereinafter referred to as "client program") for causing the computer to function as a client in the information processing system 1.

[0040] The management server 30 is a computer having a processor, a memory, a storage, and a communication device. In this example, the programs stored in the storage of the management server 30 include a program (hereinafter referred to as "management server program") for causing the computer to function as a management server in the information processing system 1. The configuration of the information processing system 1 has been described so far. Next, the operation of the information processing system 1 will be described.

[0041] 2. OperationFIG. 4 is a sequence chart illustrating a method for estimating advertisement contacts in the first user group. Here, an operation example of a method for estimating the ratio of advertisement contacts among the first user group who used the first store where the information processing system 1 implemented an advertisement will be described. The information processing system 1 starts the following processing契机 when receiving a designation regarding an advertisement from a system administrator or an advertisement distributor.

[0042] In step S101, the information processing apparatus 10 makes a specific request for a user to the management server 30. This request includes data specifying the advertisement to be evaluated and an instruction for specifying a store or a user. This instruction includes a store ID, a user ID, a period, or the like. In this example, the management server 30 can perform statistical processing or aggregation processing on various data in response to the request of the information processing apparatus 10.

[0043] In step S102, the management server 30 links the store terminal 40 with the database and retrieves the specified data. Database linking means that the management server 30 shares the database (or at least a part of it) held by the store terminal 40. The store terminal 40 manages POS performance or user performance for each store and periodically outputs various data to the management server 30. This process can be performed at any time, and the management server 30 retrieves data from multiple store terminals 40 as appropriate. For example, the store terminal 40 automatically outputs POS performance or user performance for each store to the management server 30 at midnight every day. Now, let's explain the database regarding the status of users for each store.

[0044] Figure 5 illustrates the management server database 1000. In this example, the management server database 1000 contains multiple records that associate information about each store with information about users. Each record corresponds to information specific to each store. Each record includes a store ID, POS information, store information, policy information, and user information. The store ID is unique identification information for each store. The POS information is information about sales, etc., according to the POS system for each store. The store information is existing information about the store, including, for example, business type, scale of operations, and location. The policy information is information about policies, including, for example, a policy ID that indicates identification information assigned to each policy, and policy content that indicates the specific content of the advertising policy. In addition, the policy information may include various information such as the implementation period, which indicates the period during which the store implemented the target policy (number of days, weeks, or months, etc., and the start or end date). The user information is information about users who have used the store, including, for example, a user ID, payment history, location information history, and terminal history. The user ID is unique identification information for each user. Payment history is a record of past payments made by users at a store. In this example, payment history can be linked to the POS system of each store, allowing for detailed management of user payment records for each store. Location history is a record of the changes in location information obtained from base station information, etc., regarding the location of the user's device. Device history is information about operations performed by the user on their device, or operation logs recorded on the device itself. Specifically, it is information about operation logs recorded on the device itself (e.g., application usage history, browser browsing history, etc.).

[0045] Furthermore, regarding user information, the management server 30 can connect to user terminals 20 (not shown) via the network 9, acquire the various information described above, and record it in the database. This allows the management server 30 to manage the user status for each store.

[0046] Returning to Figure 4, in step S103, the management server 30 performs identification processing. In this example, the identification processing is the process of identifying information about users, stores, and performance used for evaluating the implemented advertisements and estimating those who were exposed to the advertisements. This processing includes the process of extracting data recorded in the management server database 1000. The details of the identification processing will now be explained.

[0047] Figure 6 is a flowchart illustrating a method for specific processing in the information processing system 1. This flowchart corresponds to the operation related to specific processing in step S103 described above, and specifically, it is an operation for identifying the first group of users who used the stores where the measures were implemented and the control group used for comparison. The management server 30 starts the following processing when it accesses the management server database 1000.

[0048] In step S201, the management server 30 identifies the first user group and the second user group from the management server database 1000 according to the specified measures. In this example, the management server 30 identifies the first store that implemented advertising, etc., with respect to the measures specified by the information processing device 10. The management server 30 identifies the customers who used the first store as the first user group. In this example, the conditions for the first user group are set in advance. For example, the information processing device 10 can identify the first user group as customers of the first store who visited during the period in which the measures were implemented (or around that period), such as one week before and one week after the period in which the measures were implemented.

[0049] Furthermore, the management server 30 can identify a second group of users from the management server database 1000. In this example, the management server 30 could first identify a second store that did not implement advertising, and then identify the users who used that second store as the second group of users. In this example, the management server 30 basically identifies a store as the second store whose sales performance data trends before the implementation period are similar. For example, a store whose sales trends for the month before the implementation period are similar. Alternatively, it is more desirable to standardize conditions other than whether or not advertising measures were implemented between stores, and the second store may be identified according to the size of the store, product range, or customer base.

[0050] Although the explanation used the sales performance of each store as an example, any method can be used to identify the second store. The management server 30 may set predetermined thresholds for similarity according to various conditions and identify stores with a similarity of 0.8 or higher (for example, stores with a similarity of 0.8 or higher) as the second store. In this example, the management server 30 identifies the customers of the identified second store who visited during the implementation period of the measure (or before or after it) as the second user group.

[0051] In step S202, the management server 30 further extracts comparison candidates using user-specific information from the second user group. In this example, the comparison candidates are those used for comparison with users belonging to the first user group. That is, in the present invention, this set of comparison candidates can be considered as a control group.

[0052] The criteria for comparable candidates are predetermined, and a key requirement is that they have no prior exposure to advertisements placed at the first store. Therefore, in the simplest method, comparable candidates can be identified by selecting users from the group of users who visited the second store during a specific period (i.e., the second user group), excluding those belonging to the first user group. For example, one condition might be that they have no history of staying near the first store for a certain period of time or longer. This identification is based, for example, on payment history, location history, or device history included in user information.

[0053] Furthermore, any other method may be used to narrow down the list of comparable candidates. For example, information on the trading area of ​​each store may be used, and in this example, users of the second store whose trading area does not overlap with that of the first store on the map may be recognized as comparable candidates. The management server 30 records the set of users identified as comparable candidates as a control group in the database.

[0054] In step S203, the management server 30 acquires performance data for the first user group and the control group. In this example, the management server 30 extracts predetermined performance data for each user from the database, such as payment amount, number of payments, payment frequency during a specific period, or payment unit price, which are indicators of advertising effectiveness evaluated by DID analysis. Note that which indicators (i.e., performance data) are used for DID analysis is determined according to the specifications of the information processing device 10.

[0055] Return to Figure 4. In step S104, the information processing device 10 obtains data on performance from the management server 30. The following describes the process of classifying users belonging to the first user group into those who have been exposed to advertisements and those who have not.

[0056] In step S105, the information processing device 10 calculates the ATT for one user belonging to the first user group. In this example, the ATT per user is calculated based on a method (or definition formula) known as the difference-of-differences method. For example, the ATT is calculated by the following formula (1).

[0057] In this example, "DID" y "DID" is an indicator of advertising effectiveness calculated by the difference between the change in performance of one user belonging to the first user group at the first store and the change in performance of one user belonging to the control group, under the conditions of two or more periods and before and after the implementation of the advertisement. y This corresponds to the ATT for one user belonging to the first user group. Also, "y GXTXRegarding the variables expressed as "GX=G1", GX=G0 indicates that the user belongs to the first user group, and TX=T1 indicates that the user belongs to the control group. Additionally, TX=T1 represents the period after the advertisement was implemented, and TX=T0 represents the period before the measure was implemented.

[0058] In equation (1), for example, "y G1T1 The variable labeled " " represents actual values ​​corresponding to the payment amount by a single user belonging to the first user group at the first store during the advertising period. In DID analysis, the effect on individual users belonging to the first user group to which the advertising intervention was applied can be calculated according to equation (1). Here, we will explain the analytical model that shows how ATT is calculated.

[0059] Figure 7 illustrates a DID analysis model for ATT of the first user group. Figure 7 is an example of a schematic diagram representing a simplified model for calculating ATT for each user belonging to the first user group. In this example, the first user group is the set of users who visited the stores where the campaign was implemented, and this includes both those who were exposed to the advertisement and those who were not, but it is not possible to distinguish between them for individual users. Therefore, there is variation in the performance of individual users in the first user group according to the degree of exposure to the advertisement. On the other hand, the control group consists of those who were not physically exposed to the advertisement. Therefore, the performance of individual users in the control group does not reflect the effect of the advertisement. In this example, the information processing device 10 performs a one-to-one comparison of the ATT of each individual in the first user group using DID analysis.

[0060] Specifically, the information processing device 10 applies the above-mentioned formula (1) using the performance of one user belonging to the first user group at the first store and the performance of one user belonging to the control group. The figure illustrates an example in which user U11 and user U21 are compared using DID.

[0061] Furthermore, based on the principles of DID analysis, it is appropriate to homogenize conditions other than "presence or absence of advertising exposure" as much as possible for users in each user group when making comparisons. Therefore, the information processing device 10 is required to introduce methods such as propensity scores to select a user from a control group (e.g., user U21) that is more similar to a user from the first user group (e.g., user U11). Any other method may be used to select similar users.

[0062] The information processing device 10 repeats the same process until the comparison of all users in the first user group is complete. This allows the information processing device 10 to calculate the ATT (Average User Time) for each individual user belonging to the first user group. The information processing device 10 records the ATT calculation results in a database or other appropriate location.

[0063] Returning to Figure 4, in step S106, the information processing device 10 generates a distribution map of the ATT for the first user group. The information processing device 10 can graph the ATT of individual users in a predetermined manner. Here, the distribution of ATT for the first user group will be explained.

[0064] Figure 8 is an example of the ATT distribution 2000 for the first user group. Figure 8 shows a distribution map generated based on the ATT of individual users belonging to the first user group calculated by the information processing device 10. In Figure 8, the horizontal axis represents ATT, and the vertical axis represents the number of people. In this example, the ATT distribution 2000 plots the number of people who received ATT, corresponding to the horizontal axis. The dots plotted on the horizontal axis for advertisers and those exposed to advertisements conceptually represent the distribution of advertisers and those exposed to advertisements (they are not plots of data). Region A is the graph region formed when the ATT distribution 2000 is visualized as a histogram representing the distribution of the number of people who received ATT.

[0065] Here, it is presumed that the ATT of users who actually viewed the advertisement within the first user group will be calculated from a different distribution than the ATT of users who did not actually view the advertisement. If the effect of the advertisement is positive, the ATT of those exposed to the advertisement will have the advertising effect added to it, so it is likely to show an increasing trend (i.e., the plot will be located on the increasing side of the horizontal axis) compared to the ATT of those not exposed to the advertisement. On the other hand, the ATT of those not exposed to the advertisement will have a distribution with a peak close to zero, because the effect of the advertising effect is removed.

[0066] From this, the ATT distribution 2000 can distinguish between region A1, which is formed from the set of people exposed to the advertisement, and region A2, which is formed from the set of people not exposed to the advertisement. In this example, the peak of the graph waveform representing region A2 corresponds to a distribution where the ATT peak is close to a value of 0. Here, assuming that the advertising effect is added as a positive value, the peak of the graph waveform for region A1 is distributed at a position that shows a value higher than the average ATT of people not exposed to the advertisement (i.e., 0). Note that whether or not a user was exposed to the advertisement is represented by the presence or absence of hatching in each plot (i.e., exposure = hatched).

[0067] Returning to Figure 4, in step S107, the information processing device 10 estimates the number of people exposed to the advertisement using the generated distribution map. In this process, the information processing device 10 analyzes the ATT distribution 2000 and classifies the users of the first user group into those who were exposed to the advertisement and those who were not. The method for estimating those exposed to the advertisement will now be explained.

[0068] Figure 9 is a flowchart illustrating a clustering method for the first user group. This flowchart shows the details of the process in step S107 described above. The information processing device 10 starts the following process upon obtaining the ATT distribution 2000.

[0069] In step S301, the information processing device 10 identifies regions in the ATT distribution 2000 that are indicated by a histogram or the like. In this example, the information processing device 10 detects a region A1 formed from the set of people exposed to the advertisement and a region A2 formed from the set of people not exposed to the advertisement, from the normal distribution generated based on the plots of individual ATTs. Preprocessing of the ATT distribution 2000 may be performed here.

[0070] In step S302, the information processing device 10 performs clustering on the distribution map using a predetermined algorithm. In this example, various algorithms such as GMM (Gaussian Mixture Model), nonparametric Bayes, K-means, or hierarchical clustering are applied as methods for clustering those exposed to the advertisement and those not exposed to the advertisement from the histogram of ATT for the first user group.

[0071] Here, GMM is an algorithm capable of analyzing combinations of multiple Gaussian distributions (or normal distributions). The GMM method, for example, when each Gaussian distribution in the ATT distribution 2000 has a different mean and a predetermined variance (or data spread), can analyze these distribution structures and classify individual users in the first user group. Using this method, it is possible to classify each user belonging to the first user group from the ATT distribution 2000 into those exposed to advertisements and those not exposed to advertisements.

[0072] In step S303, the information processing device 10 estimates the proportion of the first user group that has been exposed to the advertisement. The information processing device 10 determines the proportion of the entire first user group that has been exposed to the advertisement using a quantitative indicator (for example, in units of %). With respect to the proportion of those exposed to the advertisement, the information processing device 10 can calculate the advertisement exposure rate according to the following equation (2).

[0073] In this example, "DID" D This parameter relates to the advertising exposure rate (the so-called intervention rate). s This indicates the number of people exposed to the advertisement within the entire first user group. vindicates the total number of people in the entire first user group.

[0074] The information processing apparatus 10 records the calculation result regarding the intervention rate in the database. As described above, even when the contact state of each user with respect to the advertisement is unknown, the information processing apparatus 10 can estimate the ratio of advertisement contacts.

[0075] Returning to FIG. 4, in step S108, the information processing apparatus 10 evaluates the effect of the measure based on the estimated ratio of advertisement contacts. In this example, the information processing apparatus 10 calculates the implementation effect regarding the advertisement using a predetermined calculation method known as Fuzzy-DID. For example, an index (hereinafter referred to as "effect degree W") quantified as the advertisement effect is calculated by the following formula (3).

[0076] In this example, "DID" Y is a variable indicating the difference in differences between the performance of the first user group that can be calculated using the same calculation formula as formula (1) and the performance of the control group. In formula (1), one user belonging to the first user group (or one user belonging to the control group) was the application target, whereas in formula (3), the entire first user group (or the entire control group) is included as the target for evaluating the influence of the advertisement. "DID" D is a variable regarding the intervention rate, and the ratio (for example, the rate) of advertisement contacts in the entire first user group calculated using formula (2) is applied here.

[0077] In Fuzzy-DID, since an adjustment regarding the intervention rate indicating the advertisement contact situation is performed, if an error occurs in the numerical value of DID D there is a possibility of underestimating or overestimating the influence on the advertisement effect. Therefore, by using the clustering method described so far, even when the contact state of each user with respect to the advertisement is unknown, it becomes possible to obtain an appropriate numerical value of DID D

[0078] The information processing device 10 calculates the effectiveness score W and records the result in a database. Alternatively, the information processing device 10 connects to various devices and outputs the evaluation results (i.e., effectiveness score) of advertising measures in response to requests from other systems, other devices, or various businesses. In this example, "business" refers to a business that operates or manages the information processing system 1, and includes, for example, store operators, advertising companies, or various service providers.

[0079] As a result, the information processing device 10 can estimate the effectiveness of advertising measures at stores. Even when the target audience for the measures is unknown, the information processing device 10 can appropriately and efficiently classify individual users who have used the stores where the measures are implemented into those who have been targeted and those who have not. This allows the information processing system 1 to more effectively estimate the target audience for advertising and evaluate advertising measures from the perspectives of privacy, as well as time costs, monetary costs, and human costs (effort).

[0080] 3. Modifications The present invention is not limited to the embodiments described above, and various modifications are possible. Several modifications are described below. Two or more of the matters described below may be combined and applied.

[0081] (1) Information Processing System 1 The hardware configuration and network configuration of the information processing system 1 are not limited to those exemplified in the embodiments. The information processing system 1 may have any hardware configuration and network configuration as long as it can realize the required functions. For example, multiple physical devices may cooperate to function as the information processing system 1. For example, at least a part of the functions of the information processing device 10 may be implemented as a software program, an application program, or a machine learning device (such as AI). For example, an external device different from the information processing device 10 may have at least a part of the functions related to the information processing device 10. The information processing system 1 may cooperate with existing systems such as a payment system, a base station system, or an SNS (social networking service) system. For example, it may cooperate with a payment system to acquire the user's purchase history, cooperate with a base station system to acquire the user's location information, or cooperate with an SNS system to acquire the user's attribute information.

[0082] (2) Information Processing Device 10 Some of the functions of the information processing device 10 may be implemented on other servers. These servers may be, for example, physical servers or virtual servers (including so-called clouds). Furthermore, the correspondence between functional elements and hardware is not limited to those illustrated in the embodiments. For example, at least some of the functions described in the embodiments as being implemented on the information processing device 10 may be implemented on other devices or systems, or conversely, at least some of the functions described as being implemented on other devices or systems may be implemented on the information processing device 10. In this example, at least some of the information processing device 10 may be implemented on the user terminal 20, the management server 30, or the store terminal 40. This allows the processing load of the information processing device 10 to be distributed and the processing speed of the entire system to be improved. For example, the information processing device 10 may cooperate with a base station system and a POS system, etc., and directly acquire data held by the user terminal 20 and the store terminal 40. The management server 30 may, instead of having a function to aggregate data specified by the information processing device 10, appropriately output data requested by the information processing device 10.

[0083] (3) User terminal 20, management server 30, and store terminal 40 The user terminal 20, management server 30, and store terminal 40 are not limited to those exemplified in the embodiment. The user terminal 20, management server 30, and store terminal 40 may perform the above-described processing using any display screen, input device, external device, or various UI. The management server 30 may implement any functions necessary for recording and managing information related to the data specified by the information processing device 10. The management server 30 may also have a function to automatically collect data from the user terminal 20 and store terminal 40.

[0084] (4) Method for estimating advertisers in the first user group The sequence chart shown in Figure 4 is merely an example of the operation, and the operation of the information processing system 1 is not limited thereto. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In steps S101 to S102, the information processing system 1 may acquire data used for estimating advertisers by any means. The information processing system 1 may cooperate with various systems to which the user terminals 20 and store terminals 40 belong, depending on the data type, to acquire predetermined data.

[0085] In step S105, the information processing device 10 may calculate ATT by any method. Any performance data may be used as an indicator for ATT. For example, purchase amount, number of items purchased, number of website accesses, or application usage time may be used. Equation (1) described in the embodiment may be modified in any way.

[0086] In step S106, the information processing device 10 may use any method to generate the distribution diagram, and the classes set on the coordinate axes of the ATT distribution 2000 may be defined in any way. The settings of the histogram shown by the ATT distribution 2000 may be defined in any way.

[0087] In step S108, the information processing device 10 may use any method to estimate the effectiveness of the advertising measures. The formula (3) described in the embodiment may be modified in any way, as long as it takes into account the proportion of people exposed to the advertisement. The information processing device 10 may also use a predetermined threshold to determine whether or not the measures are effective based on the estimated effectiveness.

[0088] (5) Method for specific processing The flowchart shown in Figure 6 is merely an example of operation, and the operation of the information processing system 1 is not limited thereto. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In steps S201 to S202, the information processing device 10 may identify the first user group and the control group in any way depending on the form of the advertising campaign. In this example, the form of the advertising campaign can be anything (it may be a web advertisement, for example). For example, if the advertising campaign is an outdoor advertisement (i.e., a billboard), the information processing device 10 may identify users of the first store who have a history of visiting the area where the billboard is displayed as the first user group, based on information from the base station data of the user terminal 20, and identify users of the second store who do not have a history of visiting the billboard area as the second user group. The control group may be selected and extracted from those recorded in the management server database 1000 by combining various conditions.

[0089] (6) Clustering Method for the First User Group The flowchart shown in Figure 9 is merely an example of operation, and the operation of the information processing system 1 is not limited thereto. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In steps S301 to S302, the information processing device 10 may perform clustering of the first user group in any way. The algorithm applied may be changed depending on the type of histogram shown by the ATT distribution 2000. In step S303, the information processing device 10 may calculate the proportion of advertisers included in the first user group in any way. Equation (2) described in the embodiment may be modified in any way.

[0090] (7) Database (Data) The database (or the data itself) of the information processing system 1 shown in Figure 5 is not limited to those exemplified in the embodiment. In this example, the data registered in the database can be anything. The management server database 1000 may record any kind of data. For example, user information may have timestamps, tags, keywords, labels, or various metadata attached to it. The data recorded in the database can be anything, for example, text, images, graphs, or lists.

[0091] (8) Other programs executed by the processor 101 may be provided by download via a network such as the Internet, or they may be provided recorded on a computer-readable non-temporary recording medium such as a DVD-ROM. Each processor may be, for example, a CPU, an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit).

[0092] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more of the above devices.

[0093] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmission unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.

[0094] For example, the information processing device 10 in one embodiment of the present disclosure may function as a computer that performs the processing of the present disclosure.

[0095] Each aspect or embodiment described in this disclosure may be applied to at least one of the following: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA®, GSM®, CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi®), IEEE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0096] The processing procedures, sequences, flowcharts, etc., of each aspect or embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements in an exemplary order and are not limited to the specific order presented.

[0097] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be sent to other devices.

[0098] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, by comparing with a predetermined value).

[0099] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0100] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name. Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technologies (such as infrared or microwave), at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0101] The information, signals, etc., described herein may be represented using any of the following different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., which may be referred to throughout the above description, may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof. Terms used herein and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meaning.

[0102] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0103] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0104] Any reference to elements using the designations “First,” “Second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the First and Second elements do not imply that only two elements may be employed, or that the First element must precede the Second element in any way.

[0105] In the above-described configuration of each device, the term "part" may be replaced with "means," "circuit," "device," etc.

[0106] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0107] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0108] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0109] 1...Information processing system, 10...Information processing device, 20...User terminal, 30...Management server, 40...Store terminal, 9...Network, 11...Acquisition unit, 12...Output unit, 13...Calculation unit, 14...Clustering unit, 15...Estimation unit, 16...Evaluation unit, 191...Storage unit, 192...Control unit, 101...Processor, 102...Memory, 103...Storage, 104...Communication device, 1000...Management server database, 2000...ATT distribution, A...Area, G...Group, M...Store, U...User

Claims

1. An information processing device having an acquisition unit that acquires the effect of the measure on each individual user belonging to the first user group, calculated from a comparison of the performance of the first user group that used the first store where the measure was implemented with the performance of the control group in the measure; and an output unit that outputs the result of clustering the first user group into a group that was exposed to the measure and a group that was not exposed to the measure, based on the effect of each individual user.

2. The information processing apparatus according to claim 1, further comprising a calculation unit that calculates the effect of the measure on the first user group based on a comparison of the performance of one user belonging to the first user group at the first store and the performance of one user belonging to the control group.

3. The information processing apparatus according to claim 1, further comprising a clustering unit that clusters the first user group into a group of people who have been exposed to the measure and a group of people who have not been exposed to the measure based on the distribution of ATT that shows the effect on individual users.

4. The information processing apparatus according to claim 3, wherein the measure includes an advertisement relating to the first store, and the apparatus has an estimation unit that estimates the proportion of the first user group that has been exposed to the measure based on the clustering results.

5. The information processing apparatus according to claim 4, further comprising an evaluation unit that evaluates the impact of the measures implemented at the first store based on the proportion of the group of people exposed to the measures estimated by the estimation unit.

6. The information processing apparatus according to claim 5, wherein the evaluation unit evaluates the impact of the measures using Fuzzy-DID.

7. The information processing device according to claim 1, wherein the aforementioned performance is performance related to settlement amounts.

8. The information processing apparatus according to claim 1, wherein the control group includes a second group of users who used the second store where the measures were not implemented, and the apparatus has a calculation unit that calculates the effect of the measures on the first group of users from a comparison of the performance of one user belonging to the first group of users at the first store and the performance of one user belonging to the second group of users at the second store.

9. An information processing method comprising the steps of: obtaining the effect of the measure on individual users belonging to the first user group, calculated by comparing the performance of the first user group that used the first store where the measure was implemented with the performance of the control group in the measure; and outputting the result of clustering the first user group into a group that was exposed to the measure and a group that was not exposed to the measure, based on the effect of the individual users.

Images