Information processing device, information processing method, and program

Abstract

This information processing method is for calculating, in consideration of a prescribed event or situation, information for assisting in formulating measures to appeal to a prescribed person for effective actions or behaviors, or targets of same, and comprises: a step in which an input unit inputs the actions or behaviors, or the targets of same; a step in which the input unit inputs information pertaining to a measure that has been implemented when the measure has been implemented at a plurality of time points on the basis of the time point of the prescribed event or situation; a step in which the input unit inputs the content of reverberation of the implemented measure; a step in which a reverberation transition calculation unit calculates a reverberation transition of the content of reverberation of the measure; and a step in which an influence level calculation unit extracts and / or sets, on the basis of the reverberation transition, a feature amount of the information pertaining to the event or situation and the measure through machine-learning, and calculates the level of influence of the feature amount on the content of reverberation.

Description

Information Processing Apparatus, Information Processing Method, and Program

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program that can be applied to the consideration or formulation of measures for a target person.

[0002] A direct-mail creation support device that proposes optimal conditions for direct mail has been proposed (see Patent Document 1).

[0003] Japanese Unexamined Patent Application Publication No. 2021-184212

[0004] An object of the present invention is to provide an information processing apparatus, an information processing method, and a program that can output information useful for considering or formulating effective measures.

[0005] 1. Information Processing Method An information processing method for calculating information for supporting the formulation of measures for appealing to an effective action or behavior for a predetermined person, or the target of the action or behavior, in consideration of a predetermined event or situation, including: a step of inputting an action or behavior, or the target of the action or behavior, by an input unit; a step of inputting, by the input unit, information on the implemented measures when the measures are implemented at a plurality of time points based on the time point of the predetermined event or situation; a step of inputting, by the input unit, the response content of the implemented measures; a step of calculating, by a response transition calculation unit, the response transition of the response content of the measures; and a step of calculating, by an influence degree calculation unit, at least one of extracting and setting feature quantities of information on the event or situation and the measures by machine learning based on the response transition, and calculating the degree of influence of the feature quantities on the response content.

[0006] The information processing method of the present invention may include a step of estimating the response rate when measures are implemented for a predetermined period from the occurrence time point of the same or the same type of event as the event, based on the degree of influence on the response rate.

[0007] In the above invention, the information processing method may include a step of inputting or deriving the relationships between feature quantities. The information processing device may include an input unit or a derivation unit for inputting the relationships between feature quantities. When calculating the degree to which the feature quantities have an influence on the feedback content, a more accurate degree of influence can be calculated.

[0008] The information processing method of the present invention may include a step of estimating the response rate when a measure is implemented at a predetermined time, based on a correspondence table that associates the feature quantity, the degree of influence on the response rate, and the time from the occurrence of an event identical or of the same type as the reference event with the amount of change in the response rate.

[0009] The present invention provides an information processing method for calculating information to support the formulation of a strategy to appeal to a predetermined person to an effective action or behavior, or to the target of an action or behavior, taking into consideration a predetermined event or situation, and includes the steps of: inputting an action or behavior, or the target of an action or behavior; inputting information about the strategy implemented when the strategy is implemented at multiple points in time based on the time of the predetermined event or situation; inputting the content of the response to the implemented strategy; calculating the response transition of the corresponding response content when each of the strategies is implemented by a response transition calculation unit; and calculating the degree of influence of the feature quantity on the response content using a mathematical model in which the feature quantity is a variable, obtained by extracting and setting at least one of the information about the event or situation and the strategy created by machine learning, taking into consideration the information about the strategy, based on the response transition.

[0010] The present invention provides an information processing method for calculating information to support the formulation of a strategy to effectively appeal to a predetermined person or target of an action or action, taking into consideration a predetermined event or situation, and includes the steps of: inputting an action or action or target of an action or action into an input unit; inputting information about the strategy implemented when the strategy is implemented at multiple points in time based on the time of the predetermined event or situation; inputting the content of the response to the implemented strategy into an input unit; calculating the response transition of the corresponding response content when each of the strategies is implemented into an response transition calculation unit; and calculating the degree of influence of the feature quantity on the response content using a mathematical model in which the feature quantity extracted or set from the event or situation and the strategy information created by machine learning is used as a variable, taking into consideration the information about the strategy based on the response transition.

[0011] 2. Information Processing Device The information processing device of the present invention is an information processing device for calculating information to support the formulation of a strategy to appeal to a predetermined person to an effective action or behavior, or the target of an action or behavior, taking into consideration a predetermined event or situation, and may include: an input unit for inputting an action or behavior, or the target of an action or behavior; an input unit for inputting information about the strategy implemented when the strategy is implemented at multiple points in time based on the time of the predetermined event or situation; an input unit for inputting the content of the response to the implemented strategy; a response transition calculation unit for calculating the response transition of the content of the response to the strategy; and an influence degree calculation unit that, based on the response transition, extracts and sets at least one of the feature quantities related to the event or situation and the strategy using machine learning, and calculates the degree to which the feature quantities have an influence on the content of the response.

[0012] The present invention is an information processing device for calculating information to support the formulation of a strategy to appeal to a predetermined person to an effective action or behavior, or to the target of an action or behavior, taking into consideration a predetermined event or situation, and may include: an input unit for inputting an action or behavior, or the target of an action or behavior; an input unit for inputting information about the strategy implemented when the strategy is implemented at multiple points in time based on the time of the predetermined event or situation; an input unit for inputting the content of the response to the implemented strategy; a response transition calculation unit for calculating the response transition of the corresponding response content when each of the strategies is implemented; and an influence degree calculation unit that calculates the degree to which the feature has an influence on the response content, using a mathematical model in which the feature is a variable obtained by extracting and setting at least one of the information about the event or situation and the strategy created by machine learning, taking into consideration the information about the strategy, based on the response transition.

[0013] The present invention is an information processing device for calculating information to support the formulation of a strategy to appeal to a predetermined person to an effective action or behavior, or to the target of an action or behavior, taking into consideration a predetermined event or situation, and may include: an input unit for inputting an action or behavior, or the target of an action or behavior; an input unit for inputting information about the strategy implemented when the strategy is implemented at multiple points in time based on the time of the predetermined event or situation; an input unit for inputting the content of the response to the implemented strategy; a response transition calculation unit for calculating the response transition of the corresponding response content when each of the strategies is implemented; and an influence degree calculation unit that calculates the degree to which the feature quantities have an influence on the response content, using a mathematical model in which the feature quantities extracted or set from the event or situation and the strategy information created by machine learning are variables, taking into consideration the information about the strategy based on the response transition.

[0014] 3. Program The program of the present invention is a program for causing a computer to execute an information processing method for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person or target of an action or behavior, taking into consideration a predetermined event or situation, and the program can be configured to cause the computer to execute the following steps: an input unit inputting an action or behavior, or target of an action or behavior; an input unit inputting information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; an input unit inputting the content of the reactions to the implemented measures; a reaction transition calculation unit calculating the reaction transition of the content of the reactions to the measures; and an influence degree calculation unit, based on the reaction transition, extracting and setting feature quantities related to the event or situation and the measures using machine learning, and calculating the degree of influence of the feature quantities on the content of the reactions.

[0015] The present invention is a program for causing a computer to execute an information processing method for calculating information to support the formulation of effective actions or behaviors, or strategies for appealing to a predetermined person or target of an action or behavior, taking into consideration a predetermined event or situation, and the program can be configured to cause the computer to execute the following steps: an input unit inputting an action or behavior, or target of an action or behavior; an input unit inputting information about the implemented strategy when the strategy is implemented at multiple points in time based on the point in time of the predetermined event or situation; an input unit inputting the content of the response to the implemented strategy; a response transition calculation unit calculating the response transition of each corresponding response content when each strategy is implemented; and an influence degree calculation unit calculating the degree of influence of the feature quantities on the response content based on the response transition, taking into consideration the information about the strategy, using a mathematical model in which the feature quantities obtained by extracting and setting at least one of the information about the event or situation and the strategy created by machine learning are variables.

[0016] The present invention is a program for causing a computer to execute an information processing method for calculating information to support the formulation of effective actions or behaviors, or strategies for appealing to a predetermined person or target of an action or behavior, taking into consideration a predetermined event or situation, and the program can be configured to cause the computer to execute the following steps: an input unit inputting an action or behavior, or target of an action or behavior; an input unit inputting information about the implemented strategy when the strategy is implemented at multiple points in time based on the point in time of the predetermined event or situation; an input unit inputting the content of the response to the implemented strategy; a response transition calculation unit calculating the response transition of each corresponding response content when each strategy is implemented; and an influence degree calculation unit calculating the degree of influence of the feature quantities on the response content based on the response transition, taking into consideration the information about the strategy, using a mathematical model in which the feature quantities extracted or set from the information about the event or situation and the strategy created by machine learning are variables.

[0017] Multiple points in time, based on the point in time of a specified event or situation, shall include points in time before and after the point in time of the specified event or situation.

[0018] Actions or acts are a concept that includes all actions or acts, such as participation, purchasing, transferring or lending, borrowing, using, watching or listening, providing services, receiving services, and movement. The objects of actions or acts include, for example, objects to be transferred or acquired (including purchased objects), objects to be used, objects to be lent, objects for which consideration is paid, and objects from which a designated person receives some kind of service. More specifically, it is a concept that includes goods and services, real estate, events and gatherings and any kind of meeting, and objects used in the provision of services or objects made for the purpose of providing services.

[0019] The degree of impact of the reactions to the referenced events or situations may also be calculated. For example, the degree of impact can be shown in the form of, "Regarding legal amendment A, feature X had this much impact."

[0020] The degree of impact of the reactions to the content of the implementation measures may also be calculated. For example, the degree of impact can be shown in the form of, "Measure A had this much impact on feature Y."

[0021] In this specification, machine learning is a concept that includes deep learning.

[0022] According to the present invention, it is possible to realize an information processing device, an information processing method, and a program that can output information useful for considering or formulating effective measures.

[0023] This shows a conceptual diagram of information processing according to the embodiment. This shows the information processing flow according to the embodiment. This shows an example of the relationship between each point in time to grasp the response rate in relation to the implementation of the measures and to obtain the response trend. This is an example of displaying the contribution of features. This shows a screen that shows information on the implementation of the measures, an example of output for estimating influencing factors, and an example of displaying improvement suggestions. This is an image diagram of the model interpretation technology. This shows an example of an information processing device. This shows an example of an information management screen. This shows an example of an analysis screen. This is an example of a diagram that organizes candidate features and the relationships between each feature in a tree format when deciding "what features to include" in machine learning. This is an enlarged view of the display example of the output of influencing factor estimation (calculation of the degree of influence for each feature) from Figure 5.

[0024] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0025] The basic concept of the information processing method according to this embodiment supports the realization of the following steps 1 to 4, for example: Step 1: Data formatting and input Step 2: Model creation using machine learning (feature extraction, etc.) Step 3: Calculation of the degree of influence of each feature through model interpretation Step 4: Recommendation of the optimal DM strategy

[0026] "At least one step of feature extraction and setting" is one of the tasks in Step 1. By decomposing information such as the project creative list described later into elements (i.e., features) and applying machine learning, a mathematical model can be created with the features as variables. After reviewing the results of Steps 2 and 3, one can return to Step 1 to create a more accurate model. This process of reviewing the results of Steps 2 and 3 and returning to Step 1 can be repeated. The contribution of a feature is a concept that includes both a coefficient (so-called mechanical contribution) and a visualization that is easy for humans to understand (so-called human-oriented contribution such as SHAP values). The model formula is a concept that includes a first model formula shown in terms of features and their contributions, and a second model formula that uses the first model formula. The second model formula is calculated by returning to Step 1, reviewing the features, and repeatedly inputting and processing to improve the accuracy of the model formula. This is the execution of machine learning for model interpretation to make it visualized and easy to understand. By making it visible, improvement measures can be considered using steps of estimation and deduction.

[0027] The method for creating machine learning models can be appropriately selected depending on the data and field. In machine learning, for example, there are three ways to use it. The first way to use it is to extract features and set which features to include. The second way to use it is to machine learn from the input data, calculate the contribution, and create a mathematical model (3a + 4d + ...) that can be understood by the machine using the model method. The third way to use it is to machine learn (visualize) so that it can be interpreted by humans. When selecting which features to include, the candidate features shown in Figure 10 and the relationships between each feature can be organized in a tree format to select the features.

[0028] In this specification, "DM" is used as an example of "strategy" and "appeal strategy." The term "DM" is not limited to "DM," but can be interpreted as being broadly applicable to "strategy" and "appeal strategy." Therefore, the use of "DM" does not exclude appeal strategies that are not DM; rather, "DM" is used as an example, and descriptions replacing "DM" with "strategy" and "appeal strategy" are also within the scope of disclosure in this specification.

[0029] As shown in Figure 1, the information processing method according to this embodiment takes information about an event or situation, a sales strategy, and the response to the sales strategy as input, extracts and sets features using machine learning, calculates the degree of influence of the features, and can then calculate an inference model or evaluation model based on that degree of influence.

[0030] As shown in Figure 2, one concrete example of the concept of an information processing method is presented. This information processing method is an example of a series of processing flows and may include a step S1 for inputting information on an event or situation, a step S2 for inputting information on each appeal, a step S3 for inputting the response rate for each appeal, a step S4 for calculating the trend of the response rate, a step S5 for extracting or setting features using machine learning, a step S6 for calculating a model equation with the features as variables, a step S7 for calculating the contribution of each feature to the response rate, and a step S8 for displaying the contribution of each feature. Machine learning can be performed in each of the feature extraction and / or setting step S5 and the model equation calculation step S6. Machine learning can also be performed in the feature contribution calculation step S7 and the feature contribution display step S8.

[0031] Figure 3 shows an example of the relationship between different time points for understanding the response rate and obtaining the response trend in relation to the implementation of a strategy. Based on the time point at which the event or situation was understood, response rates a, b, and c are obtained when appeal strategies A, B, and C are implemented, respectively. The time points at which the response rate is understood from the date of implementation of the strategy can be arbitrarily determined. Multiple response rates may be understood for a single appeal strategy at different time points.

[0032] The information processing method according to the embodiment may be used to calculate information to support the formulation of effective actions or behaviors, or strategies to appeal to a specific person, taking into consideration a predetermined event or situation. The information processing method may include a step in which an input unit inputs an action or behavior, or the target of an action or behavior; a step in which the input unit inputs information about the implemented strategy when the strategy is implemented at multiple points in time based on the point in time of the predetermined event or situation; a step in which the input unit inputs the content of the response to the implemented strategy; a step in which a response transition calculation unit calculates the response transition of the content of the response to the strategy; and a step in which an influence degree calculation unit extracts and sets feature quantities related to the event or situation and the strategy using machine learning based on the response transition, and calculates the degree of influence of the feature quantities on the content of the response. The information processing method may include a step in which relationships between feature quantities are input or derived. Relationships between feature quantities include, for example, constraints, increases and decreases, denominator effects, and numerator effects between feature quantities, as shown in Figure 10. It is possible to derive or set the following relationships: budget and number of shipments are constraints, shipping method and shipping cost are inversely related, number of shipments and CVR are inversely related, and cost and CPA (Cost per Acquisition) are inversely related. By including a process of inputting or deriving the relationships between features, it is possible to calculate a more accurate degree of influence of the features on the response content, and this is also useful when calculating the model equation.

[0033] The impact calculation unit can calculate the degree to which features influence the response content based on the response progression, taking into account information about the measures, and using a mathematical model in which features extracted and set from information about events or situations and measures created by machine learning are used as variables. The impact calculation unit may also calculate the degree to which features influence the response content based on the response progression, taking into account information about the measures, and using a mathematical model in which features extracted or set from information about events or situations and measures created by machine learning are used as variables.

[0034] (1) Information on events or circumstances to be referenced. Information on events or circumstances to be referenced includes timing, client (=DM issuer), customer information (client's customers), competitor information, economic trends, social trends, political trends, and technological trends. Examples of timing include seasons, industry events, and industry peak periods.

[0035] As for the client (=DM issuer), examples include industry, market share, target audience, price, number of employees, sales volume, sales growth rate, and capital relationships. As for customer information, examples include industry, number of employees, sales volume, sales growth rate, and capital relationships. As for competitor information, examples include market share, target audience, price, frequency of DM distribution, and content / design of DM. As for economic trends, examples include industry growth rate, corporate investment intentions, economic fluctuations, inflation / deflation situation, and exchange rates. As for social trends, examples include changes in consumer purchasing behavior and differences in values ​​between generations. As for political trends, examples include changes in laws and regulations, tax reforms, changes in trade relations, government support for the industry, and government subsidies. As for technological trends, examples include the progress of DX, data analysis technology, and AI / automation technology.

[0036] An event or situation can be a concept that includes external factors that affect products, services, or direct mail campaigns, such as legal revisions or industry events. The implementation of the promotional strategy itself can also be considered an event or situation.

[0037] Information on a company's willingness to invest can be considered not only in terms of numerical data such as "the target company's total annual investment amount," but also in terms of non-numerical data such as "references to the relevant sector in financial statements."

[0038] (2) Information strategies regarding strategies refer to taking some measures to make them known to the target audience, and are a concept that includes appeal strategies, with direct mail (DM) being a more specific example.

[0039] Information on the strategy includes at least one selected from the group consisting of the content of the appeal, conversion (results), conversion path, benefits, costs from the production to the sending of the appeal strategy, the implementation date (sending date) of the appeal strategy, the number of implementations (number of sent) of the appeal strategy, the specifications of the strategy, the content of the strategy (published content), information on the design of the strategy, and the list of destinations. The appeal strategy refers to, for example, an information transmission strategy that can be perceived by the target person, and includes direct mail and sales promotion materials with printing, and concepts including mail.

[0040] Information on the strategy will be explained by taking projects, creatives, and lists as examples.

[0041] The definitions of project, creative, and list managed and analyzed in "Management and Analysis of DM (Direct Mail) Data and Result Data" are as follows.

[0042] A project is defined as information on measures other than creative lists. A project can include, for example, the content of the appeal (such as a guide to a business seminar or an introduction to a new service), conversion (such as an inquiry or a service application), conversion path (such as a website, email, or fax), offer or benefit (such as three months free or a sample gift), cost (such as the cost from the production to the sending of the DM), sending date (such as the date when the DM was sent), and number of sent (such as the number of DMs sent).

[0043] A creative refers to information on the specifications, published content, and design of the mailed item. Examples of the specifications of a creative include the type of material (such as an envelope, postcard, OPP bag (transparent envelope)), quantity (such as three items enclosed or one item enclosed in an envelope), size / shape (such as A4, B5, square, die-cut, etc.). Examples of the published content include composition, content, amount of text, etc. Examples of the design include color, font, motif, etc.

[0044] Examples of a list include information on the destination (such as area, industry, name of the addressee, etc.).

[0045] Priorities may be set relatively among various pieces of information. Information before and after implementing a marketing strategy may be input. Specifically, it may be possible to compare the response rates of the strategies before and after implementing the marketing strategy. Qualitative information can be indexed by keywords.

[0046] Fluctuations in consumer purchase behavior can be examined based on, for example, data on the number of searches for keywords over a predetermined period (e.g., monthly).

[0047] (3) Response content As the response content, for example, the conversion rate (CVR) can be cited. The conversion rate (CVR) can be defined as the number of conversions (CVs) per number of sent items in each project (DM sending). The conversion rate (CVR) is a value in percentage (%) calculated based on Equation 1.

[0048] (Equation 1) Number of conversions ÷ Number of sent items = CVR

[0049] (4) Feature quantity Feature quantities can be derived by various analysis methods and can be extracted by known feature quantity extraction methods. When extracting feature quantities, a process of quantifying the feature quantities is implemented as necessary so that they can be incorporated into machine learning. The process of quantification may be set or calculated using software.

[0050] (5) Degree of Contribution As an example of the degree of contribution, the SHAP value can be cited. The SHAP value indicates the degree of contribution of each feature to the predicted value. An example of the display is shown in Figure 4. The vertical axis is a list of features, and the horizontal axis shows the distribution of the influence value of each feature in the input data, which in this example is the SHAP value. The content of each policy can be shown in different colors. As features, in the example in Figure 4, from top to bottom, they are "cost", "number_of_shipment", "visual_type", "main_copy_number_of_char", "package", "count", "size", "all_number_of_cha", "offer", "include_personal_name", "address", "tool_count", "base_color", "content", "area", "data_source", "did_tel_marketing", and "shipment_method". For example, when calculating a machine learning model based on data from 10 direct marketing (DM) campaigns, the SHAP value can be plotted as red and blue circles. For instance, DM Campaign 1 might have a SHAP value of 100, DM Campaign 2 might have a SHAP value of -90, ... DM Campaign 10 might have a SHAP value of 30. This allows for a graph that visualizes the magnitude of influence for each DM campaign.

[0051] 3. Hardware Configuration Diagram 7 shows an example of the hardware configuration of the information processing device. The above information processing method can be executed by the information processing device.

[0052] The information processing device 10 according to this embodiment can be implemented by one or more computers. The input device 20 can be implemented using known input means such as a keyboard, mouse, or touch input device, and is a concept that includes a data acquisition unit that acquires information from hardware resources (such as a server) connected to a communication network (including an internet connection). The input device 20 may include an event or situation input unit 20a, a policy input unit 20b, an echo input unit 20c for inputting echo content, a relevance input unit 20d for setting the degree of relevance of information, and a feature quantity setting unit 20e for setting feature quantities. The input device 20 can be configured by one or more means having input functions.

[0053] Functionally, the processing unit 30 may include a correlation calculation unit 30a that calculates the degree of correlation of information, a response transition calculation unit 30b that calculates the response transition from multiple response contents, a feature extraction unit 30c that extracts feature quantities, a contribution calculation unit 30d that calculates the contribution of feature quantities, a model formula calculation unit 30e that calculates a model formula from the contribution of feature quantities, a response result prediction unit 30f that predicts response results from policies and various information, and an effect verification unit 30g that verifies the effects from the implementation details of policies and response results, and verifies the model formula. The processing unit 30 may also include an influence calculation unit 30h. These processes may be implemented by a single arithmetic unit or by multiple arithmetic units. The processes may also be implemented by multiple computers. The processing unit 30 can be implemented by an arithmetic unit such as a CPU. The information processing device 10 may have a storage device 50 that stores various input data and calculated data. The storage device 50 can be stored in known storage devices such as ROM, hard disks, or external storage devices (CDs, DVDs, etc.). The storage device 50 may be configured separately from the arithmetic unit, or it may be configured within the arithmetic unit. The program that causes the information processing device 10 to perform information processing can be stored in a storage device (for example, ROM, hard disk) included in the information processing device 10. As an example of the processing of the response result prediction unit 30f, it is possible to perform information processing such as calculating a predicted CVR of 0.5% in the response result prediction unit 30f, finding that the feature with a high contribution is "reward," and therefore changing the content of "reward" and predicting again with the response result prediction unit 30f, resulting in a predicted CVR of 0.8%. The information processing device may include an input unit or a derivation unit for inputting the relationships between feature quantities.

[0054] The information processing device 10 may include a display device 40 that displays various processing results, such as the degree of contribution of feature quantities. The display device 40 can be a known display such as a liquid crystal display or an organic EL display.

[0055] The information processing device 10 may consist of an input device 20, a processing device 30, a display device 40, and a storage device 50, all of which are part of a single piece of hardware, or they may consist of multiple pieces of hardware in a communication-enabled manner. The information processing device 10 may consist of a single computer or multiple computers. 4. Effects and Examples of Use The effects and examples of use of the information processing method according to this embodiment will be described below.

[0056] (1) According to this embodiment, for example, it is possible to estimate the strength of the influence of the information on the effectiveness of direct mail, and to use it for analyzing and predicting the response results of direct mail.

[0057] This embodiment, as an example, serves the following purpose.

[0058] 1) Management and analysis of direct mail data and results data 2) Proposal of improvements to direct mail measures using machine learning models This embodiment has the following effects.

[0059] When implementing a direct mail (DM) campaign, it's often unclear which target audience to send to, when to send it, what content to include, and what type of DM to use for maximum effectiveness. This information processing method allows for the detection of characteristics of highly effective DMs by managing and analyzing information related to past DM campaign projects and creative lists.

[0060] After implementing a direct mail (DM) campaign, it's often difficult to accurately analyze the reasons for high or low effectiveness during the effectiveness evaluation, making it challenging to formulate effective improvement plans. By managing and analyzing information related to past DM campaign projects, creatives, and lists, we can detect the reasons for high or low effectiveness.

[0061] (2) As shown in Figure 5, it is possible to propose improvements to direct mail (DM) strategies using machine learning models. When implementing DM strategies, it is not possible to determine which target audience to send what content to, at what time, and with what type of DM to maximize effectiveness. By performing machine learning and using model interpretation techniques based on the following considerations, it is possible to identify features (≒ elements) that have a large / small impact on DM strategies.

[0062] (Information to consider) • Project information, creative information, list information, and other internal information: sales / number of employees of companies to which direct mail is sent, etc. • Other external information: stock prices, legal changes, weather, etc.

[0063] (3) By constructing an inference model using extracted or set features, for example, the following can be done: 1) Before implementing a measure, by inputting information about the measure, it is possible to calculate "predicted number of conversions," "predicted conversion rate," and "predicted cost per acquisition (CPA)." 2) Before implementing a measure, by inputting information about the measure, it is possible to provide improvement suggestions and the expected improvement range based on those suggestions. 3) After implementing a measure, by inputting information about the measure and the CVR, it is possible to analyze the cause of the discrepancy between the "predicted number of conversions / CVR / CPA" and the actual results, and to improve the machine learning model. 4) It is useful in recommending the optimal DM strategy. It contributes to recommendations. 5) Since the contribution of features can be calculated, it can play a role as a method for calculating model formulas, methods for estimating effects, and methods for calculating support data for formulating measures to improve effects. 6) It can be used to manage and analyze data and outcome data of appeal measures (e.g., DM). It is possible to make suggestions for improving appeal measures using machine learning models. By applying machine learning to information, it is possible to estimate the strength of the influence of that information on the effectiveness of the appeal measure. Furthermore, it allows for the analysis and prediction of the response to promotional strategies.

[0064] (4) As shown in Figure 6, by inputting data into a machine learning model, it is possible to learn and output complex relationships that humans cannot grasp. The overall process involves inputting data into a machine learning model, identifying the basis for the model's predictions and judgments using model interpretation technology, updating the machine learning model, and improving the description of relationships, for example, improving a specific level of contribution. It is possible to make suggestions for improving DM measures using the machine learning model. In other words, as shown in Figure 5, 1) it becomes easy to estimate the factors that affect the effectiveness of DM using a machine learning model that has learned from the information on DM measures managed by this system, and 2) it becomes easy to automatically suggest areas for improvement in the effectiveness of new DM measures based on the estimated influencing factors and their degree of impact.

[0065] 5. Specific Examples: The number of internet searches or corporate investment intentions that can be obtained are examples of events or situations. It is also possible to estimate the relationship between the number of internet searches that can be obtained and the response rate, or the relationship between corporate investment intentions and the response rate.

[0066] (1) The first specific example is given below.

[0067] (Premise) This example uses a direct mail (DM) promotion campaign for a human resources management tool, conducted monthly from April to June 2024, as a review and analysis. (Results of DM promotion) The content and effects of the DM promotion campaign promoting product "X" are assumed to be as follows: <DM sent in April 2024> Number sent: 10,000 Number of conversions (responses): 10 CVR (response rate): 0.1% <DM sent in May 2024> Number sent: 10,000 Number of conversions (responses): 40 CVR (response rate): 0.4% <DM sent in June 2024> Number sent: 10,000 Number of conversions (responses): 120 CVR (response rate): 1.2% (Factor analysis) When analyzing the factors that led to the results of the above DM promotion campaign, if we investigate the "number of internet searches for related keywords," which can be considered one of the factors, the analysis results can be shown as follows.

[0068] Assuming that a search server on the internet retrieves keyword search information and finds that the number of internet searches for the related keyword "personnel management" during the period was as follows: • April 2024: 10,000 searches / month • May 2024: 30,000 searches / month • June 2024: 60,000 searches / month Based on these results, and taking into account the influence of other factors, machine learning analysis calculates that the influence of "number of internet searches" on the CVR (conversion rate) is ●●●. The nature of the influence will vary depending on the program used to calculate the influence. In this way, the influence can be calculated.

[0069] (2) A second specific example The information processing may be as follows: Assume that in a direct mail campaign promoting "product X," it is known that there is a high correlation with legal amendment A (implementation date: mid-July), and that the number of internet searches for related keywords was: April: 10,000 times, May: 30,000 times, June: 60,000 times. Assume that the direct mail sent in April, with 10,000 copies, resulted in 10 conversions (response rate 0.1%). The direct mail sent in May, with 10,000 copies, resulted in 40 conversions (response rate 0.4%). The direct mail sent in June, with 10,000 copies, resulted in 120 conversions (response rate 1.2%). Based on these results, legal amendment A has a certain degree of impact on the response rate of direct mail for product X, and the response rate changes by △△% as the implementation date approaches. On the other hand, the output can be that internet search volume has a certain degree of influence on □□□.

[0070] The term "degree of impact" can have several different meanings depending on the "event" being input. For example, in the case of a legal amendment, the degree of impact would be calculated in terms of a causal relationship, such as "the implementation of this legal amendment increases the need for introducing this service, and thus its introduction progresses." In the case of internet search volume, the degree of impact would be calculated in terms of a correlation, such as "sending direct mail during periods when the number of searches for the related keyword 'human resources management' is high tends to result in a higher conversion rate (CVR)."

[0071] From the first and second specific examples, it is possible to calculate the degree of influence using internet search volume and CVR as input information. It is also possible to calculate the degree of influence such as, "Sending direct mail during periods of high search volume tends to increase the CVR (response rate) of that direct mail."

[0072] (3) A third specific example: This can also be used to determine the extent to which the response rate of each direct mail campaign was affected by various legal revisions and the dates of those revisions.

[0073] The following is an example of the specifications when a legal amendment is an event. Step 1 Set the degree of relevance between the "products / services to be promoted in the DM" and the "legal amendment" in question for each case (e.g., very high / high / medium / low / very low) Step 2 Classify the "products / services to be promoted in the DM" and the various "legal amendments" for each case (e.g., [products / services: medical / construction / IT...], [legal amendments: laws / government ordinances / ministerial ordinances / regulations / written laws]) Step 3 Group each category and degree of relevance, and calculate the response trend by aggregating the "interval between the mailing date and the amendment date" and the "response rate for each case" in each group. Step 4 For the calculated response trend, use machine learning to improve the accuracy of the prediction model by taking into account the influence (degree of contribution) of other features. As a grouping method, "amendments to the Labor Standards Act" can be divided into five categories: "1. Laws related to the Labor Standards Act / 2. Cabinet Orders / 3. Ministerial Ordinances / 4. Regulations / 5. Written Laws and Regulations," and a degree of relevance can be assigned to each. For example, in relation to goods or services, "1. Laws related to the Labor Standards Act" could be considered to have a "very high" degree of relevance, while "4. Regulations related to the Labor Standards Act" could be considered to have a "low" degree of relevance.

[0074] In the case of legal revisions, it is possible to calculate the degree of impact in terms of causal relationships, such as "the implementation of this legal revision will increase the need for introducing this service, and thus promote its adoption." In the case of internet search volume, it is also possible to calculate the degree of impact in terms of correlations, such as "sending direct mail during periods when the number of searches for the related keyword 'human resources management' is high tends to result in a higher conversion rate (CVR)."

[0075] Legal amendment A has an impact of XX, and the response rate will change by □□% as the implementation date approaches. On the other hand, it may also be possible to output that internet search volume has an impact of △△. It is also possible to output that "In a project (DM campaign) with a response rate of 0.3%, legal amendment A has an impact of ●●●." Changes in laws and regulations can be referenced from the government's legal amendment and regulatory information database, and the response rate can be estimated by aggregating response trends from past data for the period before and after the legal amendment.

[0076] "Aggregating the trends in response rates before and after legal amendments" can be illustrated with the following example: By aggregating data such as the response rates of direct mail sent one year, six months, three months, one month, and one month after legal amendment A; the response rates of direct mail sent one year, six months, three months, one month, one month, and three months after legal amendment B; and the response rates of direct mail sent one year, six months, three months, and three months after legal amendment C, it is possible to quantitatively present predictions such as the conversion rate (CVR) increasing by approximately +10% month-on-month until three months before the legal amendment, then surging by +80% month-on-month from three months before to the amendment date, and then declining sharply after the amendment date. This can function as a method to identify "how much the response rate of each direct mail campaign was affected by various legal amendments and the dates of those amendments." A model of response trends can be constructed.

[0077] 6. Examples of Management and Analysis Screens DM data and performance data may be managed and analyzed on the management screen (see Figure 8) and analysis screen (see Figure 9). Information and effects (results) of past DM campaigns may be registered and managed. Based on the managed data, aggregation and analysis may be performed by campaign attributes (project, creative, list) and time series. Based on the aggregation and analysis, future DM campaigns may be refined.

[0078] 7. Regarding the period related to the response trend, the following are examples of how to consider the period assumed when deriving the response trend: (1) The period leading up to the deadline (e.g., the implementation date) (2) The period elapsed from the start date (e.g., the product release date) (3) The period before and after a specified period (e.g., the industry peak period) (4) The period from the initial implementation date of the measure to the most recent implementation date

[0079] The periods (1) to (3) above are set assuming that a specific event will occur. Period (1) is specifically the period applied when considering the trend of responses to measures taken before the deadline. Period (2) has a fixed start date and is applied when considering the trend of responses to measures taken after that start date. Period (3) is a period during which an event occurs and is applied when considering the trend of responses to measures taken before and after that event. Period (4) is a period during which no specific event occurs, for example, when you want to look at a measure purely for six months or a year. In the case of period settings (1) to (3), the degree of influence can be shown in the form of, for example, "Regarding legal amendment A, feature X had this much influence," or "Regarding policy A, feature Y had this much influence." In the case of period setting (4), the degree of influence can be shown in the form of, for example, "Regarding policy A, feature Y had this much influence."

[0080] The period in (1) will be explained using a legal amendment as an example. As a premise, it is assumed that a preparation period of about two years is provided from the date of notification to the date of enforcement, as companies will need to take prescribed measures in accordance with this amendment. When promoting software services to support the response in accordance with this amendment using direct mail, the response is expected to change as follows depending on the date of implementation of the measure: Early period: Within one month from the date of notification (about two years until enforcement): Many responses are obtained from companies that will respond quickly, such as those with a large business impact. Mid-period: About one year from the date of notification (about one year until enforcement): Companies that will respond early have already started, and companies that are late still have about a year until the deadline, so there is not much response. Late period: About two years from the date of notification (just a few days until enforcement): Many responses are obtained due to "last-minute demand" from companies that were late in responding.

[0081] As described above, by inputting and calculating the response trends for each implementation day of a strategy, it is possible to estimate the appropriate implementation date, as well as calculate the appropriate creative and list conditions for each implementation day, and the degree of influence of each feature. By calculating the degree of influence of features in this way, for example, the following evaluations can be obtained: Example 1: In the first half of the year, the conversion rate (CVR) will be increased by narrowing the recipients (list) to listed companies that have a high business impact and require a quick response. Example 2: In the middle of the year, since the number of responses will be low, the strategy will be to reduce the cost per strategy and increase the number of strategies to "fewer missed opportunities within the same budget". Example 3: In the later period, a "rush demand" is expected, and many competing services will also be conducting promotions, so the conversion rate (CVR) will be increased by offering incentives.

[0082] (2) The period will be explained using a new product release as an example. As a premise, the product is novel and meets market needs, but the barriers to entry are not high, and competitors will enter the market one after another. When promoting this service using direct mail, the response is expected to change as follows depending on the date the strategy is implemented. Early period: Within one year from the product release date (no competitors): Many responses are obtained from innovators and early adopters. Mid-period: One to three years after the product release (several competitors enter): The growth of responses gradually slows down as companies compete for the share of the early majority. Late period: More than three years after the product release (many competitors enter): Many competitors are working to acquire the late majority and to get customers to switch brands from other companies, and the growth of responses stagnates.

[0083] As described above, by inputting and calculating the response trends for each day of implementation of a strategy, it is possible to estimate appropriate implementation dates and cost allocations, as well as calculate appropriate creative and list conditions for each implementation date, and the degree of influence of each feature. By calculating the degree of influence of features in this way, for example, the following evaluations can be obtained: Example 1: In the early stages, the conversion rate (CVR) is increased by narrowing the recipients (list) to companies that are highly interested in highly innovative services. Example 2: In the middle stages, the CVR is increased by using creatives that appeal to functional differences, as this is a time to improve the product based on customer feedback and differentiate from competitors. Example 3: In the later stages, the market has matured and the products of each company have become standardized, so the number of responses is increased by appealing to achievements and increasing customer touchpoints by increasing the number of times the strategy is implemented.

[0084] (3) The period will be explained using the peak season of a certain industry as an example. As a premise, the peak season for this industry is December, and the demand for a certain product aimed at this industry tends to increase in December, the peak season. When promoting this service using direct mail, the response is expected to change as follows depending on the date the measure is implemented. Early period: 3 months before the peak season: A lot of response is obtained as a preparation period for the peak season. Mid-period: Peak season: A moderate response is obtained as products that were in short supply during the peak season are replenished. Late period: 3 months after the peak season: The response settles down as the off-season begins.

[0085] As described above, by inputting and calculating the response trends for each day of implementation of a strategy, it is possible to estimate appropriate implementation dates and cost allocations, as well as calculate appropriate creative and list conditions for each implementation date, and the degree of influence of each feature. By calculating the degree of influence of features in this way, for example, the following evaluations can be obtained: Example 1: In the early period, increase the conversion rate by differentiating from competitors through incentives, etc. Example 2: In the middle period, since it is expected that there will be an urgent need to replenish products, use creative that emphasizes the ability to respond quickly. Example 3: In the later period, allocate segments and costs according to the expected number of purchases and total purchase amount for each company, and implement different strategies for each segment.

[0086] Regarding the period in (4), we will explain using a new strategy for an existing product as an example. As a premise, we will assume that two types of strategies (Strategy A and Strategy B) with new appeals will be implemented in a market that already has a certain level of awareness of the product. When implementing Strategy A and Strategy B using direct mail, if each is implemented three times with intervals in between, the response is expected to change as follows depending on the implementation date of the strategy: Early period. 1st implementation date: Many responses were obtained for Strategy A, and a moderate response was obtained for Strategy B. Mid-period. 2nd implementation date (6 months after the 1st implementation date): A moderate response was obtained for Strategy A, and a moderate response was obtained for Strategy B. Late period. 3rd implementation date (1 year after the 1st implementation date): A small number of responses were obtained for Strategy A, and a moderate response was obtained for Strategy B.

[0087] As described above, by inputting and calculating the response trends for each day of implementation of a strategy, it is possible to estimate the appropriate number of times to implement each strategy, as well as calculate the conditions such as creatives and lists that can suppress the decrease in the number of responses for each strategy, and the degree of influence of each feature. Example 1: Strategy A shows a significant decrease in the number of responses with each implementation, so the number of implementations should be limited to 2, and the recipients (list) for the second mailing should be narrowed down based on the characteristics of customers who responded in the first mailing to increase the CVR. Example 2: Strategy B shows only a slight decrease in the number of responses with each implementation, so the number of implementations should be limited to 7, and the number of responses should be increased by only changing the content of the incentive.

[0088] The above embodiments can be modified in various ways within the scope of the gist of the present invention.

[0089] 10 Information processing device 20 Input device 30 Processing device 40 Display device 50 Storage device

Claims

1. An information processing method for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person, taking into consideration a predetermined event or situation, the method comprising: a step of inputting an action or behavior, or a target of an action or behavior; a step of inputting information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; a step of inputting the content of the reactions to the implemented measures; a step of calculating the trend of reactions to the content of the reactions to the measures by a reaction trend calculation unit; and a step of calculating the degree of influence of the feature quantities on the content of the reactions by extracting and setting feature quantities related to the event or situation and the measures using machine learning, based on the trend of reactions.

2. An information processing device for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person, taking into consideration a predetermined event or situation, the information processing device comprising: an input unit for inputting actions or behaviors, or the target of actions or behaviors; an input unit for inputting information about the implemented measures when the measures are implemented at multiple points in time based on the time of the predetermined event or situation; an input unit for inputting the content of the reactions to the implemented measures; a reaction trend calculation unit for calculating the trend of the reaction content of the measures; and an influence degree calculation unit that, based on the reaction trend, extracts and sets at least one of the feature quantities related to the event or situation and the measures using machine learning, and calculates the degree to which the feature quantities have an influence on the reaction content.

3. A program to cause a computer to execute an information processing method for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person or target of an action or behavior, taking into consideration a predetermined event or situation, the program to cause a computer to execute the following steps: an input unit inputting an action or behavior, or target of an action or behavior; an input unit inputting information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; an input unit inputting the content of the reactions to the implemented measures; a reaction trend calculation unit calculating the reaction trend of the content of the reactions to the measures; and an influence degree calculation unit, based on the reaction trend, extracting and setting feature quantities related to the event or situation and the measures using machine learning, and calculating the degree of influence of the feature quantities on the content of the reactions.

4. An information processing method for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person, taking into consideration a predetermined event or situation, the method comprising: a step of inputting an action or behavior, or a target of an action or behavior; a step of inputting information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; a step of inputting the content of the reactions to the implemented measures; a step of calculating the reaction progression of the corresponding reaction content when each of the measures is implemented by a reaction progression calculation unit; and a step of calculating the degree of influence of the feature quantities on the reaction content, based on the reaction progression and taking into consideration information about the measures, using a mathematical model in which feature quantities are variables obtained by extracting and setting at least one of information about the event or situation and the measures created by machine learning.

5. An information processing device for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person, taking into consideration a predetermined event or situation, the information processing device comprising: an input unit for inputting actions or behaviors, or the target of actions or behaviors; an input unit for inputting information about the implemented measures when the measures are implemented at multiple points in time based on the time of the predetermined event or situation; an input unit for inputting the content of the reactions to the implemented measures; a reaction trend calculation unit for calculating the reaction trends of the corresponding reaction content when each of the measures is implemented; and an influence degree calculation unit for calculating the degree to which the features have an influence on the reaction content, using a mathematical model in which features are variables obtained by extracting and setting at least one of the information about the event or situation and the measures created by machine learning, taking into consideration the information about the measures, based on the reaction trends.

6. A program to cause a computer to execute an information processing method for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person or target of an action or behavior, taking into consideration a predetermined event or situation, the program comprising: a step in which an input unit inputs an action or behavior, or target of an action or behavior; a step in which an input unit inputs information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; a step in which an input unit inputs the content of the reactions to the implemented measures; a step in which a reaction transition calculation unit calculates the reaction transition of each corresponding reaction content when each of the measures is implemented; and a step in which an influence degree calculation unit calculates the degree of influence of the feature quantities on the reaction content based on the reaction transitions, taking into consideration the information about the measures, using a mathematical model in which the feature quantities obtained by extracting and setting at least one of the information about the event or situation and the measures created by machine learning as variables.

7. An information processing method for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person, taking into consideration a predetermined event or situation, the method comprising: a step of inputting an action or behavior, or a target of an action or behavior; a step of inputting information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; a step of inputting the content of the reactions to the implemented measures; a step of calculating the reaction progression of the corresponding reaction content when each of the measures is implemented by a reaction progression calculation unit; and a step of calculating the degree of influence of the feature quantities on the reaction content, based on the reaction progression and taking into consideration information about the measures, using a mathematical model in which feature quantities extracted or set from information about the event or situation and the measures created by machine learning are variables.

8. An information processing device for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person, taking into consideration a predetermined event or situation, the information processing device comprising: an input unit for inputting actions or behaviors, or the target of actions or behaviors; an input unit for inputting information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; an input unit for inputting the content of the reactions to the implemented measures; a reaction trend calculation unit for calculating the reaction trends of corresponding reaction content when each of the measures is implemented; and an influence degree calculation unit for calculating the degree to which the features have an influence on the reaction content, using a mathematical model in which features extracted or set from the event or situation and the information about the measures created by machine learning as variables, taking into consideration the information about the measures based on the reaction trends.

9. A program to cause a computer to execute an information processing method for calculating information to support the formulation of effective actions or behaviors, or measures to appeal to a predetermined person or target of an action or behavior, taking into consideration a predetermined event or situation, the program comprising: a step in which an input unit inputs an action or behavior, or target of an action or behavior; a step in which an input unit inputs information about the implemented measures when the measures are implemented at multiple points in time based on the point in time of the predetermined event or situation; a step in which an input unit inputs the content of the reactions to the implemented measures; a step in which a reaction transition calculation unit calculates the reaction transition of each corresponding reaction content when each of the measures is implemented; and a step in which an influence degree calculation unit calculates the degree of influence of the features on the reaction content based on the reaction transition, taking into consideration the information about the measures, using a mathematical model in which features extracted or set from the information about the event or situation and the measures created by machine learning are used as variables.

Images