Message generation device, method, and program

Abstract

In an embodiment of the present invention, learning data including an action target, attribute information, and an intervention condition for a user is acquired, and a first prompt to be input to a message generation model is generated on the basis of the acquired action target, attribute information, and intervention condition. Next, the generation data output from the message generation model in response to the input of the first prompt is separated into a thought portion and a message portion, a second prompt for thought evaluation and a third prompt for message evaluation are generated on the basis of the thought portion and the message portion that have been separated, and the second prompt and the third prompt are input to an evaluation model, so as to acquire a first score representing the degree of appropriateness of the thought portion and a second score representing the degree of appropriateness of the message portion from the evaluation model. A loss function is then calculated on the basis of the first score and the second score, and parameters of the message generation model are updated on the basis of the calculated loss function.

Description

Message generation apparatus, method, and program

[0001] One aspect of this invention relates to a message generation device, method, and program that generate messages that encourage users to change their behavior.

[0002] In recent years, applications that encourage healthy behavior through exercise and other means have become widespread. These types of applications generally encourage users to behave better by presenting behavioral change messages to them, for example, using pop-up notifications.

[0003] However, there are challenges in actually using behavior change messages to encourage users to change their behavior, as follows:

[0004] (1) Message Content: Messages need to be tailored to the user's state, habits, and surrounding environment. For example, on extremely hot days, indoor training should be encouraged instead of walking. Also, telling someone who habitually walks 10,000 steps a day to walk 8,000 steps is pointless.

[0005] (2) Message format The message format needs to take into account user characteristics and preferences, such as present bias and loss aversion. Also, repetition of the same sentence can easily lead to boredom and reduce the effectiveness of the message. For example, for users with a strong loss aversion, it is necessary to convey that they will lose out if they do not take action, such as, "If you don't eat vegetables, you will be deficient in vitamins and are more likely to develop diseases like XX." In addition, lines from characters that users like are more effective than mechanically formatted messages.

[0006] (3) Timing of messages The appropriate timing varies depending on the content of the message. Also, the appropriate message varies depending on the timing. For example, it is meaningless to present a message that says, "Taking caffeine before exercise increases fat burning," after exercise. If a message is presented at a time when the user cannot check it, such as when they are doing something or sleeping, the user will not be able to check it. Also, by the time they check it, it may be too late to take action. Sending messages too frequently can also cause stress to the user.

[0007] In order to address the above problems, for example, Non-Patent Document 1 describes a technique for optimizing the transmission timing of intervention messages in order to encourage more exercise while preventing excessive message sending.

[0008] Also, Non-Patent Document 2 describes a method in a large language model (LLM) of splitting the output into a thinking part and a response part, and evaluating the output of the response part to train the model.

[0009] Peng Liao, Kristjan Greenewald, Predrag Klasnja, Susan Murphy, “Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity”, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 4, No. 1, Article 18. Publication date: March 2020, Internet <URL: https: / / dl.acm.org / doi / abs / 10.1145 / <3381007> Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, Sainbayar Sukhbaatar, “Thinking LLMs: General Instruction Following with Thought Generation”, arXiv:2410.10630v1 [cs.CL] 14 Oct 2024, Internet <URL: http: / / arxiv.org / abs / 2410.10630>

[0010] However, although the method described in Non-Patent Document 1 personalizes the transmission timing of messages, it does not personalize the message content. Therefore, appropriate messages cannot be generated for each user.

[0011] On the other hand, in the method described in Non-Patent Document 2, the message response portion is optimized, but the thinking portion is not learned in a way that reflects its appropriateness. As a result, the thinking portion and the message content may not be consistent, and it is not possible to generate a message that appropriately reflects the thought process.

[0012] This invention was made in view of the above circumstances and aims to provide a technology that can generate messages that are appropriately personalized for each user and that appropriately reflect their thought process.

[0013] To solve the above problems, one embodiment of the message generation device or message generation method according to the present invention, when generating a behavior change support message using a message generation model, first acquires learning data including the user's behavior goals, attribute information, and intervention conditions, and generates a first prompt to be input to the message generation model based on the acquired behavior goals, attribute information, and intervention conditions. Next, the generated data output from the message generation model in response to the input of the first prompt is separated into a thinking portion and a message portion, and a second prompt for thinking evaluation and a third prompt for message evaluation are generated based on the separated thinking portion and message portion, respectively, and the second prompt and the third prompt are input to an evaluation model to obtain a first score representing the appropriateness of the thinking portion and a second score representing the appropriateness of the message portion from the evaluation model. Then, a predetermined loss function is calculated based on the first score and the second score, and a learning process is performed to update the parameters of the message generation model based on the loss function.

[0014] According to one aspect of this invention, for example, user attributes such as the user's state, habits, characteristics, and preferences, along with intervention conditions such as the surrounding environment and timing, are directly input to the message generation model M along with behavioral goals, without going through predefined intermediate states, and a message is generated. As a result, it becomes possible to generate effective behavioral change support messages whose content is appropriately personalized for each user.

[0015] In other words, according to one aspect of this invention, it is possible to provide a technology that enables the generation of messages whose content is appropriately personalized for each user and which appropriately reflect the user's thought process.

[0016] Figure 1 is a diagram showing an example of a behavioral change support system equipped with a message generation device according to one embodiment of the present invention. Figure 2 is a block diagram showing an example of the hardware configuration of a message generation device according to one embodiment of the present invention. Figure 3 is a block diagram showing an example of the software configuration of a message generation device according to one embodiment of the present invention. Figure 4 is a diagram showing an overview of the learning process of the message generation model executed by the message generation device shown in Figure 3 during the learning phase. Figure 5 is a flowchart showing an example of the processing procedure and processing content of the model learning process executed by the message generation device shown in Figure 3 during the learning phase. Figure 6 is a flowchart showing an example of the processing procedure and processing content of the message generation process executed by the message generation device shown in Figure 3 during the inference phase.

[0017] Embodiments of this invention will be described below with reference to the drawings.

[0018] [One Embodiment] (Configuration Example) (1) System Figure 1 is a diagram showing an example of a behavioral change support system equipped with a message generation device according to one embodiment of the present invention.

[0019] In one embodiment, the behavioral change support system enables data communication via a network NW between a message generation device SV and terminals UT1 to UTn used by service users or administrators.

[0020] Terminals UT1 to UTn consist of, for example, personal computers and communicate with the message generation device SV to send and receive data entered by the user or administrator and messages that encourage behavioral change. Other devices such as smartphones, tablet devices, or wearable devices may also be used as terminals UT1 to UTn.

[0021] A network (NW) includes, for example, a wide-area network centered on the Internet, and an access network for accessing this wide-area network. The access network may include, but is not limited to, public data communication networks employing 4G, 5G, or 6G standards, or LANs (Local Area Networks) employing Wi-Fi® standards.

[0022] (2) Message generation device SV The message generation device SV is installed on a server computer located on the Web or the cloud, for example.

[0023] Figures 2 and 3 are block diagrams showing examples of the hardware and software configurations of the message generation device SV.

[0024] The message generation device SV includes a control unit 1 that uses a hardware processor such as a Central Processing Unit (CPU), and a storage unit having a program storage unit 2 and a data storage unit 3, and a communication interface (hereinafter referred to as I / F) unit 4 are connected to the control unit 1 via a bus 5.

[0025] The communication interface unit 4 transmits and receives various data necessary to support user behavior change between terminals UT1 to UTn, in accordance with the communication protocol defined in the network NW.

[0026] The program storage unit 2 is configured, for example, as a storage medium, by combining a non-volatile memory that can be written to and read at any time, such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), with a non-volatile memory such as ROM (Read Only Memory). In addition to middleware such as an OS (Operating System), it stores various programs necessary to execute various control processes according to one embodiment of this invention.

[0027] The data storage unit 3 is configured, for example, as a storage medium, by combining a non-volatile memory that can be written to and read at any time, such as an HDD or SSD, with a volatile memory such as RAM (Random Access Memory). The storage area is provided with an input data storage unit 31 and a model data storage unit 32.

[0028] The input data storage unit 31 stores training input data, for example, sent from terminals UT1 to UTn, in order to train the message generation model.

[0029] The model data storage unit 32 includes a message generation model storage unit 321 for storing model data of a message generation model, a thought evaluation model storage unit 322 for storing a thought evaluation model, and a message evaluation model storage unit 323 for storing a message evaluation model.

[0030] Message generation models are constructed using, for example, Large Language Models (LLMs), but are not limited to these. Similarly, thought evaluation models and message evaluation models are expected to be constructed using LLMs, but are not limited to these.

[0031] The control unit 1 includes, as processing functions necessary for realizing one embodiment of this invention, an input data acquisition processing unit 11, an input formatting processing unit 12, a learning processing unit 13, an inference processing unit 14, an output formatting processing unit 15, and a message transmission processing unit 16.

[0032] Each of the above-mentioned processing units 11 to 16 is implemented by having the hardware processor of the control unit 1 execute an application program stored in the program storage unit 2. Note that some or all of the above-mentioned processing units 11 to 16 may be implemented using hardware such as LSI (Large Scale Integration) or ASIC (Application Specific Integrated Circuit).

[0033] In the learning phase, the input data acquisition processing unit 11 acquires learning input data from terminals UT1 to UTn via the communication interface unit 4 and stores it in the input data storage unit 31. In the inference phase, the input data acquisition processing unit 11 also acquires input data related to the supported user from terminals UT1 to UTn via the communication interface unit 4. The input data includes, for example, the user's personal information, intervention conditions, and behavioral goals.

[0034] The input formatting processing unit 12 generates a prompt for input to the message generation model based on the acquired input data.

[0035] In the learning phase, the learning processing unit 13 performs a learning process that updates and optimizes the parameters of the message generation model stored in the message generation model storage unit 321 based on the learning input data stored in the input data storage unit 31. An example of the learning process will be explained in the operation example.

[0036] In the inference phase, the inference processing unit 14 obtains generated data by inputting prompts based on input data obtained from the terminals UT1 to UTn of the supported user into the message generation model stored in the message generation model storage unit 321, and outputs the obtained generated data to the output formatting processing unit 15.

[0037] The output formatting processing unit 15 splits the generated data output from the message generation model into a thinking portion and a message portion. The output formatting processing unit 15 then inputs the split thinking portion and message to the input formatting processing unit 12 during the learning phase and outputs them to the message transmission processing unit 16 during the inference phase.

[0038] The message transmission processing unit 16 generates a behavior change support message based on the thought portion and message portion of the data output from the output formatting processing unit 15, and transmits the generated behavior change support message from the communication interface unit 4 to the terminals UT1 to UTN of the supported user.

[0039] (Operation Example) Next, an operation example of the message generation device SV configured as described above will be explained.

[0040] (1) Learning Phase (Overview) Figure 4 is a diagram illustrating the overview of the learning process of the message generation model that the message generation device SV performs in the learning phase.

[0041] (1-1) Acquisition of input data The input data acquired includes the behavioral goal g, user information, and intervention conditions u.

[0042] Of these, behavioral goal g refers to the behavioral goal that encourages the user to take action, and is set by the user themselves or by healthcare professionals, etc.

[0043] On the other hand, user information and intervention conditions u include user attribute information, intervention timing, weather conditions, etc., and are assumed to be all verbalized, but may also be measured data. In the case of measured data, the values ​​will be verbalized using a separately prepared machine learning model. Furthermore, the amount of information described in user information and intervention conditions u may differ depending on the user. For example, it is assumed that only information for which user permission has been obtained or responses have been received will be used.

[0044] (1-2) Formatting of input data The above input data is formatted by the input formatting processing unit 12 to form a prompt P for input into the message generation model. M The input is formatted to (g,u). The input formatting process involves, for example, preparing a prompt template for a message generation model, and writing the action goal, user information, and intervention conditions included in the above input data in the necessary parts of this template, thereby generating the prompt P M Generate (g,u).

[0045] (1-3) Operation of message generation model M The message generation model M is configured using, for example, LLM, and the above prompt P M When (g,u) is input, appropriate thought processing is performed based on the behavioral goal g and user information / intervention conditions u, and personalized message data y is generated and output for each user. In this case, the message data y consists of data in which a thought part t and a response part m are written consecutively.

[0046] (1-4) Formatting of Message Data y The message data y output from the message generation model M is separated by the output formatting processing unit 15 into a thinking part t and a response part (hereinafter also referred to as a message part or simply a message) m. The separated thinking part t and message m are individually input to the input formatting processing unit 12.

[0047] (1-5) Prompt P for Thinking Evaluation Model Jt and Prompt P for Message Evaluation Model Jm The prompt P for the thinking evaluation model Jt and Prompt P for Message Evaluation Model Jm is generated in the input formatting processing unit 12.

[0048] That is, the input formatting processing unit 12 has two templates prepared in advance for the thinking evaluation model and the message evaluation model, respectively. When the thinking part t and the message m are input, the input formatting processing unit 12 describes the thinking part t and the message m, and the action goal g of the input data and the user information / intervention condition u stored in the input data storage unit 31 in the respective templates, thereby generating the prompt P for the thinking evaluation model Jt and Prompt P for Message Evaluation Model Jm is generated.

[0049] (1-6) Thinking Evaluation Model J t (t) and Message Evaluation Model J m (m) The thinking evaluation model J t (t) is a model for determining the goodness or badness of the thinking process, that is, the degree of appropriateness, and is configured using, for example, a pre-trained LLM. The thinking evaluation model J t (t) takes the prompt P for the thinking evaluation model Jt as an input, determines whether the thinking process t is appropriate for generating the message m, and outputs a score S t representing the determination result.

[0050] The message evaluation model J m (m) is a model that determines the quality, or appropriateness, of a message, and this too is constructed using, for example, a pre-trained LLM. Message evaluation model J m (m) is the prompt P for the above message evaluation model. Jm The system takes the input as input and determines whether the message can encourage behavioral change, and the result is represented by a score S. m Outputs.

[0051] Note that the thinking evaluation model J t (t) and message evaluation model J m (m) may be a model specifically trained for each purpose, or a general-purpose LLM may be used. Examples of general-purpose LLM models include Llama2 and elyza.

[0052] (1-7) The learning processing unit 13 for updating the parameters of the message generation model M includes a parameter update processing unit for updating the parameters of the message generation model M. A general method such as backpropagation is used for updating the parameters, but other algorithms may also be used.

[0053] The parameter update processing unit is the same as the above-mentioned thinking evaluation model J t Score S output from (t) t And, message evaluation model J m Score S output from (m) m Using the inputs, we calculate the loss related to the following three elements.

[0054] The loss related to the first element is the loss of outputting only the thinking part t (hereinafter also called the DPO: Direct Preference Optimization loss). The loss related to the second element is the loss of outputting both the thinking part t and the message m (hereinafter also called the TPO: Thought Preference Optimization loss). The loss related to the third element is the loss representing the consistency between the thinking part t and the message m.

[0055] The parameter update processing unit then updates the parameters of the message generation model so that the loss function, which is represented by the weighted sum of each calculated loss, i.e., the weighted average, satisfies the pre-set conditions.

[0056] Regarding DPO, please refer to “Direct Preference Optimization: Your Language Model is Secretly a Reward Model,” 37th Conference on Neural Information Processing Systems (NeurIPS 2023), available online at <URL: http: / / arxiv.org / abs / 2305.18290>. Regarding TPO, please refer to Non-Patent Literature 2 for detailed information; therefore, a detailed explanation will be omitted here.

[0057] (Example) Next, an example of the processing procedure and content of the learning process for the message generation model will be described.

[0058] Figure 5 is a flowchart showing an example of the processing procedure and processing content of the learning process executed by the control unit 1 of the message generation device SV.

[0059] (1) Acquisition of learning input data For example, when a learning request is sent from the system administrator's terminal UTi, the control unit 1 of the message generation device SV detects the input of the learning request in step S10 and proceeds to step S11. Then, under the control of the input data acquisition processing unit 11, the control unit 1 executes the process of acquiring learning input data as follows.

[0060] Specifically, the system receives multiple input data sets prepared for learning purposes, transmitted from the system administrator's terminal UTi, via the communication interface unit 4, and stores the received learning input data in the input data storage unit 31. Each learning input data set has a defined behavioral goal g, user information, and intervention condition u.

[0061] Behavioral goals g are composed of text data such as, for example, "exercise by walking at least 10,000 steps per day," "limit calorie intake to XX," or "get off the train one stop early and walk the rest of the way," but they may also be expressed using audio data, graphic data, image data, etc.

[0062] User information and intervention conditions u are specifically represented as a set of user attributes and intervention conditions, such as the following:

[0063] Age: 35 Gender: Female Residence: Shibuya Ward, Tokyo Occupation: Marketing Researcher Big Five: Openness 6 / 7, Conscientiousness 4 / 7, Extraversion 3 / 7, Agreeableness 1 / 7, Neuroticism 7 / 7 Stage of Behavioral Change for Health Behavior: Preparation Health Goals: Weight loss and improved physical fitness Exercise Habits: Currently almost none Diet: Eats out frequently, nutritional balance tends to be poor Stress Relief Methods: Talking with friends, reading, and watching movies Intervention Timing: 9am, December 5, 2024 Weather: Sunny Expected High Temperature: 8 degrees Celsius.

[0064] (2) When the acquisition of input data for learning is completed, the control unit 1 of the message generation device SV then, in step S12, under the control of the input formatting processing unit 12, generates a prompt P which will be input to the message generation model based on the learning input data. M The process to generate it is executed as follows:

[0065] In other words, the input formatting processing unit 12 reads multiple learning input data one by one from the input data storage unit 31, and writes the behavioral target g and user information / intervention conditions u defined for each read learning input data in the necessary places of a pre-prepared template, thereby generating a prompt P for the message generation model. M Generate (g,u).

[0066] Below is the prompt P for the generated message generation model. MAn example of (g,u) is shown. Note that {} indicates where the corresponding information is written. • Instructions "Please answer the following user inquiry comprehensively and in detail." "Before answering, you may write down your thought process. Write your thought process after "This is my thought process," and write your answer after "This is my answer."" • Inquiry "Consider the user's information and create a message that encourages the achievement of the given behavioral goal." • Behavioral goal {Behavioral goal} • User information {User information} • Intervention conditions {Intervention conditions}.

[0067] (3) The control unit 1 of the learning processing message generation device SV then performs the learning process for the message generation model M under the control of the learning processing unit 13 as follows. Here, we will explain using the example of performing learning on an untrained dedicated model as the message generation model M, but it is also possible to use a pre-trained general-purpose LLM.

[0068] In other words, the learning processing unit 13 processes the prompt P for the message generation model generated by the input formatting processing unit 12. M (g,u) is input to the message generation model M stored in the message generation model storage unit 321.

[0069] As a result, the message generation model M generates the above prompt P according to the parameters before training. M Based on the behavioral goal g defined in (g,u) and the user information / intervention conditions u, a thought process is performed, and message data y is generated and output. Here, for example, the following explanation assumes that two message data y1 and y2 are output, but the number of message data output may be one or three or more.

[0070] The learning processing unit 13 first inputs the message data y1 and y2 output from the message generation model M to the output formatting processing unit 15. In step S14, the output formatting processing unit 15 separates the message data y1 and y2 into thinking parts t1 and t2 and response parts (messages) m1 and m2, respectively.

[0071] Messages m1 and m2 are messages designed to encourage users to change their behavior, and are examples of the following:

[0072] "A new day, a new me. Why not contribute to your health and the environment with small changes to your commute? Simply getting off the train one stop early and walking the rest of the way can lead to weight loss and improved physical fitness. Let's build confidence by accumulating small successes. Relieve stress and achieve a healthy body."

[0073] The thinking parts t1 and t2 represent the thought process that generates the message, and are, for example, as follows:

[0074] (1) Understanding the user's characteristics: The user is 35 years old and works as a marketing researcher, suggesting they are skilled at analysis and planning. In the Big Five personality test, high openness and low conscientiousness indicate they are curious but dislike dishonesty and insincerity. Low extraversion and agreeableness suggest they are not very sociable and struggle with collaboration and communication with others. High neuroticism suggests they are prone to anxiety and stress.

[0075] (2) Understanding health goals and current situation: The health goals are weight loss and improved physical fitness, and the specific action goal is to "get off the train one stop early and walk the rest of the way during commutes." Currently, there is no exercise habit, and the diet is unbalanced due to eating out frequently, so there is room for improvement.

[0076] (3) Message creation: Use positive expressions such as "A new day, a new me" to boost user motivation. Specify action goals and convey that they are easy to start, such as "Contribute to your health and the environment with small changes on your commute." Suggest a gradual and positive approach, taking into consideration neurotic tendencies, such as "Build confidence by accumulating small successes." Encourage exercise by linking it to the user's stress relief methods, such as "Relieve stress by walking the distance of one train station."

[0077] The output formatting processing unit 15 inputs the separated thought portions t1, t2 and messages m1, m2 to the input formatting processing unit 12. In step S15, the input formatting processing unit 12 writes the input thought portions t1, t2 to the necessary parts of a pre-prepared template, thereby generating a prompt P for the thought evaluation model. Jt (t1), P Jt Generate (t2).

[0078] For example, prompt P for a thinking assessment model Jt (t1), P Jt As (t2), the following prompt is generated: (Example 1) - Instructions "Please answer the following user inquiry comprehensively and in detail." "Before answering, you may write down your thought process. Write your thought process after 'This is my thought process,' and write your answer after 'This is my answer.'" - Inquiry "Consider the user information and rate on a scale of 1 to 10 whether the thought process that generates a message encouraging the achievement of the given behavioral goal considers each aspect of MINDSPACE." - Behavioral goal {Behavioral goal} - Thought process {Thinking part} - User information {User information} - Intervention conditions {Intervention conditions}.

[0079] (Example 2) Instructions: "Please answer the following user inquiry comprehensively and in detail. You may write down your thought process before answering. Write your thought process after 'This is my thought process,' and write your answer after 'This is my answer.'" Inquiry: "Please rate on a scale of 1 to 10 whether you considered each element of the user's information in your thought process to generate a message that encourages the achievement of the given behavioral goal, taking the user's information into account." Behavioral goal {Behavioral goal} Thought process {Thinking process} User information {User information} Intervention conditions {Intervention conditions}.

[0080] Furthermore, in step S17, the input formatting processing unit 12 writes the input messages m1 and m2 to the necessary parts of a pre-prepared template, thereby generating a prompt P for the message evaluation model. Jm (m1), P Jm Generate (m2).

[0081] For example, prompt P for message evaluation model Jm (m1), P Jm As (m2), the following prompts are generated: • Instructions "Please answer the following user inquiry comprehensively and in detail." "Before answering, you may write down your thought process. Write your thought process after 'This is my thought process,' and your answer after 'This is my answer.'" • Inquiry "Considering the user's information, please rate the effectiveness of the message that encourages the achievement of the given behavioral goal on a scale of 1 to 10." • Behavioral Goal {Behavioral Goal} • User Information {User Information} • Intervention Condition {Intervention Condition} • Message {Message}.

[0082] The learning processing unit 13 then, in step S16, prompts P for the thinking evaluation model. Jt (t1), P Jt (t2) is stored in the thought evaluation model J in the thought evaluation model memory unit 322. t Enter this into (t).

[0083] As a result, the thinking evaluation model J t (t) Then, the input prompt P for the above thought evaluation model is Jt (t1), P Jt Based on (t2), a process is performed to determine whether the thought processes t1 and t2 are appropriate for generating messages m1 and m2. For example, a process is performed to determine whether it follows a known theory (such as the Planned Behavior Theory or a nudge checklist like MINDSPACE) or whether it takes into account specific elements (such as occupation, hobbies, or habits). Then, the thought evaluation model J is used. t (t) represents the score S which indicates the result of that judgment.t1 , S t2 The following will be output.

[0084] In addition, the learning processing unit 13, in step S18, prompt P for the message evaluation model Jm (m1), P Jm (m2) is stored in the message evaluation model J in the message evaluation model storage unit 323. m Enter the value in (m).

[0085] As a result, the message evaluation model J m (m) Then, prompt P for the above message evaluation model is entered. Jm (m1), P Jm Based on (m2), a process is performed to determine whether messages m1 and m2 can encourage behavioral change in the user. For example, a process is performed to determine the probability or achievement rate of achieving the behavioral goal. Then, the message evaluation model J m (m) represents the score S which indicates the result of the judgment. m1 , S m2 The following will be output.

[0086] Note that the thinking evaluation model J t (t) and message evaluation model J m The score output from (m) may be in the form of an evaluation value assigned to a single input thought or message, or it may be a ranking of multiple input thoughts or messages. For example, a higher score may indicate a better evaluation, or in the case of ranking, it may be processed by multiplying by -1, etc.

[0087] The learning processing unit 13 then, in step S19, processes the above-mentioned thinking evaluation model J t Score S output from (t) t1 , S t2 And, the above message evaluation model J m Score S output from (m) m1 , S m2 Based on this, the process of updating the parameters of the message generation model M is executed.

[0088] As already mentioned, common methods such as backpropagation can be used to update the parameters. The loss function can also be expressed, for example, as a weighted sum of the losses of multiple elements. The loss for each element can be calculated based on existing alignment techniques. The choice of alignment method depends, for example, on the scoring method for each element or the number of elements.

[0089] The following describes how to calculate the loss function for each element. Here, as an example, we will examine the evaluation model J for each of the two thoughts and messages. t (t), J m This section explains how to output a score consisting of scalar values ​​from (m) and how to calculate the loss function for the first, second, and third elements using DPO based on these scores.

[0090] The learning processing unit 13 first determines the score S as the first element of the loss function. t1 , S t2 Based on this, the loss of only the thinking parts t1 and t2 (DPO loss) is calculated. For example, based on the DPO loss, S t1 > S t2 In that case, the loss of DPO is represented by Replace some of the values.

[0091] Specifically, x is P M (g,u) to y ω Set t1 to y ｌ Replace t1 with y1. You may add a section equivalent to "The following is my thought process" within the text. Also, you may add a section equivalent to "The following is my thought process" in y2 to t2.

[0092] Furthermore, the learning processing unit 13 uses the score S as the second element of the loss function. m1 , S m2 Based on this, the loss (TPO loss) is calculated, with both the thinking parts t1 and t2 and messages m1 and m2 as outputs.

[0093] Specifically, based on the loss of DPO, S m1 > S m2 In this case, we substitute some of the values ​​in equation (1) above, which represents the loss of DPO. For example, we substitute x with P M (g,u) to y ω Set y1 to y ｌ Replace y' or y'' respectively. Here, y' is obtained by replacing y1's thought process t1 with t2, and y'' is obtained by replacing y1's message m1 with m2.

[0094] Note that y' and y'' may be replaced individually or both. However, if both y' and y'' are replaced, an equation is required that consists of two terms: equation (1) with y' replaced and equation (1) with y'' replaced. This equation can be appropriately defined based on equation (1) above.

[0095] Furthermore, the learning processing unit 13 calculates a loss as the third element of the loss function, which represents the consistency between the thinking parts t1 and t2 and the messages m1 and m2.

[0096] For example, if based on the loss of DPO, substitute some of the values ​​in equation (1) above that represent the loss of DPO. For example, replace x with P M (g,u) to y ω Set y1 to y ｌ Replace each with y'. Note that y' is obtained by replacing y1's thought process t1 with t2, or by replacing y1's message m1 with m2. Either one of these replacements may be performed, or both may be performed. However, if both are replaced, there are only two possibilities: (t2, m1) and (t1, m2), and (t2, m2) is not included.

[0097] Next, the learning processing unit 13 calculates a loss function by weighting and adding the losses calculated for each of the first, second, and third elements according to, for example, a predetermined weight coefficient. Then, based on the calculated loss function, it updates the parameters of the message generation model M so that the loss value is reduced.

[0098] Furthermore, when calculating the loss function, it is also possible to use only a portion of the losses calculated for each of the first, second, and third elements, for example, only the losses for the first and third elements. In this case, the loss function can be easily calculated by setting the weighting coefficient for the loss of the second element to "0".

[0099] Furthermore, various alignment methods can be used to calculate the loss for each element.

[0100] For example, existing alignment techniques include the following methods: DPO: <URL: http: / / arxiv.org / abs / 2305.18290> KTO: <URL: http: / / arxiv.org / abs / 2402.01306> UNA: <URL: http: / / arxiv.org / abs / 2408.15339>.

[0101] The alignment method used is selected appropriately depending on the method for scoring each element and the number of score outputs. For example, in the example mentioned above, the evaluation models J for thoughts and messages are used. t (t), J m In the example where (m) outputs a score represented by a scalar value for each of the two thoughts and messages, we have explained the case where DPO is used as the method for calculating the evaluation function of each element, but other examples are also possible, such as the following.

[0102] Example 1 (when changing the method for calculating the loss function): Evaluation models J for thoughts and messages t (t), J m If (m) outputs scalar values ​​calculated for each thought and message as scores, then UNA is used for the first and second elements of the evaluation function, and KTO is used for the third element. In this example, those that were not substituted are evaluated as desirable, and those that were substituted are evaluated as undesirable.

[0103] Example 2 (when changing the score calculation method): Evaluation models J for thoughts and messages t (t), Jm When (m) outputs the relative evaluation values of thoughts and messages as scores, DPO is used as the calculation method of the evaluation function.

[0104] Example 3 (when both the score and the calculation method of the evaluation function are changed); For each evaluation model J t (t), J m When (t) and (m) output scores that represent the results of determining whether each of one thought and one message is desirable or undesirable in binary values, KTO is used as the calculation method of the loss function.

[0105] Next, in step S20, the learning processing unit 13 determines whether the loss function obtained by weighted addition of the losses of the above first, second, and third elements satisfies a preset end condition. If not, it returns to step S12. Thereafter, in steps S12 to S20, under the control of the learning processing unit 13, the above-described series of learning processes are repeatedly executed based on each of the plurality of learning input data stored in the input data storage unit 31 until the loss function satisfies the end condition. On the other hand, when the loss function satisfies the end condition, the learning process ends.

[0106] (2) Inference phase In a state where the learning of the message generation model has been completed by the above-described learning phase, the control unit 1 of the message generation device SV generates a personalized behavior variation support message according to the actual input data input from the user as follows.

[0107] FIG. 6 is a flowchart showing an example of the processing procedure and processing content of the behavior variation support message generation process executed by the control unit 1 of the message generation device SV.

[0108] (1) Acquisition of input data When a message generation request is sent from the terminal UTj of the support target user, the control unit 1 of the message generation device SV detects the message generation request in step S30 and proceeds to step S31. Then, under the control of the input data acquisition processing unit 11, the control unit 1 executes the acquisition process of the input data related to the support target user as follows.

[0109] In other words, input data transmitted from the terminal UTj of the supported user or the user's primary care medical institution is received via the communication I / F unit 4. The input data includes behavioral goals g, user information, and intervention conditions u.

[0110] Behavioral goals g are entered, for example, by the user being supported or by a representative of their primary healthcare provider. User information and intervention conditions u are collected, for example, through questionnaires given to the user being supported, or from wearable devices or smartphones used by the user, or from websites where user information is stored.

[0111] (2) Input data formatting support When input data relating to the target user is acquired, the control unit 1 of the message generation device SV then, in step S32, under the control of the input formatting processing unit 12, generates a prompt P which will be input to the message generation model based on the input data. M The process to generate it is executed as follows:

[0112] In other words, the input formatting processing unit 12, similar to the learning phase, writes the behavioral target g and user information / intervention conditions u included in the input data into the necessary parts of a pre-prepared template, thereby generating a prompt P for the message generation model. M Generate (g,u).

[0113] (3) The control unit 1 of the inference processing message generation device SV then, in step S33, under the control of the inference processing unit 14, generates the message generation model prompt P M (g,u) is input to the trained message generation model M stored in the message generation model storage unit 321.

[0114] As a result, the trained message generation model M will, according to the trained parameters, generate the above prompt P. M Based on the behavioral goal g defined in (g,u) and user information / intervention conditions u, a thought process is performed, and personalized message data y corresponding to the supported user is generated and output.

[0115] (4) Next, in step S34, the control unit 1 of the output shaping message generation device SV for message data y separates the message data y output from the message generation model M into a thinking part t and a response part (message) m under the control of the output shaping processing unit 15. Then, the output shaping processing unit 15 inputs the separated thinking part t and message m to the message transmission processing unit 16.

[0116] (5) Finally, in step S35, the control unit 1 of the transmission message generation device SV for the action modification support message generates an action modification support message for the support target user based on the thinking part t and the message m separated by the output shaping processing unit 15 under the control of the message transmission processing unit 16. Then, the message transmission processing unit 16 transmits the generated action modification support message from the communication I / F unit 4 to the terminal UTj used by the support target user.

[0117] Thus, the support target user can receive a personalized action modification support message that reflects the thinking process based on the user's own attribute information and action goals at an appropriate timing specified by the user as an intervention condition.

[0118] (Effect) In one embodiment as described above, in the learning phase, the prompt P M generated based on the learning input data is input to the message generation model to be learned, and its output is separated into a thinking part t and a message m. For each of the separated thinking part t and message m, a prompt P Jt , P Jm is generated and input to the thinking evaluation model J t (t) and the message evaluation model J m (m). By doing so, scores S t , S m indicating the appropriateness of the thinking process and the message are obtained, and the parameters of the message generation model are updated so that the loss function calculated based on the obtained scores S t , S m waits for the condition.

[0119] Therefore, in the inference phase, user attributes such as the user's state, habits, characteristics, and preferences, along with intervention conditions such as the surrounding environment and timing, are directly input to the message generation model M along with behavioral goals, without going through predefined intermediate states, and a message is generated. As a result, it becomes possible to generate effective behavioral change support messages with content personalized for each user.

[0120] In one embodiment, when updating the parameters, the following losses are calculated: a loss for the thinking portion t only, a loss for both the thinking portion t and the message m as outputs, and a loss representing the consistency between the thinking portion t and the message m. The parameters are then updated based on a loss function obtained by weighting and summing these losses.

[0121] Therefore, the appropriateness of only the thinking portion t is judged, and its loss is reflected in the learning process. In other words, the quality of the thinking process is judged, and the result is given to the message generation model as training data. This makes it possible for the message generation model to learn to consider the content that the user wants to be considered. Furthermore, the appropriateness of both the thinking portion t and the message m is judged, and its loss is reflected in the learning process. This makes it possible to generate more effective and personalized messages.

[0122] Furthermore, the consistency between the thought process t and the message m is determined, and the loss is reflected in the learning process. As a result, messages are generated that maintain consistency between the thought process and the message. This suppresses problems such as adding things not considered in the thought process to the final message, or failing to reflect things considered in the thought process in the message, and consequently, the stability of the message generation model can be improved.

[0123] [Other Embodiments] (1) In the learning phase, for example, prompt P for the message generation model M You may add the following customizations to (g,u) to guide the thought process. For example, in the prompt example mentioned above, change or add "Consider the user's information and create a message that encourages the achievement of the given behavioral goal" to "Consider the benefits that can be gained from the user's information and actions and create a message that encourages the achievement of the given behavioral goal" or "Consider the user's exercise habits and create a message that encourages the achievement of the given behavioral goal."

[0124] Furthermore, if this addition is made, the thinking evaluation model J t (t) Evaluation score S t1 , S t2 In the process of obtaining the result, it is necessary to score the prompt to confirm whether guidance has been achieved or not. Also, when updating the parameters, the prompt P for the message generation model before customization should be used. M (g,u) must be used.

[0125] By making the above additions, it becomes possible to incorporate human knowledge, such as explicitly considering existing theories, which allows the system to learn to reflect the thought process without customizing prompts during the inference phase.

[0126] (2) Prompt P based on thinking parts t1 and t2 Jt (t1), P Jt (t2) is generated, and the generated prompt P Jt (t1), P Jt (t2) is a thinking evaluation model J t Enter the value into (t) and evaluate it as score S t1 , S t2 When obtaining the thinking evaluation model J, t It is also possible to prepare multiple (t) values ​​and evaluate from multiple perspectives. In this case, when updating the parameters, it is necessary to add terms representing the loss of the first element and the loss of the third element for each perspective. By doing so, it is possible to simplify and judge things that are difficult to judge simultaneously, such as when considering multiple theories, and thereby stabilize the learning process.

[0127] (3) In the parameter update process, when calculating the loss of the third element, the y' that is replaced may be replaced with a thinking part t' or message m' generated using a different prompt PM(g',u') instead of the thinking part t2 or message m2. In this way, learning can be performed that places importance on the differences between mutually different user information / intervention conditions u,u' or behavioral goals g,g' in the thinking process.

[0128] (4) In the parameter update process, when calculating the loss of the first element, the t2 that is replaced may be a thinking portion t' generated using another prompt PM(g',u'). This also allows learning to be performed in the thinking process that emphasizes the differences between mutually different user information / intervention conditions u,u' or behavioral goals g,g'.

[0129] (5) In one embodiment, the example described was that the multiple processing units and model data storage units of the message generation device SV are provided on a server computer on the Web or in the cloud. However, they may also be provided on a personal computer used by a system administrator, for example. Alternatively, the multiple processing units and model data storage units of the message generation device SV may be distributed across multiple server computers or personal computers.

[0130] (6) In one embodiment, the case of supporting behavioral changes related to the user's health was described as an example, but this invention can also be applied to other cases, such as supporting examinees in their exam preparation, supporting workers involved in production processes such as assembly and processing, and supporting how to move the body in various sports.

[0131] Furthermore, this invention is applicable to the learning and alignment of general-purpose LLMs that are not limited to specific uses, as well as specialized LLMs for different applications. For example, this invention can be applied when a portion of the learning data includes data related to the generation of motivational messages to support behavioral change, or data related to fields such as mathematics, marketing, and translation.

[0132] More specifically, in the field of mathematics, for example, losses can be calculated by scoring the thought process using a thinking evaluation model based on whether theorem names appear in the thought process or whether there are any logical leaps at each step. Similarly, in the fields of marketing and translation, losses can be calculated by scoring the thought process using a thinking evaluation model based on whether specific terms related to marketing or translation appear in the thought process or whether there are any logical leaps in the thought process.

[0133] Furthermore, the functional configuration of the processing unit of the message generation device, its processing procedure and content, and the structure and content of the behavioral change support messages can also be modified in various ways without departing from the spirit of this invention.

[0134] Although embodiments of this invention have been described in detail above, the above description is merely illustrative in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of this invention. In other words, when implementing this invention, specific configurations may be adopted as appropriate depending on the embodiment.

[0135] In short, this invention is not limited to the embodiments described above, and in the implementation stage, the components can be modified and materialized without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiments. Moreover, components from different embodiments may be appropriately combined.

[0136] SV...Message generation device UT1-UTn...Terminals NW...Network 1...Control unit 2...Program storage unit 3...Data storage unit 4...Communication I / F unit 5...Bus 11...Input data acquisition processing unit 12...Input formatting processing unit 13...Learning processing unit 14...Inference processing unit 15...Output formatting processing unit 16...Message transmission processing unit 31...Input data storage unit 32...Model data storage unit 321...Message generation model storage unit 322...Thought evaluation model storage unit 323...Message evaluation model storage unit

Claims

1. A message generation device that generates behavioral change support messages using a message generation model, comprising: a first processing unit that acquires learning data including the user's behavioral goals, attribute information, and intervention conditions; a second processing unit that generates a first prompt to be input to the message generation model based on the behavioral goals, attribute information, and intervention conditions; a third processing unit that separates the generated data output from the message generation model in response to the input of the first prompt into a thinking portion and a message portion, and generates a second prompt for thinking evaluation and a third prompt for message evaluation based on the thinking portion and the message portion, respectively; and a fourth processing unit that inputs the second prompt and the third prompt to an evaluation model to obtain a first score representing the degree of appropriateness of the thinking portion and a second score representing the degree of appropriateness of the message portion from the evaluation model, calculates a predetermined loss function based on the first score and the second score, and performs a learning process to update the parameters of the message generation model based on the loss function.

2. The message generation device according to claim 1, wherein the fourth processing unit calculates a first loss for the thinking portion only, a second loss relating to both the thinking portion and the message portion, and a third loss representing the consistency between the thinking portion and the message portion, based on the first score and the second score, and updates the parameters based on the loss function obtained by weighting and adding the calculated first loss, second loss and third loss.

3. A message generation method comprising: an information processing device that performs a process of generating behavioral change support messages using a message generation model, the method comprising: a process of acquiring learning data for a user, including behavioral goals, attribute information, and intervention conditions; a process of generating a first prompt to be input to the message generation model based on the behavioral goals, attribute information, and intervention conditions; a process of separating the generated data output from the message generation model in response to the input of the first prompt into a thinking portion and a message portion, and generating a second prompt for thinking evaluation and a third prompt for message evaluation based on the thinking portion and the message portion, respectively; and a process of inputting the second prompt and the third prompt to an evaluation model to acquire a first score representing the degree of appropriateness of the thinking portion and a second score representing the degree of appropriateness of the message portion from the evaluation model, calculating a predetermined loss function based on the first score and the second score, and performing a learning process to update the parameters of the message generation model based on the loss function.

4. A program that causes a processor in the message generation device to perform the processing performed by each processing unit in the message generation device described in claim 1 or 2.

Images