Feedback processing program, feedback processing method, and information processing apparatus

Abstract

To present efficient feedback leading to behavior change of a target person.SOLUTION: An information processing apparatus identifies characteristics of a target person from behavior of the target person during interaction. The information processing apparatus identifies recommended behavior to the target person and a feedback method corresponding to the characteristics of the target person on the basis of the identified characteristics of the target person. The information processing apparatus presents information including motivation for the identified recommended behavior of the target person to the target person by using the identified feedback method.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a feedback processing program, a feedback processing method, and an information processing device. [Background technology]

[0002] Efforts are being made to improve crime prevention awareness among the elderly by using machine learning models that realize conversations by inputting voice data and outputting voice data in response, as well as speakers playing the role of mock criminals, to simulate specialized frauds such as "it's me" fraud and refund fraud as training.In addition, by evaluating the training and determining the risk of falling victim to fraud and providing feedback to the trainers, it is expected that crime prevention awareness will be further improved.

[0003] In recent years, technologies have become known that assess fraud risk based on increases in the trainee's breathing and heart rate, and the level of tension and confusion estimated from these values, as well as technologies that send messages that gently encourage the user to perform specific actions. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2022-163957 [Patent Document 2] Japanese Patent Application Publication No. 2023-130608 [Patent Document 3] Japanese Patent Application Publication No. 2019-212263 Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is difficult to say that the above technology provides efficient feedback that leads to behavioral change in trainees. For example, the same information is presented to all trainees, and each trainee is unable to understand what actions to take to avoid special frauds. Similar issues exist not only in special fraud training, but also in various interactive evaluations, such as mental care through customer harassment training, improving engagement through one-on-one video analysis, improving supervisor skills through video and conversation, and skill improvement through counseling analysis.

[0006] In one aspect, an object of the present invention is to provide a feedback processing program, a feedback processing method, and an information processing device that can present efficient feedback that leads to a behavioral change in a subject. [Means for solving the problem]

[0007] In a first proposal, the feedback processing program is characterized in that it causes a computer to execute a process of identifying characteristics of a subject from the subject's behavior during dialogue, identifying recommended behavior for the subject and a feedback method corresponding to the subject's characteristics based on the identified characteristics of the subject, and presenting information to the subject that includes motivation for the identified recommended behavior using the identified feedback method. [Effects of the Invention]

[0008] According to one embodiment, it is possible to provide efficient feedback that leads to behavioral changes in a subject. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a special fraud training system according to a first embodiment. [Figure 2] FIG. 2 illustrates an example of weakly persuasive feedback. [Figure 3]FIG. 3 is a diagram illustrating the feedback process of the special fraud training system according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating a specific example of the feedback process of the special fraud training system according to the first embodiment. [Figure 5] FIG. 5 is a functional block diagram of the information processing apparatus according to the first embodiment. [Figure 6] FIG. 6 is a flowchart illustrating the flow of the special fraud training process according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating the identification of psychological characteristics. [Figure 8] FIG. 8 is a diagram illustrating the identification of behavioral characteristics. [Figure 9] FIG. 9 is a diagram illustrating the identification of recommended actions. [Figure 10] FIG. 10 is a diagram illustrating the identification of recommended actions. [Figure 11] FIG. 11 is a diagram illustrating the details of the narrative processing unit. [Figure 12] FIG. 12 is a diagram illustrating the details of the narrative selection unit. [Figure 13] FIG. 13 is a diagram illustrating the narrative type. [Figure 14] FIG. 14 is a diagram illustrating the details of the avatar generation unit. [Figure 15] FIG. 15 is a diagram illustrating the details of avatar voice synthesis. [Figure 16] FIG. 16 is a diagram illustrating a specific example of a narrative. [Figure 17] FIG. 17 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following describes in detail embodiments of the feedback processing program, feedback processing method, and information processing device disclosed herein with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments. The embodiments can be combined as appropriate within a consistent range. [Example]

[0011] (Overall composition) Figure 1 is a diagram illustrating a special fraud training system according to Example 1. The patent fraud training system shown in Figure 1 is a system that aims to improve the trainee's awareness of crime prevention by having a dialogue between a trainee, such as an elderly person, and an interactive generation model (trainer) that simulates a criminal who imitates a special fraud or a fraudster who commits a special fraud, and by estimating the trainee's risk of becoming a victim of fraud from the trainee's state during the dialogue and providing feedback. Specifically, an information processing device 10 executes a training AI (Artificial Intelligence) tool and executes a fraud simulation experience function and a feedback generation function.

[0012] For example, the information processing device 10 uses a generation AI trained in conversations about special frauds using criminal psychology or the like to make utterances that lead a trainee to commit fraud and executes a dialogue simulation to acquire the trainee's speech response. In this way, the information processing device 10 acquires conversation information from the dialogue simulation (during training) in which the trainee is engaged in a simulated experience of special fraud, and also acquires physiological data (e.g., pulse rate, respiratory rate, etc.) of the trainee during the dialogue simulation. The information processing device 10 then inputs the physiological data from training into a risk estimation model to estimate the trainee's risk of falling victim to special fraud and provides the trainee with feedback on the estimation result.

[0013] As described above, the information processing device 10 allows the trainee to experience special fraud, collects the results of the trainee's experience, and feeds back the results of the experience to the trainee so that the trainee can respond appropriately even if he or she actually encounters a special fraud situation, thereby encouraging a change in the trainee's awareness and behavior.

[0014] However, the feedback information provided by the information processing device 10 may not provide sufficient conditions for behavioral change to satisfy the user's goal, such as avoiding special fraud. For example, the target behavior presented in the feedback may be predetermined and uniform for all users. Specifically, if the user's goal is "avoiding special fraud," the required behavior should be different for each trainee, such as not only "not disclosing personal information" but also "not disclosing family information."

[0015] Furthermore, if the feedback merely presents information without appealing to the trainee's emotions, it is not sufficient to bring about behavioral change. For example, if the feedback information is uniform and the appeal is weak, it is unlikely to lead to behavioral change based on the results of the experience. Figure 2 illustrates an example of feedback with a weak appeal. As shown in Figure 2, feedback information such as "To all residents of X City, we are sending you information about the latest cases of special fraud" or "X City's website contains information about special fraud methods and countermeasures. Please check it regularly." is simply a presentation of information without any emotion, and does not appeal to the trainee's emotions well, making it insufficient to bring about behavioral change.

[0016] Therefore, the information processing device 10 according to the first embodiment identifies the characteristics (psychological characteristics and behavioral characteristics) of the trainee from the trainee's behavior during the dialogue. Then, based on the identified characteristics of the trainee, the information processing device 10 identifies a recommended behavior for the trainee and a feedback method (for example, narrative type) corresponding to the characteristics of the trainee. Thereafter, the information processing device 10 presents the trainee with information including motivation for the identified recommended behavior of the trainee using the identified feedback method.

[0017] In other words, the information processing device 10 provides feedback to the trainee using "narratives" that can achieve "appeals to emotions" that easily generate empathy and self-projection and make it easier to empathize, "reduction of psychological reactance (rejection reaction)" that conveys an argument through narration rather than simply forcing it on the trainee, and "processing fluency" that places less of a processing load on the brain than statistical information (numbers) or diagrams and is more likely to be perceived positively.

[0018] 3 is a diagram illustrating the feedback process of the special fraud training system according to the first embodiment. As shown in FIG. 3, the information processing device 10 reads out and provides feedback on the importance of recommended actions for the purpose of training, which is in accordance with the characteristics of the trainee identified during training, using an avatar (e.g., an expert, a victim, or a victim's family member) in accordance with the characteristics of the trainee. In other words, the information processing device 10 provides feedback using a narrative (narration) suited to each trainee.

[0019] Fig. 4 is a diagram illustrating a specific example of feedback processing in the special fraud training system according to the first embodiment. As shown in Fig. 4, the information processing device 10 determines that for a trainee who has acquired characteristics such as "high empathy and prone to anxiety" in the mock training, speech based on real experience will be most emotionally appealing, and provides feedback using a victim avatar. In addition, the information processing device 10 determines that for a trainee who has acquired characteristics such as "easily irritated and prone to self-confidence" in the mock training, calm speech will be most emotionally appealing, and provides feedback using an expert avatar.

[0020] In this way, the information processing device 10 can present efficient feedback that improves the motivational effect and leads to behavioral changes in the subject by appealing to emotions through narrative.

[0021] (Functional configuration) 5 is a functional block diagram illustrating a functional configuration of the information processing device 10 according to Example 1. As illustrated in FIG.

[0022] The communication unit 11 is a processing unit that controls communication with other devices, and is realized by, for example, a communication interface. Specifically, the communication unit 11 acquires physiological data (vital information) from a sensor or radar (not shown), which is installed at a location where the trainee is training and performs measurements. For example, the communication unit 11 acquires, as physiological data, a biological signal including at least one of pulse rate, heart rate, respiratory rate, etc.

[0023] The communication unit 11 also acquires conversation (audio data) from microphones (not shown) installed at the location where the trainee is training and at the location between the trainee and the trainer. The communication unit 11 also transmits feedback information to devices such as smartphones of the trainee and the trainee's family.

[0024] The output unit 12 is a processing unit that displays and outputs various types of information, and is realized by, for example, a display, a touch panel, etc. For example, the output unit 12 displays and outputs the results of risk estimation estimated by the control unit 30, which will be described later, and various types of information specified by the control unit 30 before generating feedback information.

[0025] The storage unit 20 is a processing unit that stores various data and various data executed by the control unit 30, and is realized by, for example, a memory, a hard disk, etc. The storage unit 20 stores a conversation information DB 21, a physiological data DB 22, and a conversation model 23.

[0026] The conversation information DB21 is a database that stores conversation information between the trainer and the trainee while the trainee is conducting a dialogue simulation (training). Specifically, the conversation information DB21 stores, for each training session, information spoken by the trainer and information given by the trainee in response to the trainer's utterance, in association with each other. For example, the conversation information DB21 stores the trainer's statement, "To return the refund, you will need to go to an ATM. You cannot process this at the counter at city hall," in association with the trainee's response, "You don't need to worry about getting the refund now."

[0027] The physiological data DB 22 is a database that stores physiological data (vital information) that is an example of a trainee's biological signal measured while the dialogue simulation (training) is being performed. Specifically, the physiological data DB 22 stores physiological data that is a reaction during the dialogue simulation, that is, a physiological response when the trainee is performing psychological activity.

[0028] For example, the physiological data DB22 stores "trainee, time, pulse rate, heart rate" in association with each other. The "trainee" stored here is an identifier that identifies the trainee. "Time" indicates the date and time when the physiological data was measured, and "pulse rate, heart rate" is an example of physiological data measured from the trainee during training. The physiological data DB22 may store the physiological data in a table format or in a graph format with the horizontal axis representing time and the vertical axis representing measured values.

[0029] The conversation model 23 is a trained machine learning model that outputs the next utterance in response to input voice data. For example, when voice data representing the trainee's conversation is input, the conversation model 23 outputs the next conversation to be uttered by the trainer. The trainer utters the conversation output by the conversation model 23 as voice data.

[0030] The control unit 30 is a processing unit that controls the entire information processing device 10 and is realized by, for example, a processor. The control unit 30 has a training execution unit 40 and a feedback unit 50, and is a processing unit that executes an interactive simulation between the control unit 30 and a trainee to allow the trainee to experience a simulated special fraud, and provides feedback on the results of the simulated experience to the trainee. The training execution unit 40 and the feedback unit 50 are realized by, for example, electronic circuits included in the processor or processes executed by the processor.

[0031] The training execution unit 40 is a processing unit that executes a conversation execution process and a sensing process, executes a dialogue simulation with the trainee, allows the trainee to experience a simulated special fraud, and acquires physiological data of the trainee during the simulated experience.

[0032] As a conversation execution process, the training execution unit 40 executes a dialogue simulation with the trainee, acquires conversation information during the dialogue simulation, and stores the information in the conversation information DB 14. For example, the training execution unit 40 uses the conversation model 23 to make utterances to the trainee that lead to fraud, and acquires conversation information exchanged in a series of responses from the trainee to the utterances of the perpetrator.

[0033] As another example, the training execution unit 40 acquires conversation information that is exchanged in a series of events in which a person playing the role of a criminal makes utterances that lead the trainee into committing fraud in accordance with a pre-created scenario of a special fraud, and the trainee responds to the utterances of the criminal. The training execution unit 40 can also analyze the voice data during the dialogue simulation to extract the volume, pitch, tone of voice, etc., and store them in the conversation information DB 21 in association with the conversation information.

[0034] As the sensing process, the training execution unit 40 executes a process of acquiring physiological data of the trainee while the dialogue simulation is being executed by the conversation execution process and storing the acquired data in the physiological data DB 22. For example, the training execution unit 40 acquires physiological data measured by various sensors located near the trainee from the start to the end of the dialogue simulation.

[0035] The feedback unit 50 has a user information acquisition unit 51, a characteristic identification unit 52, a recommended behavior identification unit 53, a narrative processing unit 54, an avatar generation unit 55, and a presentation unit 56, and is a processing unit that evaluates the trainee's simulated experience carried out by the training execution unit 40 and feeds back the evaluation results to the trainee.

[0036] Here, before describing the detailed processing of each processing unit, the flow of processing will be described, and then the details of each processing will be described.

[0037] (Processing flow) Fig. 6 is a flowchart showing the flow of the special fraud training process according to the first embodiment. As shown in Fig. 6, after an instruction to start the process is given (S101: Yes) and the training execution unit 40 executes the special fraud training (S102), the feedback unit 50 acquires the questionnaire responses (S103). The questionnaire responses may also be acquired before the training.

[0038] Next, the feedback section 50 acquires behavioral data during training (S104), and identifies psychological and behavioral characteristics of the trainee based on the questionnaire responses or the behavioral data (S105), and also identifies recommended behaviors (S106).

[0039] Thereafter, the feedback unit 50 selects a narrative type based on the psychological characteristics and behavioral characteristics (S107).Then, the feedback unit 50 generates an avatar image according to the selected narrative type (S108) and presents the generated image to the user (trainee) (S109).

[0040] Next, the processing executed by each function will be specifically described.

[0041] (Acquisition of psychological characteristics) The user information acquisition unit 51 is a processing unit that acquires the psychological characteristics of the trainee. Specifically, the user information acquisition unit 51 acquires the user's personality traits and the user's values ​​or desires (hereinafter referred to as values / desires) as psychological characteristics through a pre-training and post-training questionnaire. For example, personality traits refer to the subject's tendencies in thinking, feeling, and behavior, and indicate the subject's characteristics that have a certain consistency, such as cooperativeness, extroversion, and gullibility. Furthermore, values / desires are what the trainee values ​​and desires, and correspond to things that motivate the subject, such as relationships with family and financial stability.

[0042] 7 is a diagram illustrating the identification of psychological characteristics. As shown in FIG. 7, the user information acquisition unit 51 conducts a questionnaire to the trainee using paper media such as a questionnaire form or a web page, and acquires the trainee's psychological characteristics based on the questionnaire results. For example, the user information acquisition unit 51 conducts questionnaires such as the five-factor model, value orientation scale, value survey, and psychological scale.

[0043] For example, the user information acquiring unit 51 identifies a five-factor model (Big Five, OCEAN model). Specifically, the user information acquiring unit 51 identifies the personality traits of the psychological characteristics by having the user answer questions using the Ten Item Personality Inventory (TIPI-J) or the like.

[0044] Furthermore, the user information acquisition unit 51 can acquire values / desires of psychological characteristics by conducting a value orientation scale questionnaire, which is a scale for measuring the degree to which an individual is oriented toward six types of universal values ​​(theory, economy, beauty, religion, society, and power). Note that the values / desires of psychological characteristics can also be acquired using the Schwartz Value Survey or the Rokeach Value Survey.

[0045] In addition, the user information acquisition unit 51 can also acquire values / desires of psychological characteristics by presenting questions such as "relationships with family, ties to the community, independent living, financial stability, physical health" and having the user select one or more applicable items.

[0046] As described above, the user information acquiring unit 51 identifies the psychological characteristics of the trainee and outputs them to the recommended behavior identifying unit 53.

[0047] (Identifying behavioral characteristics) The characteristic identification unit 52 is a processing unit that identifies behavioral characteristics, which are characteristic behaviors performed by the trainee, from the behavior of the subject during training. Specifically, the characteristic identification unit 52 analyzes the amount and content of speech during training, vital signs, facial expressions, etc., to identify behavioral patterns that the trainee is likely to exhibit, such as a tendency to be talkative or not seeking advice, as behavioral characteristics.

[0048] Fig. 8 is a diagram for explaining the identification of behavioral characteristics. As shown in Fig. 8, the characteristic identification unit 52 uses conversation information and vital information during training to perform analysis of the content of statements, quantitative analysis of statements, combined analysis of statements and vital information, estimation using an estimation model, and the like, thereby identifying behaviors, some values / desires, and some personality characteristics.

[0049] Specifically, to analyze the content of the utterances, the characteristic identification unit 52 acquires the content of the utterances, the number of keywords appearing in the utterances, and the like, and identifies the behavioral characteristics. For example, when a trainee utters personal information or a phone number, the characteristic identification unit 52 identifies the behavioral characteristic as "telling personal information to others." Furthermore, when a trainee asks the other person a predetermined number of questions, the characteristic identification unit 52 identifies the behavioral characteristic as "asking the other person about something they don't know." Furthermore, when a trainee promises to receive money in a single conversation or when the word "money" is uttered a predetermined number of times in a single conversation, the characteristic identification unit 52 identifies the behavioral characteristic as "accepting something related to money without consulting." Note that behavioral characteristics related to utterance tendencies may be identified by classifying the trainee's utterances into types such as "consulting others," "deciding on an action," and "questioning" using natural language analysis (e.g., a pre-trained machine learning model for text classification).

[0050] For example, as a quantitative analysis of utterances, the characteristic identification unit 52 identifies the behavioral characteristic "large amount of utterance per utterance" when the amount of utterances or the number of utterances of the trainee is equal to or greater than a threshold. Similarly, the characteristic identification unit 52 identifies the behavioral characteristic "fast speaking rate (fast speaking)" when the speaking rate of the trainee is equal to or greater than a threshold, and identifies the behavioral characteristic "early speaking start (speaking without pauses)" when the number of times the trainee speaks more than the other person or the number of times the trainee speaks first is equal to or greater than a threshold.

[0051] For example, in analyzing a combination of a statement and vital signs, if the characteristic identification unit 52 detects a reaction such as "increased heart rate and increased blinking frequency" after hearing a specific statement from the other person, it can estimate whether the person is wary of or agitated by the specific statement or words contained in the statement. Similarly, the characteristic identification unit 52 can estimate whether or not the person is wary of special frauds based on whether or not the heart rate increases after hearing words characteristic of frauds, such as ATM (Automatic Teller Machine) or electronic money card.

[0052] Furthermore, the characteristic identification unit 52 may be able to expand the information on values / desires by analyzing a combination of utterances and vital signs. For example, if the heart rate increases after hearing the utterance "You're causing trouble for your family," the characteristic identification unit 52 may estimate that "you place importance on family" as a value / desire.

[0053] Furthermore, the characteristic identification unit 52 may be able to expand the personality traits by using an estimation model that estimates psychological states from vital information. For example, the characteristic identification unit 52 can input time-series vital information into an estimation model to obtain fluctuations in psychological states (such as the presence or absence of tension or confusion), and estimate "gullibility" based on these fluctuations. Furthermore, the characteristic identification unit 52 can estimate tension or confusion from changes in vital signs (such as heart rate or blood pressure) during training, and estimate "easily nervous or confused" based on the degree of tension or confusion.

[0054] As described above, the characteristic identifying unit 52 identifies the behavioral characteristics of the trainee and outputs them to the recommended behavior identifying unit 53.

[0055] (Identifying recommended actions) The recommended behavior identification unit 53 is a processing unit that identifies recommended behavior for the trainee based on the characteristics of the trainee. Specifically, the recommended behavior identification unit 53 identifies recommended behavior for achieving the training objective of "not being deceived by specialized frauds" by using the psychological characteristics (personality characteristics and values / desires) acquired by the user information acquisition unit 51 and the behavior identification, part of the values / desires, and part of the personality characteristics identified by the characteristic identification unit 52. In other words, the recommended behavior identification unit 53 identifies recommended behavior for each trainee based on the characteristics of the trainee.

[0056] For example, the recommended behavior identification unit 53 prepares data that defines the degree of recommendation for each behavior for each characteristic. Then, the recommended behavior identification unit 53 can identify the degree of recommendation for each behavior for the trainee by multiplying the trainee's characteristics, such as a vector of real values ​​from 0 to 1, by a matrix of recommendation degrees. Note that the values ​​of the characteristic vector and recommendation degree matrix do not have to be real values, and may be binary (0 or 1).

[0057] 9 and 10 are diagrams illustrating the identification of recommended actions. As shown in Fig. 9, the recommended action identification unit 53 holds a characteristic table 100 in which a "characteristic ID" that identifies a characteristic, a "characteristic type" that identifies a characteristic type, and a "characteristic" that indicates a specific characteristic are associated with each other. The recommended action identification unit 53 also holds a recommendation level table 101 in which a "characteristic ID" is associated with a "recommended action ID" that identifies a recommended action. The recommended action identification unit 53 also holds a recommended action table 102 in which a "recommended action ID" is associated with a "recommended action" that indicates a specific recommended action.

[0058] In this state, it is assumed that the psychological trait "easily gullible" and the behavioral trait "tendency to be talkative" are identified. Then, the recommended behavior identification unit 53 refers to the trait table 100 and acquires the trait ID "1" corresponding to the psychological trait "easily gullible" and the trait ID "3" corresponding to the behavioral trait "tendency to be talkative."

[0059] Thereafter, the recommended action identification unit 53 refers to the recommendation degree table 101 and identifies the recommended action using "recommended action ID '1' = 1.0, recommended action ID '2' = 0.0, recommended action ID '3' = 0.5,..." which is associated with the characteristic ID '1' of the psychological characteristic 'gullible', and "recommended action ID '1' = 0.1, recommended action ID '2' = 1.0, recommended action ID '3' = 0.2,..." which is associated with the characteristic ID '3' of the behavioral characteristic 'talkative tendency'.

[0060] For example, as shown in Fig. 10, the recommended action identification unit 53 calculates the relevance a for each recommended action for the trainee from the vector t of the trainee's characteristics and the characteristic-recommended action recommendation degree matrix K, and identifies the recommended action with the highest a. Note that since t is the number of elements = number of characteristics = N, in the example of Fig. 10, "t = (1,0,1)", and since the recommendation degree matrix is ​​the number of characteristics × number of recommended actions (M), the recommendation degree table 101 applies. Note that 1 (a number other than 0, such as 0.7 or 0.9) indicates the degree of gullibility or talkativeness, and 0 indicates that it does not correspond to cooperativeness.

[0061] That is, in the process of matrix calculation, the recommended action specification unit 53 calculates the value of the recommendation level of each recommended action for the trait ID "1" of the psychological trait "easily gullible," calculates the value of the recommendation level of each recommended action for the trait ID "3" of the behavioral trait "talkative tendency," and adds them up for all the psychological traits to determine the value of the recommendation level of each recommended action (a1 to a M-1 Then, the recommended action specification unit 53 calculates the calculated (a1 to a M-1 ), the maximum value a is identified. For example, if the maximum value a is a3, the recommended action identifying unit 53 identifies the recommended action ID "3" corresponding to a3. Then, the recommended action identifying unit 53 refers to the recommended action table 102 and identifies the recommended action "consult with a reliable person" corresponding to the recommended action ID "3" as the recommended action of the trainee.

[0062] As described above, the recommended action specifying unit 53 specifies recommended actions for the trainee based on the psychological characteristics and behavioral characteristics of the trainee, and outputs the psychological characteristics, behavioral characteristics, and recommended actions to the narrative processing unit 54.

[0063] (Narrative Processing) The narrative processing unit 54 has a narrative type selection unit 54a and a narrative sentence generation unit 54b, and is a processing unit that uses the identified trainee's characteristics and recommended actions for the trainee to identify a narrative type, which is a feedback method that corresponds to the trainee's characteristics.

[0064] Fig. 11 is a diagram illustrating details of the narrative processing unit 54. As shown in Fig. 11, the narrative processing unit 54 has a narrative type selection unit 54a, a narrative sentence generation unit 54b, a narrative type storage unit 54c, a narrative sentence example storage unit 54d, a generative AI model 54e, and a base prompt storage unit 54f. The narrative type storage unit 54c, the narrative sentence example storage unit 54d, the generative AI model 54e, and the base prompt storage unit 54f are stored in the storage unit 20.

[0065] First, the narrative type selection unit 54a selects a narrative (story) appropriate for the trainee. FIG. 12 is a diagram illustrating the details of the narrative type selection unit 54a. As shown in FIG. 12, the narrative type selection unit 54a holds a characteristic table 200 in which a "characteristic ID" that identifies a characteristic, a "characteristic type" that specifies a characteristic type, and a "characteristic" that indicates a specific characteristic are associated with each other. The narrative type selection unit 54a also holds a narrative table 201 in which a "characteristic ID" is associated with a "narrative type ID" that identifies a narrative. The narrative type selection unit 54a also holds a narrative type table 202 in which a "narrative ID" is associated with a "narrative type" that indicates a narrative to be used for feedback.

[0066] In this state, the narrative type selection unit 54a refers to the characteristic table 200 to identify a "characteristic ID" corresponding to the identified psychological characteristic or behavioral characteristic. Next, the narrative type selection unit 54a refers to the narrative table 201 to identify each value of the narrative type associated with the identified characteristic ID. After that, the narrative type selection unit 54a refers to the narrative type table 202 to identify a narrative type ID using the same method as in FIG. 10. Then, the narrative type selection unit 54a refers to the narrative type table 202 to select a "narrative type" corresponding to the identified narrative type ID.

[0067] Next, the narrative type selection unit 54a obtains a template associated with the selected "narrative type" from the narrative type storage unit 54c and outputs it to the narrative text generation unit 54b. Here, the narrative type storage unit 54c stores templates for each "narrative type" classified into "narrator" and "narrative structure."

[0068] Figure 13 illustrates the narrative style. As shown in Figure 13, the "narrator" is defined as a first-person figure, such as a "person in the same position," a "stakeholder," or a "stakeholder-like person," and a second-person figure, such as an "expert" or "knowledgeable person." The "structure" defines the order in which facts, recommended actions, and emotional appeals are presented, including, for example, "emotions that contribute to behavioral change" and "motivators." Here, "emotions that contribute to behavioral change" include "intrinsic reward: finding value in the behavior itself (e.g., behavior associated with a desire)," "imagined regret: regret resulting from not taking action," and "self-efficacy: a sense of accomplishment." Furthermore, "motivators" are the types of human needs classified by evolutionary psychology research. Generally, higher-level needs become dominant with age, but this also depends on the situation. Communication that appeals to dominant needs is more likely to result in behavioral change.

[0069] Specific examples of the narrative type mentioned above include the first person (victim) and failure experience type, and the second person (expert) and lecture type. For example, in the case of the first person (victim) and failure experience type, the template matches "I was scammed like this" as the "scam method (fact)," "I did ~ because I didn't want to do ~, but in the end ~ happened instead and I regret it," "I would like to act like this in the future" as the "recommended behavior," and "By doing ~, I try not to forget" as the "self-efficacy."

[0070] When this narrative type is explained using the special fraud example in this embodiment, the narrative type template defines "premise, narrator, and structure." For example, in the case of a first-person (victim) and failure experience type, the "premise" is the background information for providing feedback, and defines specific settings such as character settings and victim settings, such as "playing the role of the victim in a special fraud story." The "narrator" is information that defines the person the avatar will play, such as the victim of a special fraud, identified values ​​and characteristics, and the trainee's characteristics (age, gender). Other information can also be defined to improve realism and enhance the trainee's empathy. The "structure" is information that defines the story told by the avatar, and defines the fraud method (facts), desire (value) + regret, recommended actions, and self-efficacy mentioned above.

[0071] As another example, in the case of a lecture-style second-person (expert) presentation, the "premise" is the information necessary for providing feedback, and defines specific settings such as the character and victim, such as "Please address me by my last name, as an expert would do." The "narrator" is information that defines the person the avatar will play, such as Professor XX of XX University. The "structure" is information that defines the avatar's reflections on the training and advice, and defines the fraudulent methods (facts), desires (values) + regrets, recommended actions, and self-efficacy mentioned above.

[0072] Next, the narrative text generation unit 54b generates a feedback document according to the selected narrative type (for example, first-person (victim) and failure experience type). For example, the narrative text generation unit 54b incorporates the narrative type and recommended actions into the base prompt stored in the base prompt storage unit 54f, and inputs the base prompt to a generation AI model 54e such as LLM (Large Language Models), thereby generating a narrative text.

[0073] Here, the base prompts include, for example, "In order to raise crime prevention awareness, be as specific as possible and use expressions that appeal to emotions," and include "instructions" that incorporate narrative premises, "character setting" that incorporates a narrative narrator, "fraudulent methods" that specify the training content, "recommended actions" that specify identified recommended actions, "structure" that incorporates narrative structure, and "output examples" that specify output examples such as narrative styles.

[0074] The narrative sentence generation unit 54b can generate more natural and effective narrative sentences by acquiring sentence examples similar to the target sentence from the narrative sentence example storage unit 54d and further incorporating them into the base prompt. For example, the narrative sentence example storage unit 54d stores sentence examples in which narrative types are associated with related information. Here, the related information includes, for example, "characteristics (psychological characteristics and behavioral characteristics)" and "attributes (e.g., having children)" that the corresponding narrative sentence has a high behavioral change effect, and "recommended actions" included in the corresponding narrative sentence.

[0075] The narrative text generator 54b then retrieves relevant example narrative texts using, for example, trainee information such as characteristics and recommended actions as a query. For example, the narrative text generator 54b selects example narrative texts with more matching related information. The generative AI model can also be fine-tuned with highly effective narratives.

[0076] As described above, the narrative processing unit 54 selects a narrative and generates a narrative text, and outputs the narrative type and the narrative text to the avatar generation unit 55.

[0077] (Avatar generation) The avatar generation unit 55 is a processing unit that generates an avatar that speaks to the trainee the "narrative type" or "narrative sentences" that are feedback methods generated by the narrative processing unit 54 or the like.

[0078] Fig. 14 is a diagram illustrating details of avatar generation unit 55. As shown in Fig. 14, avatar generation unit 55 has prompt storage unit 55a, voice generation AI model 55b, and image generation AI model 55c. The prompt storage unit 55a, voice generation AI model 55b, and image generation AI model 55c store data.

[0079] The avatar patterns generated here include "expert, victim, victim's family, perpetrator," etc. For example, for "expert," the avatar generation unit 55 uses a real person as the avatar, and for "victim, victim's family," it uses a fictional character representing a person in the same position, a related person, or a person similar to a related person, or an avatar incorporating the characteristics of the trainee. It is known that when the main character in a narrative (for example, the narrator or the protagonist of the story) is highly similar to oneself, identification occurs, which increases the persuasive effect.

[0080] Next, the avatars generated by the avatar generation unit 55, namely, "a real person," "a fictional person," and "a fictional person incorporating the characteristics of the trainee," will be described.

[0081] For example, in the case of a "real person," the avatar generation unit 55 omits avatar image generation and acquires an image of the target person (for example, an expert or knowledgeable person) from a storage unit (not shown).

[0082] Furthermore, the avatar generation unit 55 inputs the voice data of the target person (expert, knowledgeable person) and the narrative text generated by the narrative processing unit 54 into a voice generation AI model 55b that has been trained in advance using the trainee's voice data, and synthesizes avatar voice, which is voice data output by the avatar. Note that when the avatar generation unit 55 uses the zero-shot method (zero-shot text-to-speech), it is not necessary to use the trained voice generation AI model 55b, and it is also possible to generate avatar voice by inputting the "narrative text" and "reference voice data" into a general-purpose model.

[0083] Thereafter, the avatar generation unit 55 inputs the avatar image and avatar voice into a pre-trained video generation AI model 55c to generate a narrative video. Here, the inputs to the video generation AI model 55c are an "image of the target person" and the generated "avatar voice," and the output is a video in which the facial expression of the person in the image (e.g., eyes and mouth) changes in accordance with the voice data. Note that by inputting a video of the target person speaking into the video generation AI model 55c as reference data, improved accuracy and a more natural finish can be expected.

[0084] Next, in the case of a "fictional character," the avatar generation unit 55 generates an avatar image that is different from a "real person" by inputting a prompt stored in the prompt memory unit 55a based on the narrative type information identified by the narrative processing unit 54.

[0085] Specifically, the avatar generation unit 55 acquires a prompt corresponding to the narrative type from the prompt storage unit 55a, and inputs the acquired prompt (instruction text for the generated image) into the image generation AI model to generate an avatar image. As another example, the avatar generation unit 55 can also adopt a method of selecting from a group of avatar images generated in advance for each narrative. Note that the avatar voice for a "fictional character" is the same as that for a "real person," so a detailed description will be omitted.

[0086] Next, in the case of a "fictional character incorporating the trainee's characteristics," the avatar generation unit 55 incorporates the trainee's characteristics into both the avatar image and the avatar voice, so that the avatar has characteristics similar to those of the user, making it easier for the user to empathize with the trainee and to induce behavioral changes.

[0087] For example, the avatar generation unit 55 incorporates the trainee's age and attributes (e.g., gender, whether or not they have children) into a prompt and inputs this into an image generation AI model, thereby generating an avatar image that incorporates the trainee's external characteristics.

[0088] Furthermore, the avatar generation unit 55 generates an avatar voice that incorporates the trainee's characteristics through a synthesis process of the avatar voice of the "real person." FIG. 15 is a diagram explaining the details of avatar voice synthesis. As shown in FIG. 15, the avatar generation unit 55 executes a general-purpose voice synthesis process on the narrative text to generate voice data that does not include the trainee's characteristics. Next, the avatar generation unit 55 uses a technique such as "Prosody Transfer" to generate an avatar voice that incorporates the trainee's speaking characteristics from the voice data that does not include the trainee's characteristics and the trainee's actual voice data.

[0089] Generally, voice has elements of speaking style (pronunciation, intonation, pauses) and voice quality. Personality influences speaking style, for example, a careful person will speak slowly and carefully, so it is thought that by making someone's speaking style similar, it becomes easier to feel that they are similar to one another. On the other hand, voice quality has little correlation with personality, etc. Also, people tend to feel uncomfortable if the other person has the same voice as themselves. For these reasons, the avatar generation unit 55 incorporates only speaking style characteristics into the avatar.

[0090] (feedback) The presentation unit 56 is a processing unit that uses the identified feedback method to present information to the trainee, including information to motivate the trainee to take the identified recommended behavior. Specifically, the presentation unit 56 displays the avatar image generated by the avatar generation unit 55 on the output unit 12, and then displays the avatar image on the trainee's terminal or the like. In other words, the presentation unit 56 presents a narrative to the trainee.

[0091] FIG. 16 is a diagram illustrating a specific example of a narrative. As shown in FIG. 16, presentation unit 56 presents an avatar video in which avatar image 300 reads avatar voice 301 (narrative text). Here, avatar image 300 is an image of one of a "real person," a "fictional character," or a "fictional character incorporating the trainer's characteristics" generated according to the trainer's narrative type. Avatar voice 301 is a narrative text in the form of a "victim's personal story" corresponding to the trait "high empathy," incorporating "concerns about and regrets about one's son" corresponding to the trainer's trait "family values," as well as identified recommended actions such as "consulting family or the police, and taking precautions."

[0092] (effect) As described above, the information processing device 10 identifies the characteristics (psychological characteristics and behavioral characteristics) of the trainee through training, and identifies recommended actions for the trainee's goal based on the characteristics. The information processing device 10 then generates a narrative (story) including motivation according to the recommended actions and the characteristics, and presents it to the trainee. In other words, the actions required for the trainee to achieve the trainee's goal are made clear, and the motivation effect is improved by appealing to the trainee's emotions through the narrative, thereby creating conditions sufficient for behavioral change and increasing the possibility of behavioral change.

[0093] Furthermore, the information processing device 10 can identify the characteristics that the trainee is aware of based on the questionnaire and the characteristics that the trainee is not aware of based on changes in vital signs during training, and can provide feedback that takes into account each of these characteristics, thereby enhancing the effectiveness of changing the trainee's awareness after feedback.

[0094] Furthermore, the information processing device 10 can provide feedback in the form of narrative text that includes values ​​and desires that the trainee is subconsciously aware of, and can therefore provide feedback of specific advice that will lead to behavioral changes in the trainee.

[0095] Furthermore, the information processing device 10 can generate an avatar image according to the characteristics of the trainee and provide feedback using the avatar image, thereby making it possible to provide feedback that appeals to the trainee's emotions.

[0096] Furthermore, the information processing device 10 can generate avatar voice according to the trainee's characteristics and execute a narrative using the avatar voice, thereby making it possible to appeal to the trainee's emotions more effectively than when a narrative is executed using a simple avatar image.

[0097] Furthermore, the information processing device 10 can execute a narrative using an avatar image that incorporates the trainee's characteristics, if necessary, thereby appealing to the trainee's emotions and enhancing the effectiveness of the trainee's behavioral change. [Example]

[0098] Although the embodiments of the present invention have been described above, the present invention may be embodied in various different forms other than the above-described embodiments.

[0099] (Numbers, etc.) The formulas, attribute names, models, numerical values, narrative types, narrative content, prompts, etc. used in the above examples are merely examples and can be changed as desired. The process flow described in each flowchart can also be changed as appropriate within a consistent range. Furthermore, models such as image generation AI models can use generally publicly available machine learning models. Furthermore, a person in the same position, for example, refers to a victim; a related party refers to a victim's family or a perpetrator; and a person similar to a related party refers to a victim's family, rather than the trainee's family, if the trainee is determined to be a victim.

[0100] (Narrative) In the above embodiment, "narrative" is used as an example of feedback, but the meaning of "narrative" is not limited to "narrative." For example, "narrative" and "feedback" can include various meanings, including appeals to specific actions and emotions according to the characteristics of the trainee.

[0101] (Target audience) In the above embodiment, the trainees of special frauds are used as an example, but the present embodiment is not limited to this. For example, the training is not limited to special frauds, but various evaluation contents based on dialogue can be targeted, such as mental care through training in customer harassment, improving engagement through one-on-one video analysis, improving supervisor skills through video and conversation, and improving skills through counseling analysis.

[0102] (system) The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings may be changed arbitrarily unless otherwise specified.

[0103] Furthermore, the specific form of distribution or integration of the components of each device is not limited to that shown in the figure. For example, the training execution unit 40 and the feedback unit 50 may be integrated. That is, all or part of the components may be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions of each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0104] Furthermore, all or any part of the processing functions performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0105] (Hardware) Fig. 17 is a diagram illustrating an example of a hardware configuration. As shown in Fig. 17, an information processing device 10 includes a communication device 10a, an HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. The components shown in Fig. 17 are connected to each other via a bus or the like.

[0106] The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and DBs that operate the functions shown in FIG.

[0107] The processor 10d reads out a program that executes the same processes as the respective processing units shown in FIG. 5 from the HDD 10b or the like and loads it into the memory 10c, thereby operating a process that executes the respective functions described in FIG. 4 or the like. For example, this process executes the same functions as the respective processing units of the information processing device 10. Specifically, the processor 10d reads out a program having the same functions as the training execution unit 40, the feedback unit 50, or the like from the HDD 10b or the like. Then, the processor 10d executes a process that executes the same processes as the training execution unit 40, the feedback unit 50, or the like.

[0108] In this way, the information processing device 10 operates as an information processing device that executes a feedback method by reading and executing a program. The information processing device 10 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, the above-described embodiment may also be applied in the same way to cases where another computer or server executes the program, or where these execute the program in cooperation with each other.

[0109] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and may be read out from the recording medium and executed by a computer. [Explanation of symbols]

[0110] 10. Information processing equipment 11 Communications Department 12 Output section 20 Memory section 21 Conversation Information DB 22 Physiological Data DB 23 Conversation Model 30 Control Unit 40 Training Execution Department 50 Feedback section 51 User information acquisition unit 52 Characteristic identification part 53 Recommended Action Identification Department 54 Narrative Processing Unit 54a Narrative Selection Section 54b Narrative Text Generation Unit 55 Avatar Generation Unit 56 Presentation section

Claims

1. On the computer, Identifying characteristics of the subject from the subject's behavior during the dialogue; Identifying a recommended action for the subject and a feedback method corresponding to the subject's characteristics based on the identified characteristics of the subject; Using the identified feedback method, presenting information including motivation for the identified recommended behavior to the subject to the subject; A feedback processing program that causes a process to be executed.

2. The process of identifying the characteristics includes: Identifying, as characteristics of the subject, psychological characteristics including the subject's personality traits and at least one of the values ​​or desires that the subject regards as important, from the content of the responses to the questions given to the subject; 2. The feedback processing program according to claim 1.

3. The process of identifying the characteristics includes: Based on at least one of the dialogue content and vital information during the dialogue, an analysis of the content of the utterances, a quantitative analysis of the utterances, or an analysis combining the utterances and the vital information is performed; Identifying behavioral characteristics, which are characteristic behaviors performed by the subject, from the analysis results.

3. The feedback processing program according to claim 1, wherein the feedback processing program is a program for processing a plurality of inputs.

4. The process of identifying a feedback method includes: Identifying a narrative type that defines a narrator of feedback information that includes motivation for the subject's recommended behavior and a narrative structure of the feedback information based on the characteristics of the subject; The process to be presented includes: Using the identified narrative type, the feedback information is fed back to the subject by speaking it.

4. The feedback processing program according to claim 3.

5. The narrator is a person in the same position as the subject, a person related to the subject, a person similar to the person related to the subject, or an expert or knowledgeable person in the subject of the dialogue, The configuration includes emotional and motivational factors that contribute to behavioral change toward the goal of the interaction.

5. The feedback processing program according to claim 4.

6. The narrative type, the recommended action, and the narrative type example sentences are set as base prompts and input to a trained generative model, and an output result of the generative model is used to generate a narrative sentence that includes a motivation for the subject to take the recommended action; The process to be presented includes: providing feedback to the subject using video data that speaks the generated narrative text; 6. The feedback processing program according to claim 5.

7. generating an avatar image and an avatar voice according to the identified narrative type; generating video data by combining the avatar image and the avatar voice; The process to be presented includes: Using the video data, the narrative text is spoken to the subject.

7. The feedback processing program according to claim 6.

8. The generating process includes: generating the avatar image based on the narrative type, the avatar image being one of a real person, a fictional character, and a fictional character incorporating the characteristics of the subject; generating the avatar voice in response to the generated avatar image; generating the video data to move the eyes and mouth of the avatar image so that the generated avatar voice speaks the narrative structure; 8. The feedback processing program according to claim 7, wherein:

9. The computer Identifying characteristics of the subject from the subject's behavior during the dialogue; Identifying a recommended action for the subject and a feedback method corresponding to the subject's characteristics based on the identified characteristics of the subject; Using the identified feedback method, presenting information including motivation for the identified recommended behavior to the subject to the subject; A feedback processing method characterized by executing a process.

10. Identifying characteristics of the subject from the subject's behavior during the dialogue; Identifying a recommended action for the subject and a feedback method corresponding to the subject's characteristics based on the identified characteristics of the subject; Using the identified feedback method, presenting information including motivation for the identified recommended behavior to the subject to the subject; An information processing device characterized by causing a control unit to execute a program.

Images