Psychological characteristic analysis device, psychological characteristic analysis method, psychological characteristic analysis program, and recording medium

Abstract

Provided is a psychological characteristic analysis device capable of analyzing psychological characteristics of a user.　This psychological characteristic analysis device includes: a conversation log acquisition unit that acquires a conversation log of a user; a state analysis information acquisition unit that acquires state analysis information during a conversation of the user; and an output unit that, on the basis of the conversation log and / or the state analysis information, outputs a psychological state analysis in which changes in the psychological state of the user during the conversation are analyzed, and psychological characteristic information for the user in which the conversation log and the changes in psychological state are associated.

Description

Psychological characteristic analysis device, psychological characteristic analysis method, psychological characteristic analysis program, and recording medium

[0001] The present disclosure relates to a psychological characteristic analysis device, a psychological characteristic analysis method, a psychological characteristic analysis program, and a recording medium.

[0002] Attempts have been made to figure out the user's emotions and the like using wearable sensors and the like. For example, in Patent Document 1, based on a history accumulation database storing the user's biological information acquired in advance, the user's emotion information corresponding to this biological information, and the physical state, the relationship between the biological information and the emotion information is learned, and an estimator generation unit that generates an estimator for estimating emotion information from biological information for each physical state, a state determination unit that determines the physical state of the user based on the user's position information and the biological information of the user detected at the time when this position information is acquired, and an emotion determination unit that estimates the emotion information of the user by using the estimator corresponding to the physical state of the user determined among the estimators for each physical state created above to estimate the emotion information of the user from the biological information of the user detected. An emotion information estimation device is described.

[0003] Japanese Patent Application Laid-Open No. 2016-106689

[0004] In the invention as described in Patent Document 1, only the user's superficial emotions can be estimated, and a method for analyzing more specific psychological characteristics of the user is required.

[0005] Therefore, the purpose of the present disclosure is to provide a psychological characteristic analysis device, a psychological characteristic analysis method, a psychological characteristic analysis program, and a recording medium that can analyze the psychological characteristics of a user.

[0006] To achieve the above objective, the psychological characteristics analysis device of this disclosure includes a dialogue log acquisition unit, a state analysis information acquisition unit, a psychological state analysis unit, and an output unit, wherein the dialogue log acquisition unit acquires a user's dialogue log, the dialogue log is information that records the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition unit acquires state analysis information during the user's dialogue, the state analysis information includes information that links time information with at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis unit analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output unit outputs the user's psychological characteristics information that links the dialogue log and the progression of the psychological state.

[0007] The psychological characteristics analysis method of this disclosure includes a dialogue log acquisition step, a state analysis information acquisition step, a psychological state analysis step, and an output step, wherein each step is performed by a computer. The dialogue log acquisition step acquires a user's dialogue log, the dialogue log is information that records the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition step acquires state analysis information during the user's dialogue, the state analysis information includes information that links time information with at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis step analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output step outputs the user's psychological characteristics information that links the dialogue log and the progression of the psychological state.

[0008] The psychological characteristics analysis program of this disclosure includes a dialogue log acquisition procedure, a state analysis information acquisition procedure, a psychological state analysis procedure, and an output procedure, wherein the dialogue log acquisition procedure acquires a user's dialogue log, the dialogue log is information that records the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition procedure acquires state analysis information during the user's dialogue, the state analysis information includes information that links time information with at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis procedure analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output procedure outputs the user's psychological characteristics information that links the dialogue log and the progression of the psychological state.

[0009] The recording medium of this disclosure is a computer-readable recording medium that records a psychological characteristics analysis program for causing a computer to execute each of the following procedures: a dialogue log acquisition procedure, a state analysis information acquisition procedure, a psychological state analysis procedure, and an output procedure; the dialogue log acquisition procedure acquires a user's dialogue log, the dialogue log is information that records the dialogue between the user and the user's dialogue partner in chronological order; the state analysis information acquisition procedure acquires state analysis information during the user's dialogue, the state analysis information includes information that links time information with at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue; the psychological state analysis procedure analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information; and the output procedure outputs the user's psychological characteristics information that links the dialogue log and the progression of the psychological state.

[0010] According to this disclosure, it is possible to analyze the psychological characteristics of users.

[0011] Figure 1 is a block diagram showing the configuration of an example of the psychological trait analysis device of this disclosure. Figure 2 is a block diagram showing an example of the hardware configuration of the psychological trait analysis device of this disclosure. Figure 3 is a flowchart showing an example of processing in the psychological trait analysis device of this disclosure. Figure 4 is a block diagram showing the configuration of an example of the psychological trait analysis device of this disclosure. Figure 5 is a flowchart showing an example of processing in the psychological trait analysis device of this disclosure. Figure 6 is a block diagram showing the configuration of an example of the surrogate existence model construction device of this disclosure. Figure 7 is a block diagram showing an example of the hardware configuration of the surrogate existence model construction device of this disclosure. Figure 8 is a flowchart showing an example of processing in the surrogate existence model construction device of this disclosure.

[0012] Next, embodiments of the present disclosure will be described with reference to the drawings. The present disclosure is not limited to the following embodiments. In the following drawings, the same parts are denoted by the same reference numerals. Furthermore, unless otherwise specified, the descriptions of each embodiment can be used interchangeably with those of the others, and unless otherwise specified, the configurations of each embodiment can be combined.

[0013] [Embodiment 1] The psychological characteristics analysis device of this embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of an example of the psychological characteristics analysis device 10 of this embodiment. As shown in Figure 1, the psychological characteristics analysis device 10 (hereinafter also referred to as "this device 10") includes a dialogue log acquisition unit 11, a state analysis information acquisition unit 12, a psychological state analysis unit 13, and an output unit 14. In addition, although not shown, this device 10 may also include, for example, an input unit, an output unit, a display unit and / or a storage unit.

[0014] The device 10 may be, for example, a single device including the aforementioned parts, or it may be a device in which the aforementioned parts can be connected via a communication network. Furthermore, the device 10 can be connected to an external device described later via a communication network. The communication network is not particularly limited and a known network can be used, for example, it may be wired or wireless. Examples of communication networks include the Internet, WWW (World Wide Web), telephone lines, LAN (Local Area Network), SAN (Storage Area Network), DTN (Delay Tolerant Networking), LPWA (Low Power Wide Area), L5G (Local 5G), etc. Examples of wireless communication include Wi-Fi®, Bluetooth®, Local 5G, LPWA, etc. The aforementioned wireless communication may be in the form of direct communication between devices (Ad Hoc communication), infrastructure communication, or indirect communication via an access point. The device 10 may, for example, be incorporated into a server as a system. The device 10 may also be, for example, a personal computer (PC, e.g., desktop or notebook), smartphone, tablet terminal, etc., on which the program disclosed herein is installed. Furthermore, the device 10 may be in the form of cloud computing or edge computing, for example, in which at least one of the aforementioned parts is on a server and the other parts are on a terminal.

[0015] Figure 2 illustrates a block diagram of the hardware configuration of the device 10. The device 10 includes, for example, a central processing unit 101, memory 102, bus 103, storage device 104, input device 105, output device 106, communication device (communication unit) 107, etc. Each part of the device 10 is interconnected via the bus 103 through its respective interface (I / F).

[0016] The central processing unit 101 operates in coordination with other components via controllers (system controller, I / O controller, etc.) and is responsible for the overall control of the device 10. In the device 10, the central processing unit 101 executes, for example, the program disclosed herein or other programs, and also reads and writes various types of information. Specifically, for example, the central processing unit 101 functions as an interaction log acquisition unit 11, a state analysis information acquisition unit 12, a psychological state analysis unit 13, and an output unit 14. The device 10 may also include other computing devices such as a CPU, GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), or a combination thereof as computing devices.

[0017] Bus 103 can also be connected to external devices, for example. Examples of such external devices include external storage devices (external databases, etc.), electrocardiographs, printers, external input devices, external display devices, audio output devices such as speakers, external imaging devices such as cameras, and various sensors such as acceleration sensors, geomagnetic sensors, and direction sensors. The device 10 can be connected to an external network (the aforementioned communication network) by a communication device 107 connected to bus 103, for example, and can also be connected to other devices via the external network.

[0018] Memory 102 may be, for example, main memory. When the central processing unit 101 performs processing, memory 102 reads various operational programs, such as the program of this disclosure, stored in the storage device 104 (described later), and the central processing unit 101 receives data from memory 102 and executes the program. The main memory may be, for example, RAM (random access memory). Alternatively, memory 102 may be, for example, ROM (read-only memory).

[0019] The storage device 104 is also called an auxiliary storage device, for example, in relation to the main memory (primary memory). As described above, the storage device 104 stores an operating program including the program of this disclosure. The storage device 104 may be, for example, a combination of a recording medium and a drive for reading and writing to the recording medium. The recording medium is not particularly limited and may be internal or external, for example, an HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, memory card, etc. The storage device 104 may be, for example, a hard disk drive (HDD) in which the recording medium and the drive are integrated, or a solid state drive (SSD). If the device 10 includes, for example, the storage device 104 functions as the storage unit.

[0020] In this device 10, the memory 102 and storage device 104 can also store various types of information, such as log information, information obtained from an external database (not shown) or external devices, information generated by this device 10, and information used by this device 10 when executing processing. At least some of the information may be stored on an external server other than the memory 102 and storage device 104, or it may be stored in a distributed manner across multiple terminals using blockchain technology or the like.

[0021] The device 10 further includes, for example, an input device 105 and an output device 106. The input device 105 may include, for example, a pointing device such as a touch panel, trackpad, or mouse; a keyboard; imaging means such as a camera or scanner; a card reader such as an IC card reader or magnetic card reader; an audio input means such as a microphone; and so on. The output device 106 may include, for example, a display device such as an LED display or liquid crystal display; an audio output device such as a speaker; a printer; and so on. In this embodiment 1, the input device 105 and the output device 106 are configured separately, but the input device 105 and the output device 106 may be configured as an integrated unit, such as a touch panel display.

[0022] Next, an example of the psychological characteristics analysis method of this embodiment will be described based on the flowchart in Figure 3. The psychological characteristics analysis method of this embodiment can be carried out, for example, using the psychological characteristics analysis device 10 shown in Figures 1 and 2, as follows. Note that the psychological characteristics analysis method of this embodiment is not limited to the use of the psychological characteristics analysis device 10 shown in Figures 1 and 2.

[0023] The dialogue log acquisition unit 11 acquires the user's dialogue log (S1, dialogue log acquisition step). The dialogue log is information that records the dialogue between the user and the user's dialogue partner in chronological order. Specifically, the dialogue log includes, for example, the user's utterance information and the dialogue partner's utterance information as dialogue information, and each piece of utterance information is associated with utterance time information. The utterance time information may be embedded as metadata, for example. The dialogue information recorded in the dialogue log may be, for example, text data or audio data, or a combination thereof (for example, audio data recorded from a voice conversation and the transcribed text of the voice conversation). The dialogue partner is not particularly limited and may be, for example, a person other than the user or a dialogue model. The dialogue model is not particularly limited and may be, for example, a mechanism that includes an algorithm or data structure for processing dialogue between the user and a system (machine, software, etc.). The dialogue model may be, for example, the proxy existence model described later in Embodiment 4. In this disclosure, the “proxy existence model” is, for example, a machine learning model that has been trained to behave like a specific person. The method for manufacturing the proxy existence model is not particularly limited and can be manufactured by any method, but for example, it can be manufactured by the method for manufacturing the proxy existence model described in this disclosure, which will be described later. The dialogue log acquisition unit 11 may, for example, acquire the dialogue log from a recording medium on which the dialogue log is recorded, or it may acquire the dialogue log from a dialogue device used by the user to interact with the dialogue partner (for example, the user's terminal device, the system running the dialogue model).

[0024] The state analysis information acquisition unit 12 acquires state analysis information during the user's interaction (S2, state analysis information acquisition step). The state analysis information includes information that is linked to time information, with at least one selected from the group consisting of the user's image information, the user's voice information, the user's vital information, and the user's movement information during the interaction. The time information is, for example, information about the time when the state analysis information was acquired, and may also be a timestamp. The state analysis information acquisition unit 12 may, for example, acquire the state analysis information from a recording medium in which the state analysis information is recorded linked to a timestamp, or it may acquire the state analysis information from the user's terminal device such as a PC, smartphone, tablet terminal, or wearable terminal, a webcam, a microphone, etc.

[0025] The user's image information is, for example, an image of the user during a conversation captured by an imaging device such as a camera. The image may be a still image or a video. The imaging device is not particularly limited and may include, for example, the user's PC, smartphone, tablet, wearable device, or other terminal device, or a webcam.

[0026] The user's voice information is, for example, information recorded from the user's voice during a conversation. The voice may be, for example, the user's voice, or it may include the voice of the person the user is talking to. The voice recording device is not particularly limited and may include, for example, the user's PC, smartphone, tablet, wearable device, or other terminal device, a microphone, etc.

[0027] The user's vital information includes, for example, the user's biometric information, such as heart rate information, blood pressure, body temperature, electrocardiogram data, sleep data, and stress level. The heart rate information includes, for example, heart rate (HR) and heart rate variability (HRV). The sleep data includes, for example, information about the user's sleep, such as sleep duration and sleep score. The sleep score is, for example, an index for evaluating the quality of the user's sleep and can be evaluated from the user's REM and non-REM sleep transitions. The stress level includes, for example, information estimated by the wearable device regarding the user's stress state. The user's vital information may be, for example, information sensed by various sensors installed in the user's wearable device, or information estimated from information sensed by the wearable device.

[0028] The user's action information may be, for example, information about the user's movements, and may include information such as whether or not the user is performing an action (presence or absence of action, frequency of action), or information about the type of action the user is performing. The type of action is not particularly limited and may include, for example, walking, posture, gestures, etc. The action information may be obtained, for example, from the operation history of an input device, from sensors of a wearable terminal (for example, an accelerometer, gyroscope, geomagnetic sensor, etc.), or from information estimated from the image information. When obtaining action information from image information, it can be obtained, for example, by analyzing the image information using a known motion analysis algorithm, posture estimation model, etc. Furthermore, if the interaction was conducted via the user's terminal, the operation information of the terminal may be used as the user's action information.

[0029] The state analysis information acquisition unit 12 may, for example, acquire user blinking information, gaze information, facial expression information, or motion information based on the image information. The blinking information is, for example, information about the user's blinking, and includes information such as blinking frequency, number of blinks, and blinking pattern. The gaze information is, for example, information about the user's gaze, and includes information such as gaze fixation frequency, gaze direction, gaze movement speed, and gaze movement frequency. The facial expression information is, for example, information about the user's facial expression. The state analysis information acquisition unit 12 can acquire blinking information, gaze information, facial expression information, or motion information by, for example, using facial expression estimation technology, face recognition technology, etc. in known image analysis software.

[0030] The psychological state analysis unit 13 analyzes the changes in the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information (S3, psychological state analysis step). The psychological state analysis unit 13 can, for example, analyze how the analyzed psychological state changed during the dialogue by linking it with the time information contained in the state analysis information, based on the time series of the dialogue log and the time information contained in the state analysis information. The psychological state analysis unit 13 can, for example, analyze the changes in the level of concentration during the dialogue by synchronizing the speech information of the dialogue log with the estimated level of concentration, based on the speech time information of the dialogue log and the time information contained in the state analysis information.

[0031] The psychological state analysis unit 13 can, for example, estimate the degree of concentration during a conversation as the psychological state, and analyze the changes in the degree of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information. In this case, the psychological state analysis unit 13 can estimate the user's degree of concentration based on, for example, at least one selected from the group consisting of the user's heart rate, movement information, blinking information, gaze information, and facial expression information. The degree of concentration may be a qualitative evaluation, such as whether or not the user is concentrating, or a quantitative evaluation, such as what level the user's level of concentration is.

[0032] It is known that people enter a state of high stress when they are in a state of concentration. For this reason, the psychological state analysis unit 13 may estimate the user's stress based on, for example, heart rate variability (HRV), and estimate that the user is in a state of concentration if the stress load exceeds a threshold. Stress estimation based on heart rate will be described later. The stress load threshold may be stored, for example, in the memory unit of this device 10, or it may be stored on an external recording medium. The threshold is not particularly limited, and for example, any value can be set.

[0033] It is known that when a person is concentrating, their gaze is directed towards the object they are concentrating on, and their facial expression becomes tense. For this reason, the psychological state analysis unit 13 may estimate the degree of concentration of the subject based on, for example, at least one of the gaze information and the facial expression information. When estimating the degree of concentration of the subject based on gaze information, the psychological state analysis unit 13 may, for example, analyze the user's gaze information based on the image information and estimate that the user is in a state of concentration if the frequency of gaze movement falls below a threshold or the time of fixed gaze exceeds a threshold. When estimating the degree of concentration of the user based on facial expression information, the psychological state analysis unit 13 may, for example, analyze the user's facial expression information based on the image information and estimate that the user is in a state of concentration if the user's facial expression is tense.

[0034] It is known that when a person is in a state of concentration, their work efficiency increases, and their frequency of operations and body movements improves. For this reason, the psychological state analysis unit 13 may estimate the degree of concentration of the subject based on the movement information, for example. The psychological state analysis unit 13 can, for example, analyze the user's movement state based on the movement information and estimate that the user is in a state of concentration if the user's body movements or the frequency of operating the equipment exceeds a predetermined threshold.

[0035] The psychological state analysis unit 13 may, for example, estimate the user's level of concentration during the conversation using various known methods for estimating concentration levels. Examples of known methods for estimating concentration levels include, but are not limited to, the methods described in References 1 and 2 below. Reference 1: "Verification of the Accuracy of Estimating Concentration State Using Pulse Rate and Body Movement," [online], June 2017, Subaru Uchida, Naoya Isoyama, Guillaume Lopez, Multimedia, Distributed, Cooperative and Mobile (DICOMO2017), [Retrieved January 22, 2025], Internet <URL: https: / / ipsj.ixsq.nii.ac.jp / ej / ?action=repository_action_common_download&item_id=190128&item_no=1&attribute_id=1&file_no=1> Reference 2: "Study on the Relationship between Concentration State and Biometric Information during PC Work," [online], November 17, 2018, Yuki Usami, Chikako Ishizawa, Yoichi Kageyama, Motoaki Shirasu, 61st Joint Conference on Automatic Control, [Retrieved January 22, 2025], Internet <URL: https: / / www.jstage.jst.go.jp / article / jacc / 61 / 0 / 61_25 / _pdf / -char / ja>

[0036] The psychological state analysis unit 13 may, for example, estimate the progression of emotions during a conversation as the psychological state, and analyze the progression of emotional information during the conversation based on the time series of the conversation log and the time information included in the state analysis information. The emotions can be estimated by a known emotion estimation engine based on, for example, image information such as the user's facial expressions; vocal information such as voiced and voiceless sounds; text information such as the content of speech; vital data information such as blood pressure and heart rate; etc. Specific examples of the emotion estimation engine are not particularly limited and include, for example, those that estimate emotions based on image information such as facial expressions, such as Affdex, Microsoft Azure Face API, Amazon Rekognition, Realeyes, and User Local facial expression estimation AI; those that estimate emotions based on voice information, such as STEmotion, Empath, BeyondVerbal's emotion estimation software, IBM Watson Tone Analyzer, and User Local speech emotion recognition AI; those that estimate emotions using natural language processing based on text information, such as User Local text emotion recognition AI; and those that estimate emotions based on vital data, such as NEC emotion analysis solution. The emotion estimation engine may be of one type, or two or more types may be used in combination. The emotion estimation engine may be an external component of the device 10, or it may be a component of the device 10. Specific examples of the emotion types include joy, anger, sadness, fear, disgust, content, and surprise. The emotion types may also be multiple emotions as shown in Russell's Circle of Emotion model. The psychological state analysis unit 13 can analyze the progression of emotions during the dialogue by synchronizing the utterance information in the dialogue log with the estimated emotions, for example, based on the utterance time information in the dialogue log and the time information included in the state analysis information.

[0037] The psychological state analysis unit 13 may, for example, further analyze the timing of awareness and understanding during the conversation based on the changes in at least one of the concentration level and the emotion as a progression of the user's psychological state. In this case, the psychological state analysis unit 13 can estimate that the user gained at least one of awareness or understanding during the conversation if, for example, at least one of the concentration level and the emotion changes by a certain amount per unit of time during the conversation.

[0038] The psychological state analysis unit 13 may, for example, estimate the stress during the conversation as the psychological state, and analyze the progression of the stress during the conversation based on the time series of the conversation log and the time information included in the state analysis information. The psychological state analysis unit 13 can estimate the user's stress during the conversation by, for example, using various known methods for estimating stress. Specifically, the psychological state analysis unit 13 can estimate the user's stress during the conversation based on, for example, heart rate variability (HRV). In general, heart rate variability (HRV), which is the periodic fluctuation of a person's heart rate, is known to be usable as an indicator of the state of the autonomic nervous system. It is known that in a person under a certain stress level, heart rate variability is smaller compared to a person in a non-stress state. Therefore, the psychological state analysis unit 13 can estimate that the user is experiencing stress if the user's heart rate variability during the conversation is below a threshold. The psychological state analysis unit 13 may also estimate the user's stress in a manner similar to, for example, the stress measurement method used in commercially available smartwatches. The psychological state analysis unit 13 can analyze the changes in the level of concentration during the conversation by synchronizing the speech information in the dialogue log with the estimated stress, for example, based on the speech time information in the dialogue log and the time information included in the state analysis information.

[0039] The psychological state analysis unit 13 may, for example, estimate the user's psychological state using a machine learning model. In this case, the machine learning model is, for example, a machine learning model that has been trained to output the user's psychological state when the state analysis information is input. The machine learning model may, for example, be recorded in the storage unit of the device 10, or it may be recorded on an external recording medium.

[0040] In this case, the psychological state analysis unit 13 may, for example, perform necessary preprocessing on the state analysis information before inputting the information into the machine learning model. Specifically, for example, each piece of information contained in the state analysis information may be converted into interval information divided into predetermined intervals based on time information, and the converted interval information may be input into the machine learning model. In this case, the psychological state analysis unit 13 may, for example, generate multiple types of feature vectors for each piece of interval information and use the generated feature vectors as input data for the machine learning model.

[0041] Furthermore, the psychological state analysis unit 13 may, for example, use AI to estimate the user's psychological state.

[0042] In this disclosure, AI refers to, for example, a large-scale learning model called a "foundation model." The foundation model is a machine learning model pre-trained on predetermined big data, and is not limited to large-scale language models (LLMs) that have learned natural language, but may also include large-scale speech models, large-scale image models, and multimodal models (such as visual language models) that handle language, images, speech, and video across different systems. Furthermore, configurations may be adopted that involve deploying a small-scale language model (SLM) on the terminal side to cooperate with a large-scale model on the cloud side, or configurations that include search extension generation (RAG) using external knowledge sources, tool execution / function calls, agent-oriented control logic, etc. The provider of the large-scale language model is not particularly limited, and examples include, but is not limited to, various LLMs / multimodal models provided by OpenAI, Anthropique, Alphabet (Google), META, Microsoft, Cohere, Mistral, xAI, NEC Corporation, NTT, etc.

[0043] When the mental state analysis unit 13 estimates the mental state of the user using AI, the mental state analysis unit 13 can cause the AI to analyze the mental state of the user by inputting analysis instruction information instructing the AI to analyze the mental state and at least one of the dialogue log and the state analysis information into the AI. The analysis instruction information may be recorded, for example, in the storage unit of the apparatus 10, may be recorded externally, or may be input by the user each time. Specific examples of the analysis instruction information include, for example, "Provide the dialogue log (data in which the chat history is associated with the time axis) and the state analysis information (at least one selected from the group consisting of the image information of the user during the dialogue, the voice information of the user during the dialogue, the vital information of the user during the dialogue, and the motion information of the user during the dialogue, which is data associated with the time information). Integrate the dialogue log and the state analysis information based on the time information associated with each piece of information. Then, analyze and output the transition of the user's mental state (the change in the fluctuation of mental characteristics) from the integrated dialogue log and the state analysis information." etc. Note that the present disclosure is not limited to the above examples.

[0044] Also, the AI may be, for example, a model fine-tuned for estimating the transition of the mental state. In this case, for example, fine-tuning for estimating the transition of the mental state can be performed by causing the AI to learn the analysis data obtained by analyzing the transition of the mental state during the dialogue for a set of the user's dialogue log and state analysis information.

[0045] More specifically, the mental state analysis unit 13 may prepare a large number of learning data sets obtained for a plurality of users. Each learning data set may include a series of dialogue logs and state analysis information associated with time information, and a series of labels indicating the transition of the mental state associated with the series. The mental state analysis unit 13 may input these learning data sets into the AI in mini-batch units, and may repeatedly update the parameters of the AI by minimizing the loss function based on the gradient descent method or its modified algorithm.

[0046] At this time, the AI may be, for example, a multi-layer neural network model having a large number of parameters (e.g., 100,000 or more, and in some cases, millions or more). In learning, a loss function based on the difference between the estimated value of the transition of the mental state and the label sequence is defined, and multiple iterations can be performed until the loss function becomes less than or equal to a predetermined value. Since such fine-tuning processing requires repeatedly executing a huge number of numerical operations at high speed, it is realistically impossible for a human to execute it in their head or with paper and pencil, and it relies on automatic processing by an electronic computer such as a processor or GPU.

[0047] Furthermore, during the fine-tuning, the mental state analysis unit 13 may apply a combination of known regularization methods such as L2 regularization, dropout, and early stopping in order to suppress overfitting to the learning data. As a result, it is possible to obtain a model that can estimate the transition of the mental state with high generalization performance even for unknown users and unknown dialogue scenarios, and the accuracy and robustness of mental state estimation can be improved compared to simple threshold determination and rule-based processing.

[0048] The output unit 14 outputs user mental characteristic information associating the dialogue log with the transition of the mental state (S4, output step). The output unit 14 outputs at least one selected from the group consisting of the dialogue log, the transition of the concentration, the transition of the emotion, and the transition of the stress. The output unit 14 may output, for example, by visualizing the mental characteristic information. Specifically, the output unit 14 can generate and output, for example, a so-called dashboard-form screen. In the dashboard, for example, a graph display function for visualizing the user's mental state (e.g., concentration, emotion, stress, timing of awareness / understanding) for each time series of the dialogue, a display function for analysis data such as the number of fluctuations in the mental state during the dialogue and the total concentration time, and browsing of a summary report or the like may be possible. Also, in the dashboard, for example, reference and download of raw data may be possible.

[0049] According to this disclosure, the dialogue log acquisition unit acquires the user's dialogue logs, the state analysis information acquisition unit acquires state analysis information during the user's dialogue, the psychological state analysis unit analyzes the changes in the user's psychological state during the dialogue based on at least one of the dialogue logs and the state analysis information, and the output unit outputs the user's psychological characteristics information linking the dialogue logs and the changes in psychological state. Therefore, the psychological characteristics analysis device of this disclosure can, for example, analyze and visualize specific psychological characteristics related to the user's dialogue. Therefore, the psychological characteristics analysis device of this disclosure can, for example, analyze psychological characteristics based on dialogue logs between a user learning something and an instructor, making it possible to visualize the timing at which the user gained understanding and insights during learning.

[0050] [Embodiment 2] Embodiment 2 is another example of a psychological trait analysis device.

[0051] The psychological characteristics analysis device of this embodiment is the same as the psychological characteristics analysis device 10 of Embodiment 1, except that it includes a dialogue response generation unit in addition to the configuration of the psychological characteristics analysis device 10 of Embodiment 1, and the description thereof can be applied accordingly.

[0052] Figure 4 is a block diagram showing an example configuration of the psychological characteristics analysis device 10A of this embodiment. As shown in Figure 4, the psychological characteristics analysis device 10A includes a dialogue response generation unit 15 in addition to the configuration of the psychological characteristics analysis device 10 of Embodiment 1. The hardware configuration of the psychological characteristics analysis device 10A is the same as that of the psychological characteristics analysis device 10 in Figure 2, except that the central processing unit 101 has the configuration of the psychological characteristics analysis device 10A in Figure 4 instead of the configuration of the psychological characteristics analysis device 10 in Figure 1.

[0053] The processing of the dialogue response generation unit 15 will be explained below with reference to Figure 5. Figure 5 is a flowchart showing an example of processing by the device 10A. The processing of the dialogue response generation unit 15 may be inserted at any appropriate position in the flowchart of Figure 3 described in the embodiment 1 above.

[0054] The dialogue response generation unit 15 inputs the psychological characteristics information into the dialogue model and causes the dialogue model to generate a dialogue response with the user (S11, dialogue response generation step).

[0055] The dialogue response generation unit 15 may select a dialogue model from a model pool in which dialogue models suitable for each psychological characteristic are recorded, based on the psychological characteristics. The model pool may be recorded, for example, in the storage unit of the device 10A, or on an external recording medium. In this case, the dialogue response generation unit 15 can, for example, generate a dialogue response using the selected dialogue model.

[0056] According to this disclosure, for example, the dialogue response unit can input the psychological characteristics information into a dialogue model, causing the dialogue model to generate a dialogue response with the user. Therefore, according to this disclosure, it becomes possible to provide a more appropriate response to the user.

[0057] [Embodiment 3] The psychological characteristics analysis program of this embodiment is a program that causes a computer to execute each step of the psychological characteristics analysis method described above. Specifically, the psychological characteristics analysis program of this embodiment is a program that causes a computer to execute the dialogue log acquisition procedure, the state analysis information acquisition procedure, the psychological state analysis procedure, and the output procedure.

[0058] The dialogue log acquisition procedure acquires a user's dialogue log, the dialogue log is information that records the dialogue between the user and the person the user is talking to in chronological order, the state analysis information acquisition procedure acquires state analysis information during the user's dialogue, the state analysis information includes information that links time information with at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis procedure analyzes the changes in the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output procedure outputs the user's psychological characteristics information that links the dialogue log and the changes in the psychological state.

[0059] Furthermore, the psychological characteristics analysis program of this embodiment can also be described as a program that causes a computer to function as a procedure for acquiring dialogue logs, a procedure for acquiring information for state analysis, a procedure for analyzing a psychological state, and an output procedure.

[0060] The psychological characteristics analysis program of this embodiment can be adapted from the descriptions in the psychological characteristics analysis apparatus and psychological characteristics analysis method of the present disclosure. Each of the above steps can be read as, for example, "step" or "process". The program of this embodiment may also be recorded on, for example, a computer-readable recording medium. The recording medium is not particularly limited and includes, for example, random access memory (RAM), read-only memory (ROM), hard disk (HD), flash memory (e.g., SSD (Solid State Drive), USB flash memory, SD / SDHC card, etc.), optical disc (e.g., CD-R / CD-RW, DVD-R / DVD-RW, BD-R / BD-RE, etc.), magneto-optical disk (MO), floppy disk (FD), etc. The psychological characteristics analysis program of this embodiment (for example, also called a programming product or psychological characteristics analysis program product) may also be delivered, for example, from an external computer. The aforementioned "distribution" may be, for example, distribution via a communication network, or distribution via a wired connected device. The psychological characteristics analysis program of this embodiment may be installed and executed on the distributed device, or it may be executed without being installed.

[0061] [Embodiment 4] The proxy existence model manufacturing apparatus of this embodiment will be described with reference to Figure 6. Figure 6 is a block diagram showing the configuration of an example of the proxy existence model manufacturing apparatus 20 of this embodiment. As shown in Figure 6, the proxy existence model manufacturing apparatus 20 (hereinafter also referred to as "this apparatus 20") includes a knowledge information acquisition unit 21, a construction information extraction unit 22, and a model construction unit 23. In addition, although not shown, this apparatus 20 may also include, for example, an input unit, an output unit, a display unit and / or a storage unit.

[0062] The device 20 may be, for example, a single device including the aforementioned parts, or it may be a device in which the aforementioned parts can be connected via a communication network. Furthermore, the device 20 can be connected to an external device described later via a communication network. The communication network is not particularly limited and a known network can be used, for example, it may be wired or wireless. Examples of communication networks include the Internet, WWW (World Wide Web), telephone lines, LAN (Local Area Network), SAN (Storage Area Network), DTN (Delay Tolerant Networking), LPWA (Low Power Wide Area), L5G (Local 5G), etc. Examples of wireless communication include Wi-Fi®, Bluetooth®, Local 5G, LPWA, etc. The aforementioned wireless communication may be in the form of direct communication between devices (Ad Hoc communication), infrastructure communication, or indirect communication via an access point. The device 20 may, for example, be incorporated into a server as a system. Alternatively, the device 20 may be, for example, a personal computer (PC, e.g., desktop or notebook), smartphone, or tablet terminal on which the program disclosed herein is installed. Furthermore, the device 20 may be in the form of cloud computing or edge computing, for example, in which at least one of the aforementioned parts is on a server and the other parts are on a terminal.

[0063] Figure 7 illustrates a block diagram of the hardware configuration of the device 20. The device 20 includes, for example, a central processing unit 201, memory 202, bus 203, storage device 204, input device 205, output device 206, communication device (communication unit) 207, etc. Each part of the device 20 is interconnected via the bus 203 through its respective interface (I / F).

[0064] The central processing unit 201 operates in coordination with other components via controllers (system controller, I / O controller, etc.) and is responsible for the overall control of the device 10. In the device 20, the central processing unit 201 executes, for example, the program of this disclosure and other programs, and also reads and writes various types of information. Specifically, for example, the central processing unit 201 functions as a knowledge information acquisition unit 21, a construction information extraction unit 22, and a model construction unit 23. The device 20 may also include other computing devices such as a CPU, GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), or a combination thereof as computing devices.

[0065] Bus 203 can also be connected to external devices, for example. Examples of such external devices include external storage devices (external databases, etc.), electrocardiographs, printers, external input devices, external display devices, audio output devices such as speakers, external imaging devices such as cameras, and various sensors such as acceleration sensors, geomagnetic sensors, and direction sensors. The device 20 can be connected to an external network (the aforementioned communication network) by a communication device 207 connected to bus 203, for example, and can also be connected to other devices via the external network.

[0066] Memory 202 may be, for example, main memory. When the central processing unit 201 performs processing, memory 202 reads various operational programs, such as the program of this disclosure, stored in the storage device 204 (described later), and the central processing unit 201 receives data from memory 202 and executes the program. The main memory may be, for example, RAM (random access memory). Alternatively, memory 202 may be, for example, ROM (read-only memory).

[0067] The storage device 204 is also called an auxiliary storage device, for example, in relation to the main memory (primary memory). As described above, the storage device 204 stores an operating program including the program of this disclosure. The storage device 204 may be, for example, a combination of a recording medium and a drive for reading and writing to the recording medium. The recording medium is not particularly limited and may be internal or external, for example, an HD (hard disk), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, memory card, etc. The storage device 104 may be, for example, a hard disk drive (HDD) in which the recording medium and the drive are integrated, or a solid state drive (SSD). If the device 20 includes, for example, the storage device 204 functions as the storage unit.

[0068] In this device 20, the memory 202 and storage device 204 can also store various types of information, such as log information, information obtained from an external database (not shown) or external devices, information generated by this device 20, and information used by this device 20 when executing processing. At least some of the information may be stored, for example, on an external server other than the memory 202 and storage device 204, or distributed and stored across multiple terminals using blockchain technology or the like.

[0069] The device 20 further includes, for example, an input device 205 and an output device 206. The input device 205 may include, for example, a pointing device such as a touch panel, trackpad, or mouse; a keyboard; imaging means such as a camera or scanner; a card reader such as an IC card reader or magnetic card reader; an audio input means such as a microphone; and so on. The output device 206 may include, for example, a display device such as an LED display or liquid crystal display; an audio output device such as a speaker; a printer; and so on. In this embodiment 3, the input device 205 and the output device 206 are configured separately, but the input device 205 and the output device 206 may be configured as an integrated unit, such as a touch panel display.

[0070] Next, an example of the method for manufacturing a surrogate existence model according to this embodiment will be described based on the flowchart in Figure 8. The method for manufacturing a surrogate existence model according to this embodiment can be carried out as follows, for example, using the surrogate existence model manufacturing apparatus 20 shown in Figures 6 and 7. Note that the method for manufacturing a surrogate existence model according to this embodiment is not limited to the use of the surrogate existence model manufacturing apparatus 20 shown in Figures 6 and 7.

[0071] First, the knowledge information acquisition unit 21 acquires the subject's knowledge information (S21, knowledge information acquisition step). The format of the knowledge information is not particularly limited; for example, it may be text information, image information, audio information, or a combination thereof. The knowledge information is, for example, information linked to predetermined information and subject identification information that identifies the creator of the information (the subject). The predetermined information is, for example, information that includes the subject's personality information and explicit knowledge information. The personality information is, for example, information that represents the subject's thoughts from the knowledge information. The personality information is also called, for example, a partial stance. The explicit knowledge information is, for example, the part of the knowledge information excluding the personality information, and includes objective knowledge. If the knowledge information is a book, paper, etc., the explicit knowledge information may, for example, be information such as technical terms and experimental results, but is not limited to these. The subject identification information may, for example, be a name, address, telephone number, email address, identification number (for example, My Number (individual number), etc.). Specific examples of the aforementioned knowledge information include, but are not limited to, books and papers written by the subject, video data of lectures given by the subject, audio data of lectures given by the subject, and image data created by the subject. The knowledge information acquisition unit 21 may, for example, acquire knowledge information recorded in the storage unit of the device 20, or it may acquire the aforementioned knowledge information from outside the device 20 via the input device 205. The knowledge information acquisition unit 21 may, for example, acquire one type of knowledge information of the subject, or it may acquire two or more types. The knowledge information acquisition unit 21 may, for example, record the acquired knowledge information in the storage unit of the device 20.

[0072] The information extraction unit 22 for construction extracts personality information of the subject from the knowledge information (S22, information extraction step for construction). The information extraction unit 22 for construction may also, for example, further extract explicit knowledge information from the knowledge information of the subject. The information extraction unit 22 for construction may, for example, use known natural language processing techniques to extract at least one of the personality information and explicit knowledge information of the subject, or use a large-scale language model to extract at least one of the personality information and explicit knowledge information of the subject. If the knowledge information is textual information such as a book or a paper, the information extraction unit 22 for construction can extract the personality information or explicit knowledge information based, for example, on the end of sentences in a document or the chapter structure of a book. The information extraction unit 22 for construction may, for example, record at least one of the extracted personality information and explicit knowledge information in the storage unit of the device 20. In this case, the information extraction unit 22 for construction can, for example, record the personality information and explicit knowledge information linked to the knowledge information from which they were extracted and the creator identification information of the knowledge information.

[0073] The information extraction unit 22 for construction can, for example, analyze the knowledge information and extract sentences whose sentence ends with a word that expresses the author's thoughts as the personality information. Examples of words that express the author's thoughts include, but are not limited to, words such as "I want to," "I think," "I believe," and "I want." The information extraction unit 22 for construction can also analyze the knowledge information and extract sentences whose sentence ends with a word that indicates explicit knowledge as the explicit knowledge information. Examples of words that indicate explicit knowledge include, but are not limited to, words such as "It is," "It was," and "As a result."

[0074] The information extraction unit 22 for construction may, for example, analyze the knowledge information and extract sentences contained in chapters that describe the author's ideas as personality information. Examples of chapters that describe the author's ideas include the "preface," "introduction," and "afterword."

[0075] The information acquisition unit 22 for construction may, for example, use AI to extract personality information and explicit knowledge information from knowledge information. In this disclosure, AI may refer to, for example, a large-scale learning model called a "foundation model." The foundation model is a machine learning model pre-trained on predetermined big data and is not limited to a large-scale language model (LLM) that has learned natural language, but may also include a large-scale model for speech, a large-scale model for images, and a multimodal model (such as a visual language model) that handles language, images, speech, and video across the board. Furthermore, a configuration may be adopted in which a small-scale language model (SLM) is placed on the terminal side and cooperates with the large-scale model on the cloud side, a configuration that includes search extension generation (RAG) using an external knowledge source, tool execution / function call, agent-oriented control logic, etc. The providers of large-scale language models are not particularly limited. Examples include, but are not limited to, various LLM / multimodal models provided by companies such as OpenAI, Anthropique, Alphabet (Google), META, Microsoft, Cohere, Mistral, xAI, NEC Corporation, and NTT.

[0076] When the information acquisition unit 22 for construction uses AI to extract personality information and explicit knowledge information from knowledge information, the information acquisition unit 22 can cause the AI ​​to extract the user's personality information and explicit knowledge information by inputting extraction instruction information, which instructs the AI ​​to extract personality information and explicit knowledge information, along with the knowledge information. The extraction instruction information may be recorded in the storage unit of the device 10, stored externally, or input by the user each time. Specific examples of the extraction instruction information include, for example, "classifying information into the following two categories: sentences and paragraphs that show the author's thoughts and opinions, and sentences and paragraphs that show knowledge, such as scientific verification results and historical facts." This disclosure is not limited to the above examples.

[0077] Furthermore, the AI ​​may be, for example, a model finely tuned to extract personality information and explicit knowledge information from knowledge information. In this case, for example, fine tuning for extracting personality information and explicit knowledge information from knowledge information can be performed by training the AI ​​with knowledge information and a set of personality information and explicit knowledge information previously extracted from the knowledge information. In this case, the AI ​​may be, for example, a multilayer neural network model having a large number of parameters (e.g., 100,000 or more, and in some cases, several million or more). Such fine tuning processing requires the rapid and repeated execution of a huge amount of numerical calculations, making it practically impossible for a human to perform it in their head or with paper and pencil, and thus relies on automated processing by electronic computers such as processors and GPUs. During the fine tuning, in order to suppress overfitting of the training data, known regularization methods such as L2 regularization, dropout, and early stopping may be combined and applied. This makes it possible to obtain a model that can extract personality information and explicit knowledge information with high generalization performance even from unknown knowledge information, and improves the accuracy and robustness of information extraction compared to simple threshold judgment or rule-based processing.

[0078] Furthermore, the information extraction unit 22 for construction may extract the personality information of the subject by, for example, providing a large-scale language model with the subject's knowledge information and instruction information that instructs the model to extract the subject's personality information from the knowledge information based on the subject's knowledge information, thereby causing the model to extract the personality information from the knowledge information. The large-scale language model is not particularly limited and includes, but is not limited to, OpenAI's GPT-3, GPT-4, Alphabet (Google)'s BERT, LaMDA, PaLM2, META's LlaMA, NEC Corporation's LLM, NTT's LLM, etc. The instruction information that instructs the model to extract the subject's personality information from the knowledge information based on the subject's knowledge information is not particularly limited as long as it is a document that instructs the model to divide the knowledge information into parts that describe thoughts and parts that describe knowledge. Specific examples of instruction information that instructs the extraction of personality information of the subject from the knowledge information of the subject include, but are not limited to, documents such as, "Classify the information into the following two categories: - Sentences and paragraphs that show the author's thoughts and opinions, such as the author's ideas; - Sentences and paragraphs that show knowledge, such as scientific verification results and historical facts."

[0079] The information extraction unit 22 for construction may, for example, preprocess the knowledge information and extract personality information and explicit knowledge information. Specifically, for example, the information extraction unit 22 for construction may convert the knowledge information into a vectorized embedding vector sequence for each unit text, and then classify or group the knowledge information into personality information and explicit knowledge information based on the embedding vector sequence, thereby extracting personality information and explicit knowledge information based on the knowledge information. The embedding vectors may, for example, contain hundreds to thousands of real-valued elements for each unit text. For this reason, for example, a large number of embedding vectors are generated for the entire knowledge information, and the classification or grouping process consists of a large number of numerical operations, mainly matrix operations, which is a process that is practically impossible for a human to perform in their head or with paper and pencil.

[0080] The model building unit 23 constructs a surrogate existence model that mimics the target person based on the personality information (S23, model building step). The model building unit 23 can construct the surrogate existence model by, for example, providing a large-scale learning model with the personality information and instruction information that instructs the large-scale learning model to construct a surrogate existence model that mimics the target person based on the personality information. The large-scale learning model is, for example, a machine learning model that has been trained using predetermined big data. The large-scale learning model may be, for example, a model that has been trained on big data of natural language (large-scale language model), a model that has been trained on big data of speech (large-scale speech model), or a model that has been trained on big data of images (large-scale image model). The model building unit 23 can construct the surrogate existence model by, for example, providing a large-scale language model, as the large-scale learning model, with the personality information and instruction information that instructs the large-scale learning model to construct a surrogate existence model that mimics the target person based on the personality information. The instruction information is, for example, instruction information (prompt) for generating the behavior of a person handling knowledge. The aforementioned instruction information could include, but is not limited to, documents such as, "When generating text, follow the rules below: - Describe examples that reflect personal information. - Do not reflect personal information in terms of knowledge. - For example, structure the text by stating a fact that is correct as knowledge, and then expressing an opinion on that fact as personal information."

[0081] Furthermore, the model building unit 23 may generate multiple sets of training data showing the correspondence between the subject's input information and responses based on the extracted personality information. The model building unit 23 may input the training data in mini-batch units into a large-scale learning model, calculate a loss function based on the error between the model's output for the training data and the response, and train a personality model capable of outputting behavior that mimics the subject by iteratively updating a large number of model parameters using gradient descent or a modified algorithm thereof.

[0082] Furthermore, the model building unit 23 may, for example, construct an explicit knowledge model by further training a large-scale language model with the explicit knowledge information. The model building unit 23 can construct the explicit knowledge model by, for example, providing the explicit knowledge information to the large-scale language model and fine-tuning it. The explicit knowledge model is also called, for example, a large-scale language model with domain knowledge.

[0083] The aforementioned surrogate existence model is, for example, a model that has learned the personality information of the subject from the information that constitutes the knowledge information. Therefore, the surrogate existence model has learned, for example, the subject's way of thinking and understanding when handling knowledge, values ​​such as thoughts and beliefs, and how they interact with others (personality). For this reason, the surrogate existence model manufacturing method of this disclosure makes it possible to easily manufacture a model that reflects the thoughts of the information creator. Furthermore, the surrogate existence model manufactured by the surrogate existence model manufacturing method of this disclosure is capable of outputting products that are characteristic of the subject. In other words, according to this disclosure, for example, it becomes possible to construct a model that suppresses personal hallucination. Personal hallucination refers to, for example, a hallucination (illusion) of how to handle knowledge and output policies that the creator of the knowledge information would not say. Therefore, according to the surrogate existence model of this disclosure, for example, it becomes possible to more accurately extract and output the knowledge that knowledge information (e.g., a book) explicitly or implicitly contains. The output that is characteristic of the target person is not particularly limited and may be, for example, text output, voice output, or instructions to a designated machine.

[0084] The model building unit 23 may, for example, use extracted personality information to train a personality model capable of outputting behavior that mimics the subject by iteratively updating the parameters of a large-scale learning model having many parameters based on gradient descent to reproduce the correspondence between the subject's past input information and responses. Similarly, the model building unit 23 may, for example, use extracted explicit knowledge information to train an explicit knowledge model that outputs explicit knowledge information contained in the knowledge information. In this case, the model building unit 23 can, for example, generate proxy behavior information consistent with the subject's personality and output the proxy behavior information by coordinating the trained personality model and the explicit knowledge model. By configuring the personality model and the explicit knowledge model separately in this way, and coordinating the output of explicit knowledge based on the knowledge information with the output based on the personality information, it is possible to reduce the inclusion of personal expressions not included in the knowledge information and improve the consistency and reliability of the generated responses, compared to, for example, simply providing prompts to a general-purpose large-scale language model.

[0085] According to this disclosure, it is possible to create models that reflect the thoughts of the information creators. Therefore, according to this disclosure, for example, it is possible to construct surrogate models for individuals with limited human resources (e.g., busy researchers, supervisors, managers, teachers, etc.), thereby reducing the burden on those individuals.

[0086] While the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure are possible, as can be understood by those skilled in the art within the scope of the present disclosure.

[0087] This application claims priority based on Japanese Patent Application No. 2025-019427, filed on 7 February 2025, and incorporates all of its disclosures herein.

[0088] <Note> Some or all of the above embodiments may be described as follows, but are not limited to the following. (Note 1) A psychological characteristics analysis device comprising a dialogue log acquisition unit, a state analysis information acquisition unit, a psychological state analysis unit, and an output unit, wherein the dialogue log acquisition unit acquires a user's dialogue log, the dialogue log is information that records the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition unit acquires state analysis information during the user's dialogue, the state analysis information includes information that links time information with at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis unit analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output unit outputs the user's psychological characteristics information that links the dialogue log and the progression of the psychological state. (Note 2) The psychological characteristics analysis device according to Note 1, wherein the psychological state analysis unit estimates the level of concentration during the conversation as the psychological state, analyzes the progression of the level of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output unit outputs psychological characteristics information linked to the conversation log and the progression of the level of concentration. (Note 3) The psychological characteristics analysis device according to Note 1 or 2, wherein the psychological state analysis unit estimates the emotion during the conversation as the psychological state, analyzes the progression of the emotion during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output unit outputs the progression of the emotion linked to the conversation log. (Note 4) The psychological characteristics analysis device according to any one of Notes 1 to 3, wherein the psychological state analysis unit estimates the stress during the conversation as the psychological state, analyzes the progression of the stress during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output unit outputs the progression of the stress linked to the conversation log. (Note 5) A psychological characteristics analysis device according to any one of Notes 1 to 4, which includes a dialogue response generation unit, wherein the dialogue response generation unit inputs the psychological characteristics information into a dialogue model and causes the dialogue model to generate a dialogue response with the user.(Note 6) The psychological characteristic analysis apparatus according to Note 5, wherein the dialogue response generation unit selects a dialogue model from a model pool in which dialogue models suitable for each psychological characteristic are recorded, based on the psychological characteristics, and causes the selected dialogue model to generate a dialogue response. (Note 7) The psychological characteristic analysis apparatus according to Note 5 or 6, wherein the dialogue model includes a surrogate existence model that mimics a surrogate target. (Note 8) A psychological characteristics analysis method comprising a dialogue log acquisition step, a state analysis information acquisition step, a psychological state analysis step, and an output step, wherein the dialogue log acquisition step acquires a user's dialogue log, the dialogue log is information recording the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition step acquires state analysis information during the user's dialogue, the state analysis information includes information linked to time information and at least one selected from the group consisting of image information of the user during the dialogue, voice information of the user during the dialogue, vital information of the user during the dialogue, and movement information of the user during the dialogue, the psychological state analysis step analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output step outputs psychological characteristics information of the user linked to the dialogue log and the progression of the psychological state, each step being performed by a computer. (Note 9) The psychological characteristics analysis method according to Note 8, wherein the psychological state analysis step estimates the degree of concentration during the conversation as the psychological state, analyzes the progression of the degree of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output step outputs psychological characteristics information linked to the conversation log and the progression of the degree of concentration. (Note 10) The psychological characteristics analysis method according to Note 8 or 9, wherein the psychological characteristics analysis step estimates the emotions during the conversation as the psychological state, analyzes the progression of emotions during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output step outputs the progression of emotions linked to the conversation log.(Note 11) The psychological characteristics analysis method according to any one of Notes 8 to 10, wherein the psychological characteristics analysis step estimates stress during the conversation as the psychological state, analyzes the progression of stress during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output step outputs the progression of stress linked to the conversation log. (Note 12) The psychological characteristics analysis method according to any one of Notes 8 to 11, further comprising a dialogue response generation step, wherein the dialogue response generation step inputs the psychological characteristics information into a dialogue model and causes the dialogue model to generate a dialogue response with the user. (Note 13) The psychological characteristics analysis method according to Note 12, wherein the dialogue response generation step selects a dialogue model from a model pool in which dialogue models suitable for each psychological characteristic are recorded, based on the psychological characteristics, and causes the selected dialogue model to generate a dialogue response. (Note 14) The psychological characteristics analysis method according to Note 12 or 13, wherein the dialogue model includes a surrogate existence model that mimics a surrogate target. (Note 15) A psychological characteristics analysis program for causing a computer to execute each of the following procedures: a dialogue log acquisition procedure, a state analysis information acquisition procedure, a psychological state analysis procedure, and an output procedure; the dialogue log acquisition procedure acquires a user's dialogue log; the dialogue log is information recording the dialogue between the user and the user's dialogue partner in chronological order; the state analysis information acquisition procedure acquires state analysis information during the user's dialogue; the state analysis information includes information linked to time information and at least one selected from a group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue; the psychological state analysis procedure analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information; and the output procedure outputs the user's psychological characteristics information linked to the dialogue log and the progression of the psychological state. (Note 16) The psychological state analysis procedure estimates the degree of concentration during the conversation as the psychological state, analyzes the change in the degree of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output procedure outputs psychological characteristic information linking the conversation log and the change in the degree of concentration, as described in Note 15.(Note 17) The psychological characteristics analysis program according to Note 15 or 16, wherein the psychological characteristics analysis procedure estimates the emotions during the conversation as the psychological state, analyzes the progression of emotions during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output procedure outputs the progression of emotions linked to the conversation log. (Note 18) The psychological characteristics analysis program according to any one of Notes 15 to 17, wherein the psychological characteristics analysis procedure estimates the stress during the conversation as the psychological state, analyzes the progression of stress during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output procedure outputs the progression of stress linked to the conversation log. (Note 19) The psychological characteristics analysis program according to any one of Notes 15 to 18, further comprising a dialogue response generation procedure, wherein the dialogue response generation procedure inputs the psychological characteristics information into a dialogue model and causes the dialogue model to generate a dialogue response with the user. (Note 20) The dialogue response generation procedure is a psychological characteristic analysis program according to Note 19, which, based on the psychological characteristics, selects a dialogue model from a model pool in which dialogue models suitable for each psychological characteristic are recorded, and causes the selected dialogue model to generate a dialogue response. (Note 21) The psychological characteristic analysis program according to Note 19 or 20, wherein the dialogue model includes a surrogate existence model that mimics a surrogate target.(Note 22) A computer-readable recording medium that records a psychological characteristics analysis program for causing a computer to execute each of the following procedures: a dialogue log acquisition procedure, a state analysis information acquisition procedure, a psychological state analysis procedure, and an output procedure, wherein the dialogue log acquisition procedure acquires a user's dialogue log, the dialogue log is information recording the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition procedure acquires state analysis information during the user's dialogue, the state analysis information includes information linked to time information and at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis procedure analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output procedure outputs the user's psychological characteristics information linked to the dialogue log and the progression of the psychological state. (Note 23) The recording medium according to Note 22, wherein the psychological state analysis procedure estimates the degree of concentration during the conversation as the psychological state, analyzes the progression of the degree of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output procedure outputs psychological characteristic information linked to the conversation log and the progression of the degree of concentration. (Note 24) The recording medium according to Note 22 or 23, wherein the psychological characteristic analysis procedure estimates the emotion during the conversation as the psychological state, analyzes the progression of the emotion during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output procedure outputs the progression of the emotion linked to the conversation log. (Note 25) The recording medium according to any one of Notes 22 to 24, wherein the psychological characteristic analysis procedure estimates the stress during the conversation as the psychological state, analyzes the progression of the stress during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output procedure outputs the progression of the stress linked to the conversation log. (Note 26) A recording medium according to any one of Notes 22 to 25, which includes a dialogue response generation procedure, wherein the dialogue response generation procedure inputs the psychological characteristics information into a dialogue model and causes the dialogue model to generate a dialogue response with the user.(Note 27) The dialogue response generation procedure is the psychological characteristic analysis method described in Note 26, which involves selecting a dialogue model from a model pool in which dialogue models suitable for each psychological characteristic are recorded, based on the psychological characteristics, and causing the selected dialogue model to generate a dialogue response. (Note 28) The recording medium described in Note 26 or 27, wherein the dialogue model includes a surrogate existence model that mimics a surrogate target.

[0089] According to this disclosure, for example, specific psychological characteristics related to user interactions can be analyzed and visualized. Therefore, this disclosure is useful in various industries, such as the education sector.

[0090] 10, 10A Psychological Characteristics Analysis Device 11 Dialogue Log Acquisition Unit 12 State Analysis Information Acquisition Unit 13 Psychological State Analysis Unit 14 Output Unit 15 Dialogue Response Generation Unit 101 Central Processing Unit 102 Memory 103 Bus 104 Storage Device 105 Input Device 106 Output Device 107 Communication Device 20 Proxy Existence Model Construction Device 21 Knowledge Information Acquisition Unit 22 Construction Information Extraction Unit 23 Model Construction Unit 201 Central Processing Unit 202 Memory 203 Bus 204 Storage Device 205 Input Device 206 Output Device 207 Communication Device

Claims

1. A psychological characteristics analysis device comprising: a dialogue log acquisition unit, a state analysis information acquisition unit, a psychological state analysis unit, and an output unit, wherein the dialogue log acquisition unit acquires a user's dialogue log, the dialogue log is information recording the dialogue between the user and the user's conversation partner in chronological order, the state analysis information acquisition unit acquires state analysis information during the user's dialogue, the state analysis information includes information linked to time information and at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis unit analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output unit outputs the user's psychological characteristics information linked to the dialogue log and the progression of the psychological state.

2. The psychological characteristics analysis device according to claim 1, wherein the psychological state analysis unit estimates the degree of concentration during the conversation as the psychological state, analyzes the change in the degree of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output unit outputs psychological characteristics information linking the conversation log and the change in the degree of concentration.

3. The psychological state analysis unit estimates the emotions during the conversation as the psychological state, analyzes the progression of emotions during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output unit outputs the progression of emotions linked to the conversation log, according to claim 1 or 2.

4. The psychological state analysis unit estimates stress during the conversation as the psychological state, analyzes the progression of stress during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output unit outputs the progression of stress linked to the conversation log, according to any one of claims 1 to 3.

5. A psychological characteristics analysis device according to any one of claims 1 to 4, comprising a dialogue response generation unit, wherein the dialogue response generation unit inputs the psychological characteristics information into a dialogue model and causes the dialogue model to generate a dialogue response with the user.

6. The psychological characteristic analysis device according to claim 5, wherein the dialogue response generation unit selects a dialogue model from a model pool in which dialogue models suitable for each psychological characteristic are recorded, based on the psychological characteristics, and causes the selected dialogue model to generate a dialogue response.

7. The psychological characteristics analysis device according to claim 5 or 6, wherein the dialogue model includes a surrogate entity model that mimics a surrogate target.

8. A psychological characteristics analysis method comprising a dialogue log acquisition step, a state analysis information acquisition step, a psychological state analysis step, and an output step, wherein each step is performed by a computer, the dialogue log acquisition step involves acquiring a user's dialogue log, the dialogue log is information recording the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition step involves acquiring state analysis information during the user's dialogue, the state analysis information includes information linked to time information and at least one selected from a group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis step involves analyzing the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output step outputs the user's psychological characteristics information linked to the dialogue log and the progression of the psychological state.

9. The psychological characteristics analysis method according to claim 8, wherein the psychological state analysis step estimates the degree of concentration during the conversation as the psychological state, analyzes the change in the degree of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output step outputs psychological characteristics information linking the conversation log and the change in the degree of concentration.

10. The psychological characteristics analysis method according to claim 8 or 9, wherein the psychological characteristics analysis step estimates the emotions during the conversation as the psychological state, analyzes the progression of emotions during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output step outputs the progression of emotions linked to the conversation log.

11. The psychological characteristics analysis method according to any one of claims 8 to 10, wherein the psychological characteristics analysis step estimates stress during the conversation as the psychological state, analyzes the progression of stress during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output step outputs the progression of stress linked to the conversation log.

12. A method for analyzing psychological characteristics according to any one of claims 8 to 11, comprising a dialogue response generation step, wherein the dialogue response generation step involves inputting the psychological characteristics information into a dialogue model and causing the dialogue model to generate a dialogue response with the user.

13. The method for analyzing psychological characteristics according to claim 12, wherein the dialogue response generation step involves selecting a dialogue model from a model pool in which dialogue models suitable for each psychological characteristic are recorded, based on the psychological characteristics, and causing the selected dialogue model to generate a dialogue response.

14. The method for analyzing psychological characteristics according to claim 12 or 13, wherein the dialogue model includes a surrogate entity model that mimics a surrogate target.

15. A psychological characteristics analysis program for causing a computer to execute each of the following procedures: a procedure for acquiring a dialogue log, a procedure for acquiring information for state analysis, a procedure for analyzing a psychological state, and an output procedure, wherein the procedure for acquiring a dialogue log acquires a user's dialogue log, the dialogue log is information that records the dialogue between the user and the person the user is talking to in chronological order, the procedure for acquiring information for state analysis acquires information for state analysis during the user's dialogue, the state analysis information includes information that links time information with at least one selected from the group consisting of image information of the user during the dialogue, voice information of the user during the dialogue, vital information of the user during the dialogue, and movement information of the user during the dialogue, the psychological state analysis procedure analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output procedure outputs psychological characteristics information of the user that links the dialogue log and the progression of the psychological state.

16. The psychological characteristics analysis program according to claim 15, wherein the psychological state analysis procedure estimates the degree of concentration during the conversation as the psychological state, analyzes the change in the degree of concentration during the conversation based on the time series of the conversation log and the time information contained in the state analysis information, and the output procedure outputs psychological characteristics information linking the conversation log and the change in the degree of concentration.

17. The psychological characteristics analysis program according to claim 15 or 16, wherein the psychological characteristics analysis procedure estimates the emotions during the conversation as the psychological state, analyzes the progression of emotions during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output procedure outputs the progression of emotions linked to the conversation log.

18. The psychological characteristics analysis program according to any one of claims 15 to 17, wherein the psychological characteristics analysis procedure estimates stress during the conversation as the psychological state, analyzes the progression of stress during the conversation based on the time series of the conversation log and the time information included in the state analysis information, and the output procedure outputs the progression of stress linked to the conversation log.

19. A psychological characteristics analysis program according to any one of claims 15 to 18, comprising a dialogue response generation procedure, wherein the dialogue response generation procedure inputs the psychological characteristics information into a dialogue model and causes the dialogue model to generate a dialogue response with the user.

20. A computer-readable recording medium that records a psychological characteristics analysis program for causing a computer to execute each of the following procedures: a dialogue log acquisition procedure, a state analysis information acquisition procedure, a psychological state analysis procedure, and an output procedure, wherein the dialogue log acquisition procedure acquires a user's dialogue log, the dialogue log is information recording the dialogue between the user and the user's dialogue partner in chronological order, the state analysis information acquisition procedure acquires state analysis information during the user's dialogue, the state analysis information includes information linked to time information and at least one selected from the group consisting of the user's image information during the dialogue, the user's voice information during the dialogue, the user's vital information during the dialogue, and the user's movement information during the dialogue, the psychological state analysis procedure analyzes the progression of the user's psychological state during the dialogue based on at least one of the dialogue log and the state analysis information, and the output procedure outputs the user's psychological characteristics information linked to the dialogue log and the progression of the psychological state.

Images