Information processing device, information processing method, and program

Abstract

An information processing device (100) comprises: a storage processing unit (102) that stores an emotion analysis result of a subject in association with a category of work in which the subject is engaged when the emotion analysis result is under analysis; and a display processing unit (104) that displays each schedule by category of work that has been carried out or is scheduled to be carried out in association with the emotion analysis result corresponding to the category of work in a plan displaying the subject's schedule.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a recording medium.

Background Art

[0002] Patent Document 1 describes a system for predicting the level of a specific emotion of a target person in the future. The system described in this Patent Document 1 is based on the level of a specific emotion generated based on the behavior history of a user obtained from the user's posts on an SNS (Social Networking Service), etc., the date and time and location when the specific emotion occurred, and a keyword related to the occurrence of the specific emotion. It includes an accumulation unit that accumulates an emotion history in which they are associated, a future schedule of the user, and a control unit that predicts the level of a specific emotion of the user in the future based on the future schedule of the user, the level of the specific emotion stored as the emotion history, and the keyword related to the occurrence of the level of the specific emotion.

[0003] In addition, Patent Document 2 describes a technique for appropriately investigating the magnitude of stress received by a target person. The information processing apparatus described in this Patent Document 2 generates a mathematical formula that can convert a heartbeat stress value into a subjective stress value for a business, stores a stress weight and an offset value that form the mathematical formula, generates and stores a threshold value for the subjective stress value for the business, and for each week, for each business, uses the generated mathematical formula to estimate the subjective stress value based on the heartbeat stress value, determines whether the estimated subjective stress value is greater than or equal to the threshold value for each business, and if there is a business for which the estimated subjective stress value is greater than or equal to the threshold value, outputs an alert for the business for which the estimated subjective stress value is greater than or equal to the threshold value.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

[0005] The technology described in Patent Document 1 above envisions that on the morning of a business trip, the user checks the happiness forecast for the entire country, understands the level of happiness at a predetermined time at their destination or transit point, and prepares in advance. Furthermore, the technology described in Patent Document 2 above quantifies the stress experienced by the subject in the course of work, issues an alert when the stress exceeds a threshold, and enables the company to provide appropriate support to the employee. In other words, neither of the technologies described in the above-mentioned patent documents envisions estimating the subject's emotions for each task, nor does it envision visualizing the subject's emotions for each task on the schedule, either in the subject's past or future work schedules.

[0006] Furthermore, none of the technologies described in the aforementioned patent documents envision as relating keywords extracted by processing the conversational audio of a subject during work with the subject's emotions during that work. In contrast, the present inventors have considered estimating the subject's emotions for each task and visualizing the subject's emotions for each task in at least one of the subject's past and future work schedules, or visualizing as relating keywords extracted by processing the conversational audio of a subject during work with the subject's emotions during that work.

[0007] This invention has been made in view of the above circumstances, and its purpose is to provide an information processing device, an information processing method, and a recording medium that make it easier to understand the impact of work on a subject's mental state by visualizing the subject's emotions during work. [Means for solving the problem]

[0008] According to one aspect of the present invention, A memory processing means that stores the results of an emotional analysis of a subject in a memory means, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. An information processing device is provided, which includes a display processing means that displays each scheduled or previously performed task by category in relation to the sentiment analysis results corresponding to the task category on a screen displaying the schedule of the subject.

[0009] According to one aspect of the present invention, A memory processing means that stores the results of an emotional analysis of a subject in a memory means, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. Extraction means for processing the conversational audio of the subject engaged in the aforementioned work, which was the subject of the emotion analysis results, and extracting keywords that include at least one of the words and phrases contained in the conversational audio, An information processing device is provided, which includes a display processing means for displaying a screen containing information that associates the classification of the subject's feelings toward the said work with the keywords contained in the said conversation audio while the subject is engaged in the said work.

[0010] According to one aspect of the present invention, One or more computers, The results of the emotional analysis of the subject are stored in a memory device, associated with the category of work the subject was engaged in at the time the emotional analysis was performed. An information processing method is provided for displaying the schedule of the subject on a screen that shows the schedule of the subject, in association with the sentiment analysis results corresponding to the category of work, for each category of work that is scheduled or has been performed in the past.

[0011] According to one aspect of the present invention, One or more computers, The results of the emotional analysis of the subject are stored in a memory device, associated with the category of work the subject was engaged in at the time the emotional analysis was performed. The conversational audio of the subject engaged in the aforementioned work, which was the subject of the sentiment analysis results, is processed to extract keywords that include at least one of the words and phrases contained in the conversational audio. An information processing method is provided that displays a screen containing information relating the classification of the subject's feelings towards the work and the keywords contained in the conversation audio while the subject is engaged in the work.

[0012] According to one aspect of the present invention, On the computer, A procedure for storing the results of an emotional analysis of a subject in a memory device, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. A computer-readable recording medium is provided, which stores a program for executing a procedure to display, on a screen that displays the schedule of the subject, each scheduled or previously performed task by category in association with the sentiment analysis results corresponding to the task category.

[0013] According to one aspect of the present invention, On the computer, A procedure for storing the results of an emotional analysis of a subject in a memory device, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. A procedure for processing the conversational audio of the subject engaged in the aforementioned work, which was the subject of the sentiment analysis results, and extracting keywords that include at least one of the words and phrases contained in the conversational audio. A computer-readable recording medium is provided, which stores a program for causing a procedure to display a screen containing information relating the classification of the subject's feelings towards the work and the keywords contained in the conversation audio while the subject is engaged in the work. [Effects of the Invention]

[0014] According to one aspect of the present invention, an information processing apparatus, an information processing method, and a recording medium can be obtained that make it easy to grasp the influence of a target person's work on their mental state by visualizing the emotions of the target person during work.

Brief Description of the Drawings

[0015] [Figure 1] It is a diagram showing an overview of the information processing apparatus according to the embodiment. [Figure 2] It is a flowchart showing an operation example of the information processing apparatus of the embodiment. [Figure 3] It is a diagram showing an overview of the information processing apparatus according to the embodiment. [Figure 4] It is a flowchart showing an operation example of the information processing apparatus of the embodiment. [Figure 5] It is a diagram conceptually showing an example of the system configuration of the information processing system according to the embodiment. [Figure 6] It is a diagram conceptually showing another example of the system configuration of the information processing system according to the embodiment. [Figure 7] [Figure 7] It is a block diagram exemplifying the hardware configuration of a computer that realizes the information processing apparatus. [Figure 8] It is a functional block diagram showing an example of the logical configuration of the information processing apparatus of the embodiment. [Figure 9] It is a diagram showing an example of the data structure of biometric information. [Figure 10] It is a diagram showing an example of the data structure of business information. [Figure 11] It is a diagram showing an example of the data structure of score history information. [Figure 12] It is a diagram for explaining an emotion estimation method using the arousal level and the harmony level. [Figure 13] It is a diagram showing an example of the data structure of emotion analysis result information. [Figure 14] It is a diagram showing an example of a schedule screen. [Figure 15] [Figure 16] It is a diagram showing an example of the data structure of emotion information by business type.​ [Figure 17] This figure shows an example of a business ranking screen. [Figure 18] This is a flowchart showing an example of the operation of the information processing device according to the embodiment. [Figure 19] This is a functional block diagram showing a logical configuration example of an information processing device according to the embodiment. [Figure 20] This is a functional block diagram showing an example of a modified configuration of the information processing device of the embodiment. [Figure 21] This figure shows an example of a keyword ranking screen. [Figure 22] This is a flowchart showing an example of the operation of the information processing device according to the embodiment. [Figure 23] This is a flowchart showing the process of counting keywords. [Figure 24] This diagram conceptually shows an example of the system configuration of an information processing system according to the embodiment. [Figure 25] This figure shows an example of a report screen. [Figure 26] This figure shows an example of a report screen. [Figure 27] This figure shows a graph illustrating the distribution of user emotions. [Modes for carrying out the invention]

[0016] Embodiments of the present invention will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted as appropriate. Furthermore, in the following drawings, components that are not essential to the present invention are omitted and are not shown.

[0017] In the embodiments, "acquisition" includes at least one of the following: the device retrieving data or information stored in another device or storage medium (active acquisition), and the device inputting data or information output from another device into its own device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading data from another device or storage medium. Examples of passive acquisition include receiving information that is delivered (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from among the received data or information, or selecting and receiving data or information that has been delivered.

[0018] <Minimum Configuration Example 1> Figure 1 is a diagram showing an overview of the information processing device 100 according to an embodiment. The information processing device 100 comprises a storage processing unit 102 and a display processing unit 104. The memory processing unit 102 stores the results of the subject's emotional analysis, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. The display processing unit 104 displays the schedule of the subject on the screen that displays the subject's schedule, associating each scheduled or previously performed task by category with the sentiment analysis results corresponding to the task category.

[0019] <Example of operation> Figure 2 is a flowchart showing an example of the operation of the information processing device 100 of the embodiment. The memory processing unit 102 stores the results of the subject's emotional analysis, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed (step S101). The display processing unit 104 displays the schedule of the subject on the screen that displays the subject's schedule, associating each scheduled or previously performed task by category with the sentiment analysis results corresponding to the task category (step S103).

[0020] Steps S101 and S103 of this process may be executed at different times. For example, step S101 may be executed when the subject's emotional analysis results and work category are obtained, and step S103 may be executed when the subject's schedule display screen is shown.

[0021] In this information processing device 100, the memory processing unit 102 stores the results of the emotion analysis of a subject, associating them with the category of work the subject was engaged in at the time the emotion analysis was performed. The display processing unit 104 displays the schedule of the subject on the screen that displays the subject's schedule, associating each scheduled or previously performed task by category with the sentiment analysis results corresponding to the task category. Thus, this information processing device 100 makes it easier to understand the impact of work on a person's mental state by visualizing their emotions during work.

[0022] <Minimum Configuration Example 2> Figure 3 shows an overview of the information processing device 100 according to the embodiment. The information processing device 100 comprises a storage processing unit 102, a display processing unit 104, and an extraction unit 106. The memory processing unit 102 stores the results of the subject's emotional analysis, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. The extraction unit 106 processes the conversational audio of the subject at the time the sentiment analysis result was analyzed, and extracts keywords that include at least one of the words and phrases contained in the conversational audio. The display processing unit 104 displays a screen containing information that associates the subject's emotional state towards their work with keywords contained in the conversation audio while they are engaged in that work, categorized by work category.

[0023] <Example of operation> Figure 4 is a flowchart showing an example of the operation of the information processing device 100 of the embodiment. The memory processing unit 102 stores the results of the subject's emotional analysis, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed (step S101). The extraction unit 106 processes the conversational audio of the subject at the time the sentiment analysis result was analyzed, and extracts keywords that include at least one of the words and phrases contained in the conversational audio (step S111). The display processing unit 104 displays a screen containing information that associates the subject's emotional state towards the work with keywords contained in the conversation audio while the subject is engaged in the work, categorized by work type (step S113).

[0024] Steps S101, S111, and S113 of this process may be executed at different times. For example, step S101 may be executed when the subject's emotional analysis results and work category are acquired, step S111 may be executed when the subject's conversation audio is acquired, and step S113 may be executed when the screen displaying the subject's emotional analysis results is displayed.

[0025] In this information processing device 100, the memory processing unit 102 stores the results of the subject's emotional analysis, associating them with the category of work the subject was engaged in at the time the emotional analysis results were analyzed. The extraction unit 106 processes the subject's conversational audio at the time the emotional analysis results were analyzed and extracts keywords that include at least one of the words and phrases contained in the conversational audio. The display processing unit 104 displays a screen containing information that associates the subject's emotional classification towards their work with the keywords contained in the conversational audio while they were engaged in that work, categorized by work category. Thus, this information processing device 100 makes it easier to understand the impact of work on a person's mental state by visualizing their emotions during work.

[0026] The following describes a detailed example of the information processing device 100.

[0027] (First Embodiment) <System Overview> Figure 5 is a conceptual diagram showing an example of the system configuration of the information processing system 1 according to the embodiment. The information processing system 1 comprises an information processing device 100 and a wearable terminal 50. In this example, the information processing device 100 is the user terminal 60 of user U, which is a computer such as a smartphone, tablet, or personal computer. The user terminal 60 and the wearable device 50 are paired in advance to enable communication using NFC (Near Field Communication).

[0028] In Information Processing System 1, User U is the subject of sentiment analysis, and User U is, for example, an employee of a company. In this example, the user terminal 60 is owned by User U, and the wearable device 50 is lent to User U by the company. However, the user terminal 60 does not necessarily have to be owned by User U; it may be lent to User U by the company. Similarly, the wearable device 50 does not necessarily have to be lent to User U by the company; it may be owned by User U.

[0029] In this scenario, the wearable device 50 is loaned to user U by the company. User U borrows the wearable device 50 upon arrival at work, pairs it with user terminal 60, wears the wearable device 50 while working, and unpairs it with user terminal 60 and returns the wearable device 50 upon leaving work. In other words, the wearable device 50 can be shared and used by employees. Therefore, if the work schedule is shift-based, only the number of wearable devices 50 needed per person present at the office each day is required, rather than the number of employees.

[0030] The wearable device 50 measures the user U's biometric information and transmits it to the user device 60. The biometric information acquired by the wearable device 50 includes, but is not limited to, pulse rate, heart rate, body temperature, blood pressure, activity level, posture, electrocardiogram, respiratory rate, blood oxygen level, and conversation voice.

[0031] The wearable device 50 is equipped with various sensors for measuring this biometric information. Furthermore, for conversational audio, the audio collected by the speaker of the wearable device 50 is recorded, and the audio data may be transmitted to the user terminal 60. Alternatively, the user terminal 60 may be kept constantly around the user U during work hours, and the audio data collected by the speaker of the user terminal 60 may be used.

[0032] User U's user terminal 60 has an application pre-installed for using the services of the information processing system 1. User U puts on the wearable device 50, launches the application on the user terminal 60, and pairs the wearable device 50 with the user terminal 60. As a result, the information processing device 100 can realize its functions on the user terminal 60.

[0033] Furthermore, the information processing system 1 of the embodiment may have the configuration shown in Figure 6. Figure 6 is a conceptual diagram showing another example of the system configuration of the information processing system 1 of the embodiment. In this example, the information processing system 1 further includes a server device 70 in addition to the information processing system 1 of the embodiment shown in Figure 5.

[0034] The server device 70 is a server computer implemented by the computer 1000 shown in Figure 7, which will be described later. The server device 70 may also include a web server. The user terminal 60 is connected to the server device 70 via a communication network 3. The communication network 3 may be a network such as the Internet, or it may be a corporate network. The server device 70 includes a storage device 120. The storage device 120 may be located inside the server device 70 or outside of it. In other words, the storage device 120 may be hardware integrated with the server device 70, or it may be hardware separate from the server device 70.

[0035] In the configuration example shown in Figure 5, the information processing device 100 is implemented solely by the user terminal 60. However, in the configuration example shown in Figure 6, the information processing device 100 is implemented by a combination of the server device 70 and the user terminal 60. Furthermore, user U can access the server device 70 using the user terminal 60 to use the services provided by the information processing system 1. Therefore, user U registers in advance and sets up account information. When using the services of the information processing system 1, user U logs in to the information processing system 1 using their account information.

[0036] User U can use the services provided by Information Processing System 1 by installing the above-mentioned application on User Terminal 60, launching the application, and logging in to Information Processing System 1. Alternatively, User U may use the services provided by Information Processing System 1 by launching a designated browser on User Terminal 60, accessing the website for using the services provided by Information Processing System 1, and logging in using account information.

[0037] <Example Hardware Configuration> Figure 7 is a block diagram illustrating the hardware configuration of the computer 1000 that implements the information processing device 100 (user terminal 60). The wearable terminal 50 and the server device 70 are also implemented by the computer 1000. Furthermore, the administrator terminal 80, which will be described in a later embodiment, is also implemented. Computer 1000 This is achieved by [method / method].

[0038] Computer 1000 has a bus 1010, a processor 1020, memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0039] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0040] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0041] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0042] The storage device 1040 is an auxiliary storage device implemented as an HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules that implement each function of the information processing device 100 (for example, the memory processing unit 102, the display processing unit 104, the extraction unit 106, and the sentiment analysis unit 110, counting unit 112, and acquisition unit 114, which will be described later). The processor 1020 reads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to that program module. The storage device 1040 may also store data from the storage device 120, which will be described later.

[0043] The program module may be recorded on a recording medium. The recording medium on which the program module is recorded includes a non-temporary, tangible medium usable by a computer 1000, and program code that can be read by the computer 1000 (processor 1020) may be embedded in that medium.

[0044] The input / output interface 1050 is an interface for connecting the computer 1000 with various input / output devices. The input / output interface 1050 also functions as a communication interface for short-range wireless communication such as Bluetooth® and NFC (Near Field Communication).

[0045] The network interface 1060 is an interface for connecting the computer 1000 to a communication network. This communication network may be, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1060 may connect to the communication network via a wireless connection or a wired connection.

[0046] The computer 1000 then connects to necessary devices (for example, the display, touch panel, indicator, operation buttons, camera, speaker, microphone of the user terminal 60, the display, touch panel, indicator, operation buttons, speaker, microphone of the wearable terminal 50, the keyboard, mouse, speaker, microphone, printer, etc. of the server device 70) via the input / output interface 1050 or the network interface 1060.

[0047] Each component of the information processing device 100 in the embodiments shown in Figures 1, 3, and later in Figures 8, 19, and 20 can be realized by any combination of the hardware and software of the computer 1000 shown in Figure 7. Those skilled in the art will understand that there are various modifications to the implementation method and apparatus. The functional block diagrams showing the information processing device 100 in each embodiment represent blocks of logical functional units, not hardware units.

[0048] <Example of Functional Configuration> Figure 8 is a functional block diagram showing an example of the logical configuration of the information processing device 100 in this embodiment. The information processing device 100 includes the same memory processing unit 102 and display processing unit 104 as the information processing device 100 in Figure 1, and further includes an emotion analysis unit 110 and an acquisition unit 114.

[0049] The acquisition unit 114 acquires data for emotional analysis of the subject, and at the same time acquires business information related to the work the subject was engaged in. The data for sentiment analysis includes biometric information of user U (the subject) obtained from a wearable device 50 worn by user U.

[0050] The timing for acquiring biometric information from the wearable terminal 50 is not particularly limited. Depending on the memory capacity on the wearable terminal 50, a certain amount of biometric information may be stored and transmitted all at once for a predetermined period. Depending on the communication status between the wearable terminal 50 and the user terminal 60, the information may be transmitted when the communication status is good, or it may be transmitted immediately each time biometric information is acquired, or each time after a certain period of time has elapsed.

[0051] Figure 9 shows an example of the data structure of biometric information 130. Biometric information 130 stores biometric information such as time information, pulse rate, activity level, and conversation volume in association with each other. Biometric information 130 is stored in the memory 1030, storage device 1040, or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70.

[0052] Figure 10 shows an example of the data structure of business information 150. Business information 150 contains schedule information for user U during working hours. In this example, the start and end times for each business category are stored in association. Breaks are also included in the business categories. Therefore, the start and end times of breaks are also stored in association with business information 150.

[0053] The business information 150 is stored in the memory 1030, storage device 1040, or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70.

[0054] The categories of work are not particularly limited and can be set as appropriate by user U or by administrator M who manages user U at the company where user U works. The categories of work may include not only classifications based on the content of the work (meetings, business trips to clients, new customer development, new employee training, document creation, reporting, etc.), but also classifications based on a predetermined project, the product targeted by the work, the field of that product, the client (company or person in charge) targeted by the work, the team members of the work, and the person responsible for the work. The information processing device 100 may display a UI (User Interface) that accepts the setting of work categories on a predetermined operation screen (not shown) and may have a reception unit (not shown) that accepts the setting.

[0055] Furthermore, the business information 150 may be a schedule showing future plans, a schedule showing actual work performed by user U, or a combination of both.

[0056] The emotion analysis unit 110 performs an emotion analysis of user U using the emotion analysis data of user U acquired by the acquisition unit 114 from user U's wearable terminal 50, and outputs the emotion analysis results. The emotion analysis results are output, for example, as a score value indicating user U's mental state.

[0057] A score indicating mental state includes multiple scores, each representing a different type of mental state. For example, the score may include scores for arousal level and harmony level. For instance, the arousal level is a value indicating the degree of arousal, with a higher score indicating a greater degree of arousal and a lower score indicating a greater degree of drowsiness. The harmony level is a value indicating feelings such as pleasure and displeasure, with a higher score indicating a more comfortable state and a lower score indicating an unpleasant state.

[0058] The score may be, but is not limited to, a value expressed within one of the following ranges: 0 to 100, -100 to +100, 0 to 1, and -1 to +1.

[0059] Figure 11 shows an example of the data structure of the score history information 140. The score history information 140 stores time information and the score output by the sentiment analysis unit 110 in association with each other. The score history information 140 is stored in the memory 1030, storage device 1040, or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70.

[0060] Figure 12 is a diagram illustrating a method for estimating emotions using arousal and harmony levels. In this example, two scores indicating mental state, the arousal level and the harmony level, are shown as coordinate axes, and the four regions divided by the coordinate axes correspond to user U's four emotional categories. The four emotional categories, in this example, "ANGRY," "HAPPY," "SAD," and "RELAXED," are each defined by four coordinate regions. When user U's score value falls within a range exceeding the threshold T, user U is presumed to be in the emotion of that region.

[0061] Specifically, a score indicating the arousal level and a score indicating the harmony level are plotted on the graph in Figure 12. A threshold T is predetermined for at least one of the arousal level and harmony level. This threshold T is represented by a value between 0 and 100 on the Y-coordinate of the arousal level in Figure 12. The area beyond a circle with a radius based on this threshold and up to 100 on the Y-axis represents the range of each emotion. In other words, if the plotted score falls within each region that exceeds the threshold T, it is estimated that user U (the subject) is experiencing the emotion in that region. For example, Figure 12 shows data d1 plotted with the arousal level score y1 and the harmony level score x1 at time t1. Since user U's (the subject's) data d1 falls within the region labeled "ANGRY" that exceeds the threshold T, it is estimated that user U is experiencing anger. However, the range of the threshold T may not only be circular but also elliptical. In other words, the thresholds on the X-axis and Y-axis may be different values.

[0062] Furthermore, in this embodiment, the emotion analysis unit 110 may estimate whether user U's emotions fall within the negative domain and identify cases in which it is estimated that user U is experiencing negative emotions. Whether or not user U is experiencing negative emotions may be determined by whether the data d, which is a plot of user U's arousal level and harmony level scores, falls within the "ANGRY" and "SAD" domains.

[0063] Furthermore, if the hatched area in Figure 12 contains data d plotting the arousal level and harmony level scores of user U, it may be determined that the user is experiencing particularly negative emotions. The hatched area is, for example, the area of ​​negative emotion "ANGRY" that extends in a sector shape from the negative side of the x-axis coordinate in the positive direction of the y-axis coordinate at a predetermined angle θ1. It is also, for example, the area of ​​negative emotion "SAD" that extends in a sector shape from the negative side of the x-axis coordinate in the negative direction of the y-axis coordinate at a predetermined angle θ2. θ1 is set in the range of 0 to +90°, and θ2 is set in the range of 0 to -90°.

[0064] In this example, the range of negative emotions is shown at one level, but in other examples, it may be shown at multiple levels. For example, a sector-shaped region extending at angle θ3 may be set so as to be even narrower than the region defined by angle θ1, which represents the "ANGRY" region of negative emotions. Based on whether or not data d is included in the sector-shaped region extending at angle θ3, the emotion analysis unit 110 may determine whether or not the person is experiencing stronger negative emotions. Furthermore, the memory processing unit 102 may store the level of negative emotion (indicating high or low) in the emotion analysis result information 160. Note that angle θ3 is a smaller angle than angle θ1.

[0065] Furthermore, similar to negative emotions, positive emotions may also be identified by the emotion analysis unit 110 and stored in the emotion analysis result information 160 by the memory processing unit 102. Positive emotions may be determined, for example, by whether the data d plotted from the user U's arousal level and harmony level scores falls within the "HAPPY" and "RELAXED" regions. Alternatively, similar to negative emotions, the level of positive emotion may be determined by whether it falls within a fan-shaped region indicated by a predetermined angle within the "HAPPY" and "RELAXED" regions.

[0066] The memory processing unit 102 stores the sentiment analysis results of user U output by the sentiment analysis unit 110 as sentiment analysis result information 160, associating them with the category of work that user U was engaged in at the time the sentiment analysis results were analyzed. The category of work that user U was engaged in at the time the sentiment analysis results were analyzed can be obtained from the work information acquired by the acquisition unit 114.

[0067] Figure 13 shows an example of the data structure of the sentiment analysis result information 160. The sentiment analysis result information 160 stores associated time information, the sentiment category of user U at that time estimated by the sentiment analysis unit 110, and a negative flag indicating whether or not the user is experiencing negative emotions (for example, if it is determined that the user is experiencing negative emotions, "1" is set). The sentiment analysis result information 160 may also store associated positive flags indicating whether or not the user is experiencing positive emotions (for example, if it is determined that the user is experiencing positive emotions, "1" is set). Furthermore, the sentiment analysis result information 160 stores associated the business category in which user U is engaged at that time. The sentiment analysis result information 160 is stored in the memory 1030 or storage device 1040 or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70.

[0068] The display processing unit 104 displays the user U's schedule on a screen that shows the user U's schedule, associating each scheduled or previously completed task by category with the sentiment analysis results corresponding to the task category.

[0069] Figures 14 and 15 show examples of schedule screens 300. Each schedule screen 300 may be displayed by selecting the display operation for the schedule screen 300 from the operation menu. Alternatively, the schedule screen 300 may be automatically displayed on the user terminal 60 at a time specified in advance by user U or administrator M (for example, 9:00 on a workday).

[0070] Figure 14 is an example of a screen that does not include sentiment analysis results. The schedule screen 300 displays user U's schedule by business category. However, the method of displaying the schedule is not limited to these, and various display formats are possible. The legend list 310 includes the legend display unit 312 and shows the legend for each business category. User U can input and view the schedule on the schedule screen 300. In the embodiment described later, administrator M may also be able to view the schedule screen 300. When the sentiment analysis result display button 320 is pressed, the display processing unit 104 transitions to the schedule screen 300 shown in Figure 15.

[0071] The schedule screen 300 in Figure 15 displays each scheduled or previously completed task by category, associated with the sentiment analysis results corresponding to the task category. In the schedule screen 300 in Figure 15, the sentiment analysis results display button 320 is displayed in an inverted state compared to the sentiment analysis results display button 320 in Figure 14, for example, to indicate that it is selected.

[0072] The display processing unit 104 displays the schedule by business category, distinguishable by color or display element 322 according to the emotion analysis results of the user U. In this example, four types of display elements 322 are used to indicate the emotion categories of user U. For example, the display element 322 is a face mark that shows emotion through facial expression. Display element 322a is a smiling face mark that indicates the emotion category is "HAPPY". Display element 322b is a neutral face mark that indicates the emotion category is "RELAXED". Display element 322c is an angry face mark that indicates the emotion category is "ANGRY". Display element 322d is a crying face mark that indicates the emotion category is "SAD".

[0073] Furthermore, the display processing unit 104 uses a display element 322, in this example, to display categories of work where user U's emotions were strongly negative, using a color-changing display 324. The color-changing display 324 may use four colors corresponding to the four types of emotions, or, as in this example, it may only display strong emotions. For example, the negative emotion category "ANGRY" may be colored red, "SAD" purple, the positive emotion category "RELAXED" green, and "HAPPY" yellow.

[0074] By displaying User U's emotions for each task distinctly on the schedule, it becomes possible to visualize how User U felt while performing tasks in the past. Therefore, if User U was mentally fatigued, it may be possible to identify the task that caused it. Furthermore, if User U's emotions towards each task are visualized in the future schedule, it becomes immediately clear if tasks that might cause User U negative emotions are concentrated in a particular period. This makes it easier to adjust the schedule, such as distributing tasks to others or moving tasks to different days.

[0075] Furthermore, by visualizing User U's emotions towards each task, it becomes possible to understand which tasks User U excels at. For example, tasks that User U is likely to evoke positive emotions in are likely to be tasks that User U excels at. Therefore, for example, Manager M can consider User U's strengths (tasks that are likely to evoke positive emotions) and weaknesses (tasks that are likely to evoke negative emotions) and appropriately assign tasks to them.

[0076] The method for displaying the sentiment analysis results by the display processing unit 104 may be changed by user U or administrator M via the settings screen.

[0077] The division of functions between the user terminal 60 and the server device 70, and the information processing device 100, is illustrated below. (a1) The system configuration shown in Figure 5 does not include a server device 70, and all functions are performed by the user terminal 60. The following is an example of functional division in the system configuration shown in Figure 6, which includes a user terminal 60 and a server device 70. (a2) The user terminal 60 implements the function of the acquisition unit 114 to acquire biometric information from the wearable terminal 50, and the server device 70 implements the function of the acquisition unit 114 to acquire biometric information from the user terminal 60, as well as the functions of the emotion analysis unit 110, the memory processing unit 102, and the display processing unit 104. In other words, the user terminal 60 acquires biometric information from the wearable terminal 50 and transmits it to the server device 70, the server device 70 performs emotion analysis processing using the biometric information received by the user terminal 60, stores and manages the information, displays the schedule screen 300 on the user terminal 60 for the user U to view. (a3) The user terminal 60 implements the functions of the acquisition unit 114, the sentiment analysis unit 110, and the memory processing unit 102, as well as the function of transmitting the sentiment analysis results and the business category to the server device 70. The memory processing unit 102 implements the function of recording the sentiment analysis results and business category received from the user terminal 60, and the server device 70 implements the functions of the display processing unit 104. In other words, the user terminal 60 processes the association between the sentiment analysis results and the business category, the server device 70 stores and manages the information received by the user terminal 60, and displays the schedule screen 300 on the user terminal 60 for the user U to view.

[0078] The sentiment analysis results information 160 (sentiment analysis results) may include sentiment scores indicating the degree of positive and negative feelings towards User U (the subject) towards their work, categorized by work type. The emotion analysis unit 110 may identify an emotion score for each work category based on the emotion analysis results for each work category of user U over a predetermined period. The predetermined period may be any period, such as the most recent month, six months, or one year. For example, the emotion score may be identified as the percentage of time in which the score value indicating mental state falls within the above-mentioned positive emotion or negative emotion domain. More specifically, the emotion score may be identified as the percentage of time estimated to be positive or negative emotion relative to the total work time.

[0079] Figure 16 shows an example of the data structure for 170 pieces of sentiment information categorized by business function. The work-specific sentiment information 170 stores, for each work category, at least one of a positive score and a negative score associated with the start and end dates of a predetermined period. The work-specific sentiment information 170 is stored in the memory 1030, storage device 1040, or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70.

[0080] The display processing unit 104 displays a screen showing the ranking of the sentiment scores for each task for user U. The task ranking screen 330 may be displayed when the display operation for the task ranking screen 330 is selected from the operation menu.

[0081] Figure 17 shows an example of the business ranking screen 330. The business ranking screen 330 includes a category display unit 332, a legend display unit 312, and a score display unit 334. The display processing unit 104 displays the business categories on the business ranking screen 330 in descending order of negative scores, for example, based on the information stored in the business-specific sentiment information 170. Switching between displaying negative and positive scores may be handled by radio buttons (not shown) or the like.

[0082] Furthermore, the display processing unit 104, in the schedule screen 300 showing future plans, The schedule, The degree of positive and negative emotions of user U towards the work category is distinguishably displayed using a color or display element corresponding to the work category. For example, the color may be shaded according to the degree of each emotion. For instance, the stronger the emotion of "ANGRY," the darker the red, and the weaker the emotion of "ANGRY," the lighter the red.

[0083] As shown in the schedule screen 300 of Figure 15, the display processing unit 104 may refer to the business-specific sentiment information 170 and display the display elements 322 and color-coded displays 324 in a distinguishable manner based on the sentiment score corresponding to the business category of the schedule.

[0084] <Example of operation> Figure 18 is a flowchart showing an example of the operation of the information processing device 100 of the embodiment. Here, we will explain an example of the division of functions in (a1). The flow in Figure 18 includes the same steps S101 and S103 as the flow in Figure 2, and also includes steps S121 and S123. However, each step in this flowchart can operate independently, and as described above, the timing of execution for each step may differ.

[0085] First, at the user terminal 60, the acquisition unit 114 acquires sentiment analysis data of user U, and at the same time acquires business information 150 related to the work that user U was engaged in (step S121).

[0086] The sentiment analysis data includes biometric information of user U obtained from a wearable device 50 worn by user U. The obtained biometric information is stored as biometric information 130 in the memory 1030 or storage device 1040 of the user terminal 60, associated with time information. The time information is preferably the time when the biometric information was measured by the wearable device 50, but it may also be the time when the biometric information was received from the wearable device 50, or the time when it was stored as biometric information 130. The obtained business information 150 is stored in the memory 1030 or storage device 1040 of the user terminal 60.

[0087] The emotion analysis unit 110 performs an emotion analysis of user U using the emotion analysis data of user U acquired by the acquisition unit 114 from user U's wearable terminal 50, and outputs the emotion analysis result (step S123). The emotion analysis result is output as a score value indicating user U's mental state, for example. The output score value is stored in the memory 1030 or storage device 1040 of the user terminal 60 as score history information 140, associated with time information. The time information is the time information of the biometric information 130 used to estimate the score value.

[0088] The memory processing unit 102 stores the sentiment analysis results of user U as sentiment analysis result information 160, associating it with the category of work that user U was engaged in at the time the sentiment analysis results were analyzed (step S101). The sentiment analysis result information 160 is stored in the memory 1030 or storage device 1040 of the user terminal 60, associating the sentiment analysis results, sentiment classification, negative flag, and work category with the time information.

[0089] The display processing unit 104 displays the schedule screen 300, which displays the user U's schedule, by category of scheduled or previously performed tasks, associating each task with the sentiment analysis result corresponding to the task category (step S103).

[0090] As described above, according to this embodiment, the information processing device 100 comprises a storage processing unit 102, a display processing unit 104, an emotion analysis unit 110, and an acquisition unit 114. The acquisition unit 114 acquires emotion analysis data of user U and, when acquiring the emotion analysis data, acquires work information related to the work that user U was engaged in. The emotion analysis unit 110 performs an emotion analysis of user U using the emotion analysis data of user U acquired by the acquisition unit 114 from user U's wearable terminal 50, and outputs the emotion analysis results. The storage processing unit 102 stores the emotion analysis results of user U output by the emotion analysis unit 110 as emotion analysis result information 160, associating them with the category of work that user U was engaged in when the emotion analysis results were analyzed. The display processing unit 104 displays the schedule screen 300 that displays user U's schedule, associating each scheduled or previously performed work category with the emotion analysis results corresponding to the work category.

[0091] As described above, the information processing device 100 of this embodiment makes it easier to understand the impact of work on the mental state of the subject by visualizing the subject's emotions during work. Furthermore, as mentioned above, the display processing unit 104 displays the user U's emotions for each task on the schedule in a distinguishable manner, so for past schedules, it becomes possible to visualize what kind of emotions user U had while performing tasks. Therefore, if user U was mentally fatigued, it may be possible to identify the task that caused it. Also, for future schedules, the display processing unit 104 visualizes user U's emotions for each task, so if tasks that are likely to cause user U negative emotions are concentrated in a certain period, it becomes immediately clear that this is the case. Therefore, it becomes easier to make schedule adjustments such as distributing tasks to others or moving task schedules to different days.

[0092] Furthermore, by visualizing User U's emotions towards each task, it becomes possible to understand which tasks User U excels at. For example, tasks that User U is likely to evoke positive emotions in are likely to be tasks that User U excels at. Therefore, for example, Manager M can consider User U's strengths (tasks that are likely to evoke positive emotions) and weaknesses (tasks that are likely to evoke negative emotions) and appropriately assign tasks to them.

[0093] This could potentially prevent user U from suffering from mental fatigue due to work, or allow user U to recognize if they are experiencing excessive stress due to work, enabling appropriate countermeasures and potentially preventing a deterioration of their mental state. Furthermore, administrator M will be able to more easily understand user U's strengths and weaknesses, enabling them to provide appropriate training to user U.

[0094] Furthermore, the display processing unit 104 can also display rankings of user U's work categories, which can be used as a reference when scheduling work, thereby preventing excessive workload on user U.

[0095] (Second Embodiment) Figure 19 is a functional block diagram showing an example of the logical configuration of the information processing device 100 in this embodiment. This embodiment differs from the above embodiment in that it has a configuration that displays keywords contained in the conversational audio of user U, categorized by business category and associated with the emotional state of user U at the time of that business. However, the configuration of this embodiment may be combined with at least one of the configurations of the other embodiments to the extent that it does not cause any inconsistency.

[0096] <System Overview> The information processing system 1 of this embodiment may have either the system configuration shown in Figure 5 or the system configuration shown in Figure 6, as described in the above embodiment.

[0097] <Example of Functional Configuration> The information processing device 100 includes the same memory processing unit 102, display processing unit 104, and extraction unit 106 as the information processing device 100 in Figure 3, and further includes an emotion analysis unit 110 and an acquisition unit 114. The emotion analysis unit 110 performs an emotion analysis of user U (the subject) using emotion analysis data acquired from user U's wearable terminal 50, and outputs the emotion analysis results. The emotion analysis unit 110 and acquisition unit 114 are the same as those shown in Figure 8.

[0098] The acquisition unit 114 acquires sentiment analysis data of user U, and at the same time acquires business information related to the work that user U was engaged in. The sentiment analysis data includes biometric information of user U acquired from the wearable terminal 50 worn by user U. The biometric information acquired by the acquisition unit 114 is stored as biometric information 130 (Figure 9) in the memory 1030 or storage device 1040 or storage device 120 of the computer 1000 that realizes the user terminal 60 or server device 70.

[0099] The sentiment analysis results output by the sentiment analysis unit 110 are stored as score history information 140 (Figure 11) in the memory 1030, storage device 1040, or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70.

[0100] Furthermore, the emotion analysis unit 110 may, similar to the above embodiment, estimate whether user U's emotions fall within the negative domain and identify cases in which it is estimated that user U is experiencing negative emotions.

[0101] The memory processing unit 102 may store the sentiment analysis results of user U as sentiment analysis result information 160 (Figure 13) in the memory 1030 or storage device 1040 or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70, associating the sentiment analysis results of user U with the category of work that user U was engaged in at the time the sentiment analysis results were analyzed.

[0102] Furthermore, the emotion analysis unit 110 may, similar to the embodiment described above, identify an emotion score for each work category based on the emotion analysis results for each work category of user U over a predetermined period. For example, the emotion score may be the percentage of time in which the score value indicating the mental state falls within the positive emotion or negative emotion range described above. More specifically, the emotion score may be the percentage of time estimated to be positive or negative emotion relative to the total work time.

[0103] The extraction unit 106 processes the conversational audio of user U at the time the sentiment analysis result was analyzed, and extracts keywords that include at least one of the words and phrases contained in the conversational audio. The display processing unit 104 displays a screen containing information that associates the user U's emotional classification regarding the work with keywords contained in the conversation audio while the user is engaged in the work.

[0104] The display processing unit 104 may, for example, display keywords included in the conversational audio during work that correspond to at least one of the emotion categories, such as the negative emotions "ANGRY" and "SAD," or the positive emotions "HAPPY" and "RELAXED." The display of keywords for negative emotions and keywords for positive emotions may be switched using radio buttons (not shown) or the like.

[0105] Alternatively, the display processing unit 104 may display, for each of the four emotion categories, the keywords contained in the conversational audio during work that correspond to that emotion.

[0106] Figure 20 is a functional block diagram showing an example of a modified configuration of the information processing device 100 in the embodiment. In this example, the information processing device 100 further includes a counting unit 112 that counts the number of occurrences of each keyword contained in the conversational audio during work. The display processing unit 104 may display the keywords on the screen in order of frequency.

[0107] There are various ways to display keywords, but for example, they may be displayed in a word cloud format. That is, the more frequently a keyword appears as counted by the counting unit 112, the larger the letters representing the keyword, or the larger the shape surrounding the letters representing the keyword.

[0108] Figure 21 shows an example of the keyword ranking screen 350. The keyword ranking screen 350 may be displayed by selecting the display operation for the keyword ranking screen 350 from the operation menu. In this example, the keyword ranking screen 350 includes a keyword display unit 352 and an occurrence count display unit 354. The keyword display unit 352 displays keywords with a high occurrence count at the top. The occurrence count display unit 354 may, for example, display the number of occurrences of a keyword in the conversational audio during work that was the subject of sentiment analysis, or it may display the ratio of the number of occurrences of a keyword to the total number of keywords extracted from the conversational audio during work that was the subject of sentiment analysis.

[0109] Furthermore, similar to the above embodiment, the regions that identify categories of negative emotions and categories of positive emotions may be determined by whether or not they are included within a fan-shaped region indicated by a predetermined angle within the region of each emotion category.

[0110] The division of functions between the user terminal 60 and the server device 70, and the information processing device 100, is illustrated below. (b1) The system configuration shown in Figure 5 does not include a server device 70, and all functions are performed by the user terminal 60. The following is an example of functional division in the system configuration shown in Figure 6, which includes a user terminal 60 and a server device 70. (b2) The user terminal 60 implements the function of the acquisition unit 114 to acquire biometric information from the wearable terminal 50, and the server device 70 implements the function of the acquisition unit 114 to acquire biometric information from the user terminal 60, as well as the functions of the emotion analysis unit 110, memory processing unit 102, extraction unit 106, counting unit 112, and display processing unit 104. In other words, the user terminal 60 acquires biometric information from the wearable terminal 50 and transmits it to the server device 70, and the server device 70 uses the biometric information received by the user terminal 60 to perform emotion analysis processing, voice processing, etc., stores and manages the information, and displays the keyword ranking screen 350 on the user terminal 60 for the user U to view. (b3) Of the functions of the server device 70 described in (b2) above, the functions of the extraction unit 106 and the counting unit 112 may be implemented by the user terminal 60. (b4) Of the functions of the server device 70 described in (b2) above, the function of the emotion analysis unit 110 may be implemented by the user terminal 60. (b5) The user terminal 60 implements the functions of the acquisition unit 114, sentiment analysis unit 110, memory processing unit 102, extraction unit 106, and counting unit 112, as well as the function of transmitting the sentiment analysis results and the business category to the server device 70. The memory processing unit 102 implements the function of recording the sentiment analysis results and business category received from the user terminal 60, and the server device 70 implements the functions of the display processing unit 104. In other words, the user terminal 60 processes the association of sentiment analysis results with business categories, as well as keyword extraction and counting. The server device 70 stores and manages the information received by the user terminal 60, and displays the keyword ranking screen 350 on the user terminal 60 for user U to view.

[0111] <Example of operation> Figure 22 is a flowchart showing an example of the operation of the information processing device 100 of the embodiment. Here, we will explain the example of functional division of (b2). The flow in Figure 22 includes steps S101, S111, and S113, which are the same as in the flow in Figure 4, and also includes steps S121 and S123, which are the same as in the flow in Figure 18 of the above embodiment. However, each step in the flowchart can operate independently, and as described above, the timing at which each step is executed may differ.

[0112] First, in the user terminal 60, the acquisition unit 114 acquires emotion analysis data (biometric information) of user U (step S121). The acquired biometric information is associated with time information and stored as biometric information 130 in the memory 1030 or storage device 1040 of the user terminal 60. In the server device 70, the acquisition unit 114 receives the emotion analysis data (biometric information 130) of user U from the user terminal 60 and stores it in the storage device 120, associated with the identification information of user U.

[0113] In the server device 70, the acquisition unit 114 acquires business information 150 related to the work that user U was engaged in when acquiring the sentiment analysis data (step S121). Then, in the server device 70, the sentiment analysis unit 110 performs sentiment analysis on user U using the sentiment analysis data of user U acquired by the acquisition unit 114 and outputs the sentiment analysis results (step S123).

[0114] Then, in the server device 70, the memory processing unit 102 stores the sentiment analysis results of user U as sentiment analysis result information 160, associating them with the category of work that user U was engaged in at the time the sentiment analysis results were analyzed (step S101).

[0115] Then, in the server device 70, the extraction unit 106 processes the conversational audio of user U at the time the sentiment analysis result was analyzed and extracts keywords that include at least one of the words and phrases contained in the conversational audio (step S111). Then, in the server device 70, the display processing unit 104 displays a screen containing information that associates the classification of user U's feelings toward the work with the keywords contained in the conversational audio while engaged in the work, categorized by work category (step S113).

[0116] The display processing unit 104 of the server device 70 displays a screen on the display of the user terminal 60. Alternatively, the display processing unit 104 of the server device 70 may display a screen on the display of the administrator terminal 80, which will be described in a later embodiment.

[0117] Figure 23 is a flowchart showing the process of counting keywords. In the server device 70, after keywords are extracted from the conversation in step S111 of Figure 22, the counting unit 112 counts the number of occurrences for each keyword (step S131). Then, in the server device 70, the display processing unit 104 arranges the keywords in descending order of frequency and displays them on the keyword ranking screen 350 (Figure 21) (step S133). This screen is also displayed on the display of at least one of the user terminal 60 and the administrator terminal 80.

[0118] As described above, according to this embodiment, the information processing device 100 includes a storage processing unit 102, a display processing unit 104, an extraction unit 106, an emotion analysis unit 110, and an acquisition unit 114. The emotion analysis unit 110 performs an emotion analysis on user U (the subject) using emotion analysis data acquired from user U's wearable terminal 50, and outputs the emotion analysis results. The acquisition unit 114 acquires emotion analysis data from user U, and at the same time acquires work information related to the work that user U was engaged in. The memory processing unit 102 stores the emotion analysis results of user U, associating them with the category of work that user U was engaged in or the category of work that indicated that user U was on break when the emotion analysis results were analyzed. The extraction unit 106 processes the conversational audio of user U at the time the emotion analysis results were analyzed and extracts keywords that include at least one of the words and phrases contained in the conversational audio. The display processing unit 104 displays a screen containing information that associates the user U's emotional classification for work with the keywords contained in the conversational audio while engaged in that work.

[0119] Thus, according to the information processing device 100 of this embodiment, by visualizing the emotions of the subject during work, it becomes easier to understand the impact of work on the subject's mental state.

[0120] (Third embodiment) <System Overview> Figure 24 is a conceptual diagram showing an example of the system configuration of the information processing system 1 according to the embodiment. In addition to the configuration of Information Processing System 1 shown in Figure 6, Information Processing System 1 also includes an administrator terminal 80. The administrator terminal 80 is a terminal used by administrator M who manages user U at the company where user U works, and is a computer 1000 such as a personal computer, tablet terminal, or smartphone. The administrator terminal 80 can communicate with user terminals 60 via the communication network 3.

[0121] <Example of Functional Configuration> This embodiment is the same as any of the embodiments described above, except that it has a configuration that allows the various screens displayed by the display processing unit 104 of the above embodiment to also be displayed on the administrator terminal 80. Here, the information processing device 100 will be described as having the configuration of the first embodiment, and will be explained using Figure 8. However, the configuration of this embodiment may be combined with at least one of the configurations of embodiments other than the first embodiment, to the extent that it does not cause any inconsistencies.

[0122] In this embodiment, the work includes handling customer phone calls at a call center. The person described as User U in the above embodiment will be described as an operator in this embodiment. The emotion analysis unit 110 performs emotion analysis on an operator (target person) basis for each telephone interaction and outputs the emotion analysis results. Alternatively, the emotion analysis unit 110 may perform emotion analysis by processing the operator's conversation audio. In that case, the emotion analysis unit 110 may perform emotion analysis using biometric information such as the operator's pulse rate. The memory processing unit 102 stores the operator's emotion analysis results output by the emotion analysis unit 110 in the memory 1030, storage device 1040, or storage device 120 of the computer 1000 that implements the user terminal 60 or server device 70, on a per-telephone interaction basis. The display processing unit 104 associates the operator's emotional state during a phone call with a keyword for each phone call, and displays a screen (for example, the report screen 360 described later).

[0123] Furthermore, the division of functions between the user terminal 60 and the server device 70 of the information processing device 100 will be explained using the example of (b2) described above.

[0124] Figures 25 and 26 show examples of the report screen 360 displayed on the administrator terminal 80. However, the report screen 360 may also be displayed on the user terminal 60. The report screen 360 may be displayed by selecting the display operation for the report screen 360 from the operation menu.

[0125] The report screen 360 is a screen used in call centers and similar facilities to create or view reports that operators submit to record the content of phone calls they receive. The report screen 360 includes a display section that shows information such as the report identification information (in this case, the report number), the date and time the call was received, the operator's name (receptionist), the type of inquiry, the product in question (product name, model number, etc.), the content of the inquiry, the content of the response to the inquiry, and the attributes of the customer who made the call (gender, age group, etc.). If it is a report editing screen, the report screen 360 may include appropriate UI elements such as text input fields and menus for selecting categories, and may accept input from the user.

[0126] The "keywords" associated with the operator's emotional classification described above correspond to the keywords included in the inquiry's target product (such as product name or model number), the telephone inquiry classification, the inquiry content, the response to the inquiry, and the response content, all of which are displayed on the report screen 360. The display processing unit 104 can distinguish keywords included in the conversation audio that correspond to the negative emotional classifications of "ANGRY" and "SAD" on the report screen 360, in particular, by using color and at least one of the display elements. For example, the display processing unit 104 may change the color or highlight keywords included in the conversation audio that correspond to the negative emotional classification among the keywords included in the inquiry content.

[0127] Furthermore, when the emotion analysis result display button 370 is pressed, the display processing unit 104 transitions to the report screen 360 shown in Figure 26. The report screen 360 in Figure 26 is an additional emotion analysis result display unit 362 to the report screen 360 in Figure 25. In the report screen 360 in Figure 26, the emotion analysis result display button 370 is displayed in an inverted state compared to the emotion analysis result display button 370 in Figure 25, for example, to indicate that it is selected.

[0128] In the server device 70, the display processing unit 104 displays the emotional classifications in a way that they can be distinguished by at least one of color and display elements. For example, the emotion analysis result display unit 362 of the report screen 360 displays at least one of the display element 364 and the score graph 366. The display element 364 may be the same as the display element 322 of the schedule screen 300 described above. For example, the display element 364 may show the emotion category that had the highest proportion during the call.

[0129] As shown in Figure 27, the score graph 366 is a graph that shows the distribution of the operator's emotions based on the emotion analysis results of the operator during the call that was the subject of the report. For example, the display processing unit 104 identifies which category the operator's emotions were in during the call based on the score history information 140, measures the time for each emotion category, and displays the percentage of the call time for each emotion category in the score graph 366.

[0130] As described above, according to this embodiment, the information processing device 100 comprises a storage processing unit 102, a display processing unit 104, an extraction unit 106, an emotion analysis unit 110, and an acquisition unit 114. The emotion analysis unit 110 performs an emotion analysis of the operator on a per-telephone call basis and outputs the emotion analysis results. The storage processing unit 102 receives the output from the emotion analysis unit 110. Operator The emotion analysis results are stored for each phone call. The display processing unit 104 associates the operator's emotional category during the call with keywords and displays them on the screen for each phone call.

[0131] As described above, the information processing device 100 of this embodiment makes it easier to understand the impact of work on the mental state of operators by visualizing the emotions of operators during telephone calls in a call center. In particular, since the report screen 360 can be displayed on the administrator terminal 80 of administrator M, administrator M can understand the mental state of operators along with the work report, and take appropriate action.

[0132] For example, by displaying keywords extracted from phone conversations in association with emotions, it is possible to estimate the keywords causing fluctuations in the operator's emotions. This allows for understanding the elements that the operator is feeling anxious about on their own, and enables measures to alleviate that anxiety, such as strengthening study or education in areas related to the elements causing anxiety, or improving the FAQ (Frequently Asked Questions) section of the phone answering manual.

[0133] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0134] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments can be combined to the extent that their contents do not conflict.

[0135] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments. Various modifications to the structure and details of the present invention can be made, as can be understood by those skilled in the art within the scope of the present invention. Furthermore, when acquiring or using user information in this invention, such acquisition or use shall be conducted lawfully.

[0136] Some or all of the above embodiments may also be described as follows, but are not limited to the following: 1. A memory processing means that stores the results of an emotional analysis of a subject in a memory means, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. An information processing apparatus comprising: a display processing means for displaying the schedule of the subject, in a screen that displays each scheduled or previously performed task by category, in association with the sentiment analysis results corresponding to the task category. 2. In the information processing device described in 1., The display processing means is an information processing device that displays the schedule in a way that distinguishes it by category of work using colors or display elements corresponding to the sentiment analysis results of the subject. 3. In the information processing device described in 1. or 2., The emotion analysis results include an emotion score indicating the degree of positive and negative emotions of the subject towards the work for each category of work. The display processing means is an information processing device that displays a screen showing the ranking of the emotional scores of the subject for each of the aforementioned tasks. 4. In the information processing device described in 3. The aforementioned display processing means, on a schedule screen showing future plans, The schedule, An information processing device that displays, in a distinguishable manner, the degree of positive and negative feelings of the subject person toward the work corresponding to the category of work, using a color or display element. 5. In the information processing device described in any one of items 1 to 4, An information processing device comprising emotion analysis means that performs emotion analysis on the subject using emotion analysis data of the subject obtained from the subject's terminal and outputs the emotion analysis results. 6. A memory processing means that stores the results of the emotional analysis of a subject in a memory means, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. Extraction means for processing the conversational audio of the subject at the time the sentiment analysis results were analyzed, and extracting keywords that include at least one of the words and phrases contained in the conversational audio, Information processing apparatus comprising: display processing means for displaying a screen containing information relating the classification of the subject's feelings toward the said work to the said keywords contained in the said conversation audio while engaged in said work. In the information processing device described in 7.6, The system further comprises a counting means for counting the number of occurrences of each keyword included in the conversational audio while the person is engaged in the aforementioned work. The display processing means is an information processing device that arranges the keywords in order of their frequency of occurrence and displays them further on the screen. 8. In the information processing device described in 6. or 7., The aforementioned duties include customer telephone support at a call center. The system further includes an emotion analysis means that performs an emotion analysis of the subject for each telephone interaction and outputs the emotion analysis results. The memory processing means stores the emotion analysis results of the subject, output by the emotion analysis means, in the memory means on a per-telephone interaction basis. The display processing means is an information processing device that, for each telephone interaction, associates the emotional classification of the person during the telephone interaction with the keyword and displays the screen. 9. In the information processing device described in any one of items 6 to 8, The display processing means is an information processing device that displays the distinction of emotions in a distinguishable manner using at least one of color and display elements. 10. In the information processing device described in any one of items 1 to 9, An information processing apparatus comprising means for acquiring data for emotional analysis of the subject, and for acquiring business information related to the work the subject was engaged in when acquiring the emotional analysis data.

[0137] 11. One or more computers, The results of the emotional analysis of the subject are stored in a memory device, associated with the category of work the subject was engaged in at the time the emotional analysis was performed. An information processing method that, on a screen displaying the schedule of the subject, displays each scheduled or previously performed task by category in association with the sentiment analysis results corresponding to the task category. 12. In the information processing method described in 11., The aforementioned one or more computers An information processing method that displays the aforementioned schedule in a way that allows it to be distinguished by category of work using colors or display elements corresponding to the emotional analysis results of the subject. 13. In the information processing method described in 11. or 12., The emotion analysis results include an emotion score indicating the degree of positive and negative emotions of the subject towards the work for each category of work. The aforementioned one or more computers An information processing method that displays a screen showing the ranking of the emotional scores of the subject persons for each of the aforementioned tasks. 14. In the information processing method described in 13., The aforementioned one or more computers On the schedule screen showing upcoming plans, The schedule, An information processing method that displays, in a distinguishable manner, the degree of positive and negative feelings of the subject person toward the work corresponding to the category of work, using a color or display element. 15. In the information processing method described in any one of 11 to 14, The aforementioned one or more computers An information processing method that performs emotional analysis on a subject using emotional analysis data obtained from the subject's terminal and outputs the results of the emotional analysis. 16. One or more computers, The results of the emotional analysis of the subject are stored in a memory device, associated with the category of work the subject was engaged in at the time the emotional analysis was performed. The conversational audio of the subject engaged in the aforementioned work, which was the subject of the sentiment analysis results, is processed to extract keywords that include at least one of the words and phrases contained in the conversational audio. An information processing method that displays a screen containing information relating the classification of the subject's feelings towards the work and the keywords contained in the conversation audio while the subject is engaged in the work. 17. In the information processing method described in 16., The aforementioned one or more computers further The number of occurrences of each keyword in the conversation audio while engaged in the aforementioned work is counted. An information processing method that displays the aforementioned keywords on the screen in order of their frequency of occurrence. 18. In the information processing method described in 16. or 17., The aforementioned duties include customer telephone support at a call center. The aforementioned one or more computers further For each telephone interaction, an emotional analysis of the subject is performed, and the emotional analysis results are output. The emotional analysis results of the subject that have been output are stored in the storage means on a per-telephone interaction basis. An information processing method that, for each telephone interaction, associates the emotional category of the person at the time of the telephone interaction with the keyword and displays the screen. 19. In any one of the information processing methods described in 16. to 18., The aforementioned one or more computers An information processing method for displaying the aforementioned emotional classifications in a distinguishable manner using at least one of color and display elements. 20. In the information processing method described in any one of items 11 to 19, The aforementioned one or more computers further An information processing method that acquires data for emotional analysis of the subject, and at the time of acquiring the emotional analysis data, acquires business information related to the work that the subject was engaged in.

[0138] 21. To the computer, A procedure for storing the results of an emotional analysis of a subject in a memory device, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. A computer-readable recording medium storing a program for executing a procedure to display, on a screen that displays the schedule of the subject, each scheduled or previously performed task by category in association with the sentiment analysis results corresponding to the task category. 22. In the recording medium described in 21., A computer-readable recording medium storing a program that causes the computer to execute a procedure for displaying the aforementioned schedule, categorized by the category of the aforementioned work, in a manner distinguishable by color or display element corresponding to the emotional analysis results of the subject. 23. In the recording medium described in 21. or 22., The emotion analysis results include an emotion score indicating the degree of positive and negative emotions of the subject towards the work for each category of work. A computer-readable recording medium storing a program for causing the computer to execute a procedure for displaying a screen showing the ranking of the emotional scores for each of the aforementioned tasks of the subject. 24. In the recording medium described in 23., On the schedule screen showing upcoming plans, The schedule, A computer-readable recording medium storing a program for causing the computer to execute a procedure for displaying, in a manner distinguishable by color or display element, the degree of positive and negative feelings of the subject person toward the work corresponding to the category of work. 25. In the recording medium described in any one of 21 to 24, A computer-readable recording medium that stores a program for causing the computer to further execute a procedure for performing an emotional analysis of the subject using emotional analysis data of the subject obtained from the subject's terminal, and outputting the emotional analysis results. 26. To the computer, A procedure for storing the results of an emotional analysis of a subject in a memory device, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. A procedure for processing the conversational audio of the subject engaged in the aforementioned work, which was the subject of the sentiment analysis results, and extracting keywords that include at least one of the words and phrases contained in the conversational audio. A computer-readable recording medium storing a program for causing a procedure to display a screen containing information relating the subject's emotional classification to the aforementioned work to the keywords contained in the conversational audio while the subject is engaged in the said work. 27. In the recording medium described in 26., A procedure for counting the number of occurrences of each keyword in the conversation audio while engaged in the aforementioned work, A computer-readable recording medium storing a program that causes the computer to further execute a procedure for displaying the aforementioned keywords on the screen in order of their frequency of occurrence. 28. In the recording medium described in 26. or 27., The aforementioned duties include customer telephone support at a call center. A procedure for performing an emotional analysis of the subject for each telephone interaction and outputting the emotional analysis results. A procedure for storing the emotional analysis results of the subject, outputted on a per-telephone call basis, in the storage means. A computer-readable recording medium storing a program that causes the computer to further execute a procedure for displaying the screen by associating the emotional classification of the person at the time of the telephone interaction with the keyword, for each telephone interaction unit. 29. In the recording medium described in any one of 26. to 28., A computer-readable recording medium storing a program for causing the computer to perform a procedure for displaying the aforementioned distinctions of emotion in a distinguishable manner using at least one of color and display elements. 30. In any one of the recording media described in 21. to 29., A computer-readable recording medium storing a program that causes the computer to execute a procedure for acquiring emotional analysis data of the subject, and, when acquiring the emotional analysis data, acquiring business information related to the work the subject was engaged in.

[0139] 31. To the computer, A procedure for storing the results of an emotional analysis of a subject in a memory device, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. A program for causing a screen displaying the schedule of the subject to be performed to display each scheduled or previously performed task by category in association with the sentiment analysis results corresponding to the task category. 32. The program described in 31. Mu In, A program for causing the computer to perform a procedure for displaying the aforementioned schedule, categorized by the type of work, using colors or display elements corresponding to the results of the emotion analysis of the subject. 33. In the program described in 31. or 32., The emotion analysis results include an emotion score indicating the degree of positive and negative emotions of the subject towards the work for each category of work. A program for causing the computer to perform a procedure to display a screen showing the ranking of the emotional scores for each of the aforementioned tasks for the subject. 34. In the program described in 33., On the schedule screen showing upcoming plans, The schedule, A program for causing a computer to perform a procedure for displaying, in a manner distinguishable by color or display element, the degree of positive and negative feelings of the subject person toward the work corresponding to the category of work. 35. In any one of the programs described in 31. through 34., A program that causes the computer to further execute a procedure for performing an emotional analysis of the subject using emotional analysis data of the subject obtained from the subject's terminal, and outputting the emotional analysis results. 36. To the computer, A procedure for storing the results of an emotional analysis of a subject in a memory device, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. A procedure for processing the conversational audio of the subject engaged in the aforementioned work, which was the subject of the sentiment analysis results, and extracting keywords that include at least one of the words and phrases contained in the conversational audio. A program for causing the user to perform a procedure for displaying a screen containing information that associates the subject's emotional classification with the aforementioned work with the keywords contained in the aforementioned conversation audio while the subject is engaged in the said work. 37. In the program described in 36., A procedure for counting the number of occurrences of each keyword in the conversation audio while engaged in the aforementioned work, A program that causes the computer to further execute a procedure for displaying the aforementioned keywords on the screen in order of their frequency of occurrence. 38. In the programs described in 36. or 37., The aforementioned duties include customer telephone support at a call center. A procedure for performing an emotional analysis of the subject for each telephone interaction and outputting the emotional analysis results. A procedure for storing the emotional analysis results of the subject, outputted on a per-telephone call basis, in the storage means. A program that causes the computer to further execute a procedure for displaying the screen by associating the emotional classification of the person in question during the telephone call with the keyword, for each telephone call unit. 39. In any one of the programs described in 36. to 38., A program for causing the computer to perform a procedure for displaying the aforementioned distinctions of emotion in a distinguishable manner using at least one of color and display elements. 40. In any one of the programs described in 31. through 39., A program that causes the computer to execute a procedure for acquiring emotional analysis data of the subject, and, when acquiring the emotional analysis data, acquiring business information related to the work the subject was engaged in. [Explanation of Symbols]

[0140] 1. Information Processing System 3. Communication Network 50 wearable devices 60 User Terminals 70 Server Devices 80 Administrator terminal 100 Information Processing Devices 102 Memory Processing Unit 104 Display Processing Unit 106 Extraction part 110 Emotion Analysis Department 112 Counting Unit 114 Acquisition Department 120 Storage device 130 Biometric Information 140 Score History Information 150 Business Information 160 Sentiment analysis result information 170. Sentiment Information by Business Role 300 Schedule screen 310 Legend List 320 Emotion Analysis Results Display Button 330 Business Ranking Screen 350 Keyword Ranking Screen 360 report screen 362 Emotion analysis result display section 364 display elements 366 Score Graph 370 Emotion Analysis Results Display Button 1000 computers 1010 Bus 1020 Processor 1030 memory 1040 Storage Devices 1050 Input / Output Interface 1060 Network Interfaces

Claims

1. A memory processing means that stores the results of an emotional analysis of a subject in a memory means, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. The system includes a display processing means that displays each scheduled or previously performed task by category in relation to the sentiment analysis results corresponding to the task category on a screen displaying the subject's schedule, The emotion analysis results include an emotion score indicating the degree of positive and negative emotions of the subject towards the work for each category of work. The display processing means is an information processing device that displays a screen showing the ranking of the emotional scores of the subject for each of the aforementioned tasks.

2. In the information processing apparatus according to claim 1, The display processing means is an information processing device that, on a schedule screen showing future plans, displays the schedule in a way that distinguishes it using colors or display elements that indicate at least one degree of positive or negative feelings towards the work of the subject, corresponding to the category of work.

3. In the information processing apparatus according to claim 1, The display processing means is an information processing device that, on a screen displaying schedules that have been performed in the past, displays the schedules in a way that distinguishes them using colors or display elements that indicate at least one degree of positive or negative feelings of the subject towards the work, corresponding to the category of the work.

4. In the information processing apparatus according to any one of claims 1 to 3, An information processing device comprising emotion analysis means that performs emotion analysis on the subject using emotion analysis data of the subject obtained from the subject's terminal and outputs the emotion analysis results.

5. One or more computers, The results of the emotional analysis of the subject are stored in a memory device, associated with the category of work the subject was engaged in at the time the emotional analysis was performed. On the screen displaying the schedule of the subject, each scheduled or previously performed task by category is displayed in association with the sentiment analysis results corresponding to the task category. The emotion analysis results include an emotion score indicating the degree of positive and negative emotions of the subject towards the work for each category of work. An information processing method in which, when one or more computers display the schedule, a screen is displayed showing the ranking of the emotional scores of the subject persons for each of the tasks.

6. On the computer, A procedure for storing the results of an emotional analysis of a subject in a memory device, associating them with the category of work the subject was engaged in at the time the emotional analysis was performed. On the screen displaying the subject's schedule, the procedure is performed to display each scheduled or previously performed task by category in association with the sentiment analysis results corresponding to the task category. The emotion analysis results include an emotion score indicating the degree of positive and negative emotions of the subject towards the work for each category of work. A program for displaying, in the procedure described above, a screen showing the ranking of the subject's emotional score for each of the aforementioned tasks.

Images