Information processing device and information processing method

Abstract

To enable a worker to work comfortably by presenting the work considering the psychological state of the worker.SOLUTION: An information processing device comprises: an acquisition unit that acquires psychological determination information that is information for determining a psychological state of a user; a determination unit that determines the psychological state of the user based on the psychological determination information; a storage unit that stores past psychological states of the user and work information including work contents of the user and work amounts in the past mental states in association with each other; and a decision unit that acquires work information associated with a past psychological state which is the same as a current psychological state of the user from the storage unit, and decides a presentation work content to be presented to the user based on the acquired work information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing method.

Background Art

[0002] In a workplace or the like, an operator may not be able to properly convey their physical and mental discomfort to those around them even when they are unwell. If a manager assigns work without understanding the operator's discomfort, the work content and workload assigned to the operator may not be appropriate for the operator's health and mental state. Patent Document 1 discloses a technique for generating a work schedule based on biometric information such as an operator's skill and body temperature.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Even if a work schedule is generated based on measurable information such as biometric information, if the mental state of the operator is not considered, the work load on the operator with physical and mental discomfort will increase, and the mental state of the operator may deteriorate.

[0005] One aspect of the present invention aims to present work that takes into account the mental state of the operator so that the operator can work comfortably.

Means for Solving the Problems

[0006] To achieve the above object, the present invention employs the following configuration.

[0007] One aspect of this disclosure is an information processing device comprising: an acquisition unit that acquires psychological judgment information, which is information for determining the psychological state of a user; a determination unit that determines the psychological state of a user based on the psychological judgment information; a storage unit that stores the user's past psychological state in association with work information, including the content and amount of work performed by the user in that past psychological state; and a determination unit that acquires work information from the storage unit that is associated with the user's current psychological state and the same past psychological state, and determines the content of the work to be presented to the user based on the acquired work information.

[0008] Psychological assessment information includes information used to determine the user's psychological state, such as facial images, voice, and responses to questionnaires regarding psychological state. The presented tasks are those presented to the user based on past work information from users in the same psychological state as the assessed user. The information processing device can determine the presented tasks based on the acquired work information, for example, pre-set task priorities and changes in workload due to differences in psychological state. Because tasks are presented that take the worker's psychological state into consideration, the worker can work comfortably.

[0009] The work information associated with past psychological states further includes information about the user's psychological state after performing the work described in the work information. The decision-making unit may determine the work to present based on work information associated with the same past psychological state as the current psychological state, where the psychological state after the work has improved compared to before the work. Since work in which the psychological state has improved after completion is presented preferentially, work efficiency is improved, and the worker can work more comfortably.

[0010] The work information associated with past psychological states further includes information about the user's psychological state after the work information was presented, and the decision-making unit may determine the work to be presented based on work information associated with past psychological states that are the same as the current psychological state, where the psychological state after the work presentation has improved compared to before the work presentation. Since work in which the psychological state has improved after being presented with the work content in the past is given priority, work efficiency is improved, and the worker can work more comfortably.

[0011] A user's psychological state may be assigned a score for each state, and it may be determined whether the user's psychological state has improved based on changes in the score. This makes it easier for the information processing device to determine whether the psychological state has improved.

[0012] The decision-making unit may determine the content of the task to be presented based on at least one of the following: the degree to which the user's psychological state has improved, or the confidence level in the judgment of the user's improved psychological state, if there are multiple pieces of task information that correspond to the same past psychological state as the current psychological state. The decision-making unit can accurately determine whether or not the psychological state has improved and then determine the content of the task to be presented.

[0013] Psychological assessment information includes captured images of the user's face or part of their face. The assessment unit may determine the user's psychological state using a trained model that has been trained with the captured images of the face or part of the face and the user's psychological state at the time of capture as training data. The assessment unit can determine the user's psychological state from facial expressions, etc. Furthermore, by making a determination based on captured images of a part of the face that is likely to reveal the psychological state, the assessment unit can capture subtle changes in facial expressions and determine the user's psychological state more accurately.

[0014] The determination unit may identify the user by facial recognition and determine the user's psychological state using a trained model that has been trained on captured images of the user's face or a part of the user's face and their psychological state at the time of capture as training data. By using a trained model that has been trained on captured images of the person identified by facial recognition, the determination unit can determine the user's psychological state more accurately.

[0015] The psychological assessment information includes information indicating the positional relationship between the user's face and hands. The assessment unit may use a trained model, which has been trained using the information indicating the positional relationship between the user's face and hands and the user's psychological state at the time of shooting as training data, to determine the user's psychological state. The assessment unit can determine the user's psychological state more accurately based on the user's habits indicated by the positional relationship between the face and hands.

[0016] The determination unit may detect the user's face from the captured image taken when the user aligns their face within the frame displayed on the display unit. The determination unit can omit the process of detecting a face from the captured image.

[0017] Psychological assessment information includes audio data of the user's voice, and the assessment unit may determine the user's psychological state using a trained model that has been trained using the user's voice audio data and their psychological state at the time of recording as training data. Even if the information processing device is not equipped with a camera, the assessment unit can determine the user's psychological state based on the user's voice audio data.

[0018] Psychological assessment information includes the user's responses to a questionnaire designed to determine the user's psychological state, and the assessment unit may determine the user's psychological state based on the questionnaire responses. Even when it is difficult to determine the psychological state from facial expressions or tone of voice, the assessment unit can determine the user's psychological state more accurately.

[0019] When the determination unit determines the user's psychological state using multiple pieces of psychological judgment information, it may determine the psychological state with the highest confidence level among the psychological states determined from each piece of psychological judgment information as the user's psychological state. The confidence level is the confidence level when determining the user's psychological state using a predetermined trained model corresponding to the acquired psychological judgment information. By adopting the psychological state with the higher confidence level, the determination unit can determine the user's psychological state with high accuracy.

[0020] The information processing device may include a display unit that presents the task details determined by the decision unit to the user. The user can confirm their task before starting. The task details presented to the user may also be presented to the administrator. The administrator can manage the overall work schedule by understanding the task details of each user.

[0021] The information processing device may include an output unit that stores in a memory unit the psychological state of the user determined by the determination unit, the work information including the content of the proposed work determined by the decision unit and the amount of work performed by the user, in association with each other. As the information on the psychological state and the corresponding work information is stored in the memory unit, the information processing device becomes able to present appropriate work according to the user's psychological state.

[0022] Furthermore, one aspect of this disclosure is an information processing method in which a computer performs the following steps: an acquisition step of acquiring psychological judgment information, which is information for determining the psychological state of a user; a determination step of determining the psychological state of a user based on the psychological judgment information; and a determination step of acquiring work information associated with the same past psychological state as the user's current psychological state from a storage unit that stores in association with the user's past psychological states and work information including the content and amount of work performed by the user in those past psychological states, and determining the content of work to be presented to the user based on the acquired work information.

[0023] Furthermore, one aspect of this disclosure is a program that causes a computer to function as one of the means of the above-mentioned information processing device, or a computer-readable storage medium that non-temporarily stores said program.

Advantages of the Invention

[0024] According to the present invention, it is possible to present work that takes into account the mental state of the operator and enable the operator to work comfortably.

Brief Description of the Drawings

[0025] [Figure 1] FIG. 1 is a diagram showing an application example and a hardware configuration example of an information processing apparatus. [Figure 2] FIG. 2 is a diagram illustrating the functional configuration of the information processing apparatus. [Figure 3] FIG. 3 is a flowchart illustrating the work presentation process according to Embodiment 1. [Figure 4] FIG. 4 is a diagram illustrating the data of the work information storage unit in Embodiment 1. [Figure 5] FIG. 5 is a flowchart illustrating the work presentation process according to Embodiment 2. [Figure 6] FIG. 6 is a diagram illustrating the data of the work information storage unit in Embodiment 2. [Figure 7] FIG. 7 is a flowchart illustrating the work presentation process according to Embodiment 3. [Figure 8] FIG. 8 is a diagram illustrating the data of the work information storage unit in Embodiment 3. [Figure 9] FIG. 9 is a flowchart illustrating the acquisition process of the psychological determination information according to Modification Example 4-1. [Figure 10] FIG. 10 is a diagram for explaining a part of the face in Modification Example 4-2. [Figure 11] FIG. 11 is a flowchart illustrating the acquisition process of the psychological determination information according to Modification Example 4-2. [Figure 12] FIG. 12 is a diagram for explaining the positional relationship between the face and the hand in Modification Example 4-3. [Figure 13] FIG. 13 is a flowchart illustrating the acquisition process of the psychological determination information according to Modification Example 4-3. [Figure 14]Figure 14 is a flowchart illustrating the process for acquiring psychological judgment information related to Modification 4-4. [Figure 15] Figure 15 is a diagram illustrating the questionnaire for modified example 4-5. [Figure 16] Figure 16 is a flowchart illustrating the process of acquiring psychological judgment information related to Modification 4-5. [Modes for carrying out the invention]

[0026] Hereinafter, embodiments relating to one aspect of the present invention will be described based on the drawings.

[0027] <Examples of application> Referring to Figure 1, an example of the application of an information processing device that determines the psychological state of a user (worker) and presents work content according to that psychological state will be explained. The information processing device acquires information for determining the user's psychological state (hereinafter also referred to as psychological determination information), such as an image of the user's face, their voice, and their responses to a questionnaire about their psychological state. Based on the acquired psychological determination information, the information processing device determines the user's psychological state. For example, the information processing device can prepare a trained model that has been trained using an image of the user's face and their psychological state at the time of shooting as training data, and then determine the psychological state by inputting the image of the user's face into the trained model.

[0028] The information processing device determines and presents to the user the content of the work that corresponds to the psychological state of the user it has determined. The information processing device is equipped with a storage unit that stores, for each user, information on past psychological states and the content and amount of work performed in those past psychological states (hereinafter also referred to as work information) in association with each user. The information processing device can determine the content of the work to present to the user (hereinafter also referred to as presented work content) based on past work information from the same psychological state as the determined user.

[0029] When multiple pieces of past work information corresponding to the current user's psychological state exist, the information processing device can determine the content of the tasks to be presented based on the priority of the tasks and the change in workload due to a deterioration in the psychological state. Task priorities may be set in advance for each user when the manager creates the overall work schedule, taking into account the impact of the user's skills or psychological state on the tasks. The information processing device may also determine the content of the tasks to be presented by prioritizing tasks that exceed a baseline workload (for example, the average workload of multiple other users working under the same psychological state), i.e., tasks for which the user has higher skills than other users.

[0030] Figure 1 shows an example where the user was determined to be experiencing fatigue. Since the amount of work done during document creation is significantly reduced compared to when the user is relaxed, the information processing device presents "picking," which has a less significant reduction in workload than "document creation," as the task for today.

[0031] In this way, the information processing device can determine the user's psychological state and, based on the type and amount of work performed in the past under the same psychological state, present work content that is appropriate for the user's current psychological state. This allows users to work comfortably according to their psychological state. Furthermore, once the user's psychological state is determined, administrators can understand the amount of work that would be performed under that psychological state from past work information, making it easier to manage work schedules.

[0032] <Embodiment 1> (Hardware configuration) Referring to Figure 1, an example of the hardware configuration of the information processing device 10 will be described. The information processing device 10 includes a processor 11, a main memory unit 12, an auxiliary memory unit 13, an operation unit 14, a display unit 15, a camera 16, a microphone 17, and a communication unit 18. The processor 11 reads the program stored in the auxiliary memory unit 13 into the main memory unit 12 and executes it, thereby realizing the functions of each functional configuration described in Figure 2.

[0033] The operation unit 14 is, for example, a keyboard, mouse, or touch panel, and accepts user input. The display unit 15 is a display or the like. The camera 16 is, for example, a webcam or front-facing camera, a camera for capturing images of the user in front of the display. The microphone 17 picks up the user's voice and other sounds. The communication unit 18 is an interface for wired or wireless communication.

[0034] The information processing device 10 is a general-purpose computer such as a personal computer, server computer, tablet terminal, or smartphone. The information processing device 10 may be implemented, for example, by distributed computing using multiple computer devices, and some of its functional units may be implemented by a cloud server. In addition, some of the functional units of the information processing device 10 may be implemented by dedicated hardware devices such as FPGAs or ASICs.

[0035] (Functional Configuration) The functional configuration of the information processing device 10 will be explained with reference to Figure 2. Figure 2 is a diagram illustrating the functional configuration of the information processing device 10. The information processing device 10 includes an acquisition unit 101, a psychological state determination unit 102, a work information storage unit 103, a work content determination unit 104, and an output unit 105.

[0036] The acquisition unit 101 acquires psychological determination information, which is information for determining the user's psychological state. Psychological determination information is, for example, a facial image from which the psychological state can be estimated from facial expressions, etc. The psychological state determination unit 102 determines the user's psychological state at the time of shooting based on the psychological determination information. The psychological state determination unit 102 determines the user's psychological state using a trained model generated using deep learning such as a CNN (Convolutional Neural Network), with the facial image and the psychological state at the time of shooting as training data. The acquisition unit 101 and the psychological state determination unit 102 are examples of an "acquisition unit" and a "determination unit," respectively.

[0037] The work information storage unit 103 is a storage unit that stores, for each user, information (work information) relating to their past psychological state and the content and amount of work performed in that past psychological state. The work information storage unit 103 stores and accumulates the user's psychological state as determined by the psychological state determination unit 102, and the content and amount of work actually performed by the user in that psychological state.

[0038] The actual work performed by the user and the amount of work performed may be entered by the user themselves from the operation unit 14, or if it is possible to acquire and measure the work content and amount of work using a work terminal, etc., they may be transmitted from the work terminal. Furthermore, since the amount of work may change even in the same psychological state, the information processing device may store the average value of the amount of work performed over the most recent predetermined number of times (for example, 5 times) in the same psychological state as the amount of work performed in that psychological state. The work information storage unit 103 is an example of a "storage unit".

[0039] The work content determination unit 104 obtains work information from the work information storage unit 103 that corresponds to the user's current psychological state as determined by the psychological state determination unit 102, and that is associated with the user's past psychological state. Based on the obtained work information, the work content determination unit 104 determines the work content to be presented to the user. The work content determination unit 104 is an example of a "determination unit".

[0040] The output unit 105 presents the proposed work content determined by the work content determination unit 104 to the user by displaying it on the display unit 15. The output unit 105 also associates the user's psychological state determined by the psychological state determination unit 102 with the proposed work content determined by the work content determination unit 104 and the amount of work performed by the user, and stores this information in the work information storage unit 103. The output unit 105 is an example of a "presentation unit" and an "output unit".

[0041] Although the above functions are described assuming they are implemented by a single computer, they may be implemented by multiple computers. For example, in situations such as remote work, the acquisition unit 101 of the information processing device 10 may receive psychological judgment information acquired by the user's work terminal via the communication unit 18. In addition, the output unit 105 may transmit the presented work content determined by the work content determination unit 104 to the user's work terminal via the communication unit 18 and display it on the work terminal.

[0042] (Work assignment process flow) Referring to Figure 3, the workflow of the task presentation process according to Embodiment 1 will be explained. The task presentation process shown in Figure 4 is started, for example, when a user who is about to start a task turns their face towards the display (display unit 15).

[0043] In S101, the acquisition unit 101 acquires psychological judgment information. The psychological judgment information is, for example, an image of the user's face, which is captured by the camera 16. In S102, the psychological state determination unit 102 determines the user's psychological state based on the image of the user's face. The psychological state determination unit 102 identifies the user's face from the captured image and determines the user's psychological state by inputting the captured image of the face and the psychological state at the time of capture into a trained model that is generated as training data. The captured image of the face used as training data may be an image of the user's own face, or it may be an image of another user's face.

[0044] In S103, the work content determination unit 104 obtains past work information from the work information storage unit 103 that is associated with the same past psychological state as the user's psychological state determined in S102.

[0045] Here, with reference to Figure 4, past psychological states and past work information stored in the work information storage unit 103 will be described. Figure 4 is a diagram illustrating the data of the work information storage unit in Embodiment 1. Psychological states are, for example, predefined psychological states such as fatigue, relaxation, irritation, tension, worry, and calmness, but are not limited to these.

[0046] Work information includes information on work content and workload. Work content can be predefined by the administrator. Work workload can be the amount of work per unit time. Unit time may be specified in days, hours, or minutes. In addition, different unit times may be specified for each task. The psychological state and work information exemplified in Figure 4 are stored in the work information storage unit 103 for each user.

[0047] In S104, the work content determination unit 104 determines the work content to present to the user based on past work information acquired in S103. For example, if in S102 the user's psychological state is determined to be "fatigue," the work content determination unit 104 acquires work information for "picking," "stocking," and "document creation" in the example shown in Figure 4. If multiple work information is available, the work content determination unit 104 may, for example, determine the presented work content based on the first work information acquired, or it may determine the presented work content based on a predetermined priority order among the multiple work information.

[0048] Furthermore, if multiple pieces of work information exist for a single psychological state, the work content determination unit 104 The task content may be determined by combining multiple tasks. The task content determination unit 104 may also determine the priority of tasks based on the user's skills and the total remaining workload, and determine the task content to be presented based on the priority. Furthermore, the task content determination unit 104 may determine which tasks to present to the user are those in which the workload fluctuates less due to changes in psychological state compared to other tasks.

[0049] In S105, the output unit 105 displays the task to be performed, determined in S104, on the display to the user. The output unit 105 may also display the amount of work performed in the past as an estimate of the amount of work, along with the task to be performed.

[0050] In Embodiment 1, the information processing device 10 determines the user's current psychological state based on an image of the user's face, and then determines the type of work the user should perform based on information about past work performed under the same psychological state and the amount of work performed. This allows the information processing device 10 to present tasks while considering the worker's psychological state and adjust the worker's workload.

[0051] <Embodiment 2> In Embodiment 1, the information processing device 10 determines the content of the task to be presented based on past work information performed in the same psychological state as the user's current psychological state. In Embodiment 2, the information processing device 10 determines the content of the task to be presented based on whether or not the user's psychological state has improved after performing the task in the same psychological state as the user's current psychological state.

[0052] The hardware configuration and functional configuration of the information processing device 10 according to Embodiment 2 are the same as those of the information processing device 10 according to Embodiment 1, so a description is omitted. Figure 5 is a flowchart illustrating the task presentation process according to Embodiment 2. Processes that are the same as those in the task presentation process shown in Figure 3 (Embodiment 1) are denoted by the same reference numerals and their descriptions are omitted.

[0053] In S204, the work content determination unit 104 determines the work content to be presented to the user based on the user's psychological state after the work. In Embodiment 2, the work information storage unit 103 stores the user's psychological state after actually performing the work as one of the work information associated with the psychological state.

[0054] The user's psychological state after a task can be determined at the moment the user turns their face towards the display (display unit 15), such as immediately after the task or before starting work the following day, and stored in the work information storage unit 103. For tasks performed while looking at the display, the psychological state determination unit 102 may also determine the user's psychological state during the task and, if there is a change, store the changed psychological state in the work information storage unit 103.

[0055] Here, with reference to Figure 6, the past psychological state and past work information stored in the work information storage unit 103 will be described. Figure 6 is a diagram illustrating the data of the work information storage unit in Embodiment 2. In Embodiment 2, the work information includes not only the work content and amount of work, but also information on the psychological state after the work. The items other than the psychological state after the work (psychological state, work content, amount of work) are the same as in Figure 4, so their explanation will be omitted.

[0056] In the example in Figure 6, when the psychological state was "fatigue," after "picking," the psychological state changed to "a sense of accomplishment." Also, when the psychological state was "fatigue," after "document creation," the psychological state changed to "irritation." The work information storage unit 103 stores the psychological state before and after the work in association with each other. This allows the work content determination unit 104 to determine whether the psychological state has improved before and after the work.

[0057] Whether or not the psychological state after the work has improved can be determined, for example, as follows: Psychological states are classified in advance into positive and negative groups, and the work content determination unit 104 can determine that the psychological state has improved if it changes from a negative psychological state to a positive psychological state. For example, the negative psychological states may be "fatigue," "irritation," "tension," and "worry," while the positive psychological states may be "relaxation," "calmness," and "sense of accomplishment."

[0058] In the example in Figure 6, when a person's psychological state is "fatigue" and they perform "picking," their psychological state after the task is "a sense of accomplishment." Their psychological state has changed from "fatigue" (a negative group) to "a sense of accomplishment" (a positive group), indicating an improvement. On the other hand, when a person's psychological state is "fatigue" and they perform "document creation," their psychological state after the task is "irritation." Their psychological state has changed from "fatigue" (a negative group) to "irritation" (another negative group), indicating no improvement.

[0059] Furthermore, whether or not the psychological state has improved is not limited to classifying the psychological state into two groups, but may also be determined by classifying the psychological state into three or more groups. In addition, a score is assigned to each of the user's psychological states, and whether or not the user's psychological state has improved is determined according to the change in the score. The work content determination unit 104 can determine that the psychological state has improved if the score for the psychological state has increased after the work is completed.

[0060] The work content determination unit 104 determines the presented work content, for example, as follows: If the user's current psychological state is "fatigue," the work content determination unit 104, in the example shown in Figure 6, obtains two pieces of work information, "picking" and "document creation," from the work information storage unit 103 as candidates for presented work content. The user's psychological state improves to "a sense of accomplishment" after the "picking" task, but remains "irritated" and does not improve after the "document creation" task. In this case, the work content determination unit 104 can determine "picking," which improves the user's psychological state, as the presented work content.

[0061] Furthermore, if multiple work options exist for a single psychological state, the work content determination unit 104 may determine the presented work content based on the degree to which the user's psychological state has improved. The degree to which the psychological state has improved can be, for example, determined by assigning a score to each psychological state, and then subtracting the score of the psychological state before the work from the score of the psychological state after the work. The work content determination unit 104 will then determine the work content that shows the greatest degree of improvement in the psychological state after the work as the presented work content.

[0062] Furthermore, if multiple work options exist for a single psychological state, the work content determination unit 104 may determine the presented work content based on the confidence level of the judgment of the user's psychological state after improvement. Here, the confidence level is the confidence level when judging the user's psychological state after the work using a predetermined trained model corresponding to the acquired psychological judgment information. The work content determination unit 104 determines the work content with a higher confidence level in the judgment of the user's psychological state after the work as the presented work content.

[0063] In Embodiment 2, the information processing device 10 determines the content of the tasks to be presented to the user based on the user's psychological state after performing the task. Tasks that improve the user's psychological state after completion are given priority, thus improving work efficiency and allowing the user to work more comfortably.

[0064] <Embodiment 3> In Embodiment 2, the information processing device 10 determines the content of the task to be presented based on the psychological state after performing the same task in the past while in the same psychological state as the current psychological state. In Embodiment 3, information The processing unit 10 determines the task to be presented based on the psychological state after being presented with the same task in the past while in the same psychological state as the current psychological state.

[0065] The hardware configuration and functional configuration of the information processing device 10 according to Embodiment 3 are the same as those of the information processing device 10 according to Embodiment 1, so a description is omitted. Figure 7 is a flowchart illustrating the task presentation process according to Embodiment 3. Processes that are the same as those in the task presentation process shown in Figure 3 (Embodiment 1) are denoted by the same reference numerals and their descriptions are omitted.

[0066] In S304, the work content determination unit 104 determines the content of the work to be presented to the user based on the user's psychological state after the work is presented. In Embodiment 3, the work information storage unit 103 stores the user's psychological state after the work content is presented as one of the work information items associated with the psychological state.

[0067] The psychological state after task presentation refers to the psychological state after the task content has been presented. The psychological state after task presentation is determined from the user's facial image captured after the task content has been presented and can be stored in the task information storage unit 103.

[0068] Here, with reference to Figure 8, the past psychological state and past work information stored in the work information storage unit 103 will be explained. Figure 8 is a diagram illustrating the data of the work information storage unit in Embodiment 3. In Embodiment 3, the work information includes not only the work content and workload, but also information on the psychological state after the work is presented. The items other than the psychological state after the work is presented (psychological state, work content, workload) are the same as in Figure 4, so their explanation will be omitted.

[0069] In the example in Figure 8, when the psychological state is "fatigue" and "picking" is presented, the psychological state after the task is presented becomes "worried." Also, when the psychological state was "fatigue," after document creation is presented, the psychological state changes to "calm." The task information storage unit 103 stores the psychological state before the task is presented and the psychological state after the task is presented in association with each other. This allows the task content determination unit 104 to determine whether the psychological state has improved before and after the task is presented. Whether the psychological state has improved can be determined in the same way as in Embodiment 2.

[0070] The work content determination unit 104 determines the presented work content, for example, as follows: If the user's current psychological state is "fatigue," the work content determination unit 104, in the example shown in Figure 8, obtains two pieces of work information, "picking" and "document creation," from the work information storage unit 103 as candidates for presented work content. After the presentation of the "picking" task, the user's psychological state remains "worried" and has not improved, but after the presentation of the "document creation" task, it has improved to "calm." In this case, the work content determination unit 104 can determine "document creation," which represents an improvement in the user's psychological state, as the presented work content.

[0071] Furthermore, if multiple work options exist for a single psychological state, the work content determination unit 104 may determine the presented work content based on the degree to which the user's psychological state has improved. The degree to which the user's psychological state has improved can be, for example, determined by assigning a score to each psychological state, by subtracting the score of the psychological state before the work was presented from the score of the psychological state after the work was presented. The work content determination unit 104 will then determine the work content that shows the greatest degree of improvement in the psychological state after the work was presented as the presented work content.

[0072] Furthermore, if multiple work pieces of information exist for a single psychological state, the work content determination unit 104 may determine the presented work piece based on the confidence level of the judgment of the user's psychological state after improvement. Here, the confidence level is the confidence level when the user's psychological state after the work is presented is determined using a predetermined trained model. The work content determination unit 104 determines the presented work piece based on the user's psychological state after the work is presented. The work tasks that offer a higher level of confidence in determining the state of the environment will be selected as the proposed work tasks.

[0073] In Embodiment 3, the information processing device 10 determines the content of the task to be presented to the user based on the user's psychological state after the task has been presented. Tasks in which the user's psychological state has improved after being presented with the task in the past are given priority, thus improving work efficiency and allowing the worker to work more comfortably.

[0074] <Embodiment 4> Embodiment 4 is an embodiment relating to various modifications of the method for acquiring psychological judgment information and determining psychological state. Each modification can be implemented in combination with each of Embodiments 1 to 3. The hardware configuration and functional configuration of the information processing device 10 according to Embodiment 4 are the same as those of the information processing device 10 according to Embodiment 1, so a description is omitted.

[0075] (Variation 4-1) Modification 4-1 is an example in which a user is identified using facial recognition technology, and the psychological state of the identified user is determined using a trained model that has been trained on the captured image of the user's face and their psychological state at the time of capture as training data. By using a trained model that has been trained on the user's captured images, the information processing device 10 can determine the user's psychological state more accurately.

[0076] Figure 9 is a flowchart illustrating the process for acquiring psychological judgment information related to Modification 4-1. The process shown in Figure 9 is a specific step of the processes S101 and S102 in Figures 3, 5, and 7.

[0077] In S111, the acquisition unit 101 identifies the user using known facial recognition technology. In S112, the acquisition unit 101 acquires an image of the user. The acquisition unit 101 may use the image acquired in S111 for facial recognition as the image acquired in S112. In S113, the acquisition unit 101 detects the user's face from the image acquired in S112 and acquires the detected face image as psychological judgment information.

[0078] In S114, the psychological state determination unit 102 inputs the face image detected in S113 into a pre-trained model that has been trained using the captured image of the user's face identified in S111 and the user's psychological state at the time of shooting as training data. The psychological state determination unit 102 can determine the user's psychological state from the output result of the pre-trained model.

[0079] (Modification 4-2) Modification 4-2 is an example of determining the user's psychological state using a trained model that has been trained using captured images of a portion of the face and the user's psychological state at the time of capture as training data. By making a determination based on captured images of a portion of the face that are likely to reveal the psychological state, the information processing device 10 can capture subtle changes in facial expressions and determine the user's psychological state more accurately.

[0080] Referring to Figure 10, a part of the face will be described. A part of the face is, for example, the upper region 1001 of the face including the eyes, and the lower region 1002 of the face including the nose and mouth. For users whose psychological state changes tend to appear in the upper region 1001 of the face, the psychological state may be determined using the captured image of region 1001, and for users whose psychological state changes tend to appear in the lower region 1002 of the face, the psychological state may be determined using the captured image of region 1002.

[0081] Furthermore, the part of the face is not limited to the upper region 1001 and the lower region 1002 of the face, but may also be facial features such as the eyes, nose, and mouth. In addition, the information processing device 10, in combination with modified example 4-1, uses an image of a part of a specific user's face and their psychological state at the time of shooting as training data. The psychological state of the user may be determined using a pre-trained model that has been trained in this manner.

[0082] Figure 11 is a flowchart illustrating the process for acquiring psychological judgment information according to Modification 4-2. The process shown in Figure 11 is a specific example of the processes S101 and S102 in Figures 3, 5, and 7.

[0083] In S121, the acquisition unit 101 acquires an image of the user. In S122, the acquisition unit 101 selects a target (part of the face) for determining the psychological state. The target is, for example, the upper face region 1001 and the lower face region 1002 as described in Figure 10. Alternatively, the target may be facial features such as the eyes, nose, and mouth. The acquisition unit 101 can, for example, select a region or facial feature that has been pre-set for each user as the target for determination. The acquisition unit 101 may also select the target for determination based on the user's operation. In S123, the acquisition unit 101 detects the target for determination selected in S122 from the image acquired in S121 and acquires an image of the detected part of the face as psychological determination information.

[0084] In S124, the psychological state determination unit 102 inputs the image of the part of the face detected in S123 into a pre-trained model that has been trained using the captured image of a part of the face and the psychological state at the time of shooting as training data. The psychological state determination unit 102 can determine the user's psychological state from the output result of the pre-trained model.

[0085] (Modification 4-3) Modification 4-3 is an example of determining the user's psychological state using a trained model that has been trained using information indicating the positional relationship between the face and hands, and the user's psychological state at the time of shooting, as training data. The information processing device 10 can more accurately determine the user's psychological state based on the user's habits indicated by the positional relationship between the face and hands.

[0086] Referring to Figure 12, the positional relationship between the face and hands will be explained. The rectangle 1200 encloses the position of the face detected from the captured image. The hands of a user who has a habit of scratching or touching their head are detected in the head region 1201. The hands of a user who has a habit of scratching or touching their ears are detected in the left and right face regions 1202. The hands of a user who has a habit of covering or touching their mouth or nose are detected in the lower face region 1203. The hands of a user who has a habit of placing their hands near their heart are detected in the chest region 1204.

[0087] Furthermore, the information processing device 10 may not be limited to the position of the hand in the captured image, but may also record a video for a predetermined time (for example, about 5 to 10 minutes) and determine the user's psychological state based on the frequency with which the hand was moved to each region as described in Figure 12 within that predetermined time. In this case, the information processing device 10 can determine the user's psychological state using a trained model that has also learned the frequency with which the hand was moved to each region within the predetermined time as input data.

[0088] Figure 13 is a flowchart illustrating the process for acquiring psychological judgment information related to Modification 4-3. The process shown in Figure 13 is a specific example of the processes S101 and S102 in Figures 3, 5, and 7.

[0089] In S131, the acquisition unit 101 acquires an image of the user. In S132, the acquisition unit 101 detects the user's face from the image acquired in S131. In S133, the acquisition unit 101 detects the user's hand from the image acquired in S131.

[0090] Furthermore, when acquiring the user's image in S131, the acquisition unit 101 may display a frame on the display indicating the position of the face and prompt the user to position their face within the frame. By acquiring a user's image so that its face is included, the acquisition unit 101 can omit the face detection process in S132.

[0091] In S133, the acquisition unit 101 acquires information indicating the positional relationship between the face and the hand as psychological judgment information. For example, the acquisition unit 101 can acquire, as part of the positional relationship between the face and the hand, which region of the region illustrated in Figure 12 in which the user's hand was detected. The information indicating the positional relationship between the face and the hand may also include captured images of the face and hand, the ratio of the area where a part of the face and the hand overlap to the original area, and the frequency with which the hand overlaps with a part of the face.

[0092] In S134, the psychological state determination unit 102 inputs the information indicating the positional relationship between the face and hands, acquired in S133, into a pre-trained model that has been trained using information indicating the positional relationship between the face and hands and the psychological state at the time of shooting as training data. The psychological state determination unit 102 can determine the user's psychological state from the output result of the pre-trained model.

[0093] (Modification 4-4) Modification 4-4 is an example of determining the user's psychological state using a trained model that has been trained using the user's voice audio data and their psychological state at the time of recording as training data. The information processing device 10 prompts the user to speak by asking them a question. The trained model can determine the user's psychological state by learning data such as voice tremor or pitch and the corresponding psychological state. The information processing device 10 can determine the user's psychological state based on the user's voice audio data even if the camera 16 is not available.

[0094] Figure 14 is a flowchart illustrating the process for acquiring psychological judgment information related to Modification 4-4. The process shown in Figure 14 is a specific example of the processes S101 and S102 in Figures 3, 5, and 7.

[0095] In S141, the acquisition unit 101 acquires the user's voice audio data as psychological judgment information. In S142, the psychological state determination unit 102 inputs the audio data acquired in S141 into a pre-trained model that has been trained in advance using the voice audio data and the user's psychological state at the time of recording as training data. The psychological state determination unit 102 can determine the user's psychological state from the output result of the pre-trained model.

[0096] (Modifications 4-5) Modification 4-5 is an example of conducting a questionnaire to determine the user's psychological state and determining the user's psychological state based on the user's responses to the questionnaire. Even when it is difficult to determine the psychological state from facial expressions or tone of voice, the user's psychological state can be determined more accurately by appropriately setting the questionnaire questions and answer options.

[0097] Refer to Figure 15 to explain the questionnaire. The questionnaire questions should be designed to easily estimate the user's psychological state, such as "How are you feeling today?", "Did you sleep well?", and "Did anything fun happen?". The information processing device 10 may display the next question in response to the user's answers to the questionnaire questions. For example, the questionnaire questions can be a set of questions in which the next question is selected based on the user's answers, narrowing down the possible psychological states.

[0098] Figure 16 is a flowchart illustrating the process for acquiring psychological judgment information related to the modified example 4-5. The process shown in Figure 16 is a specific example of the processes S101 and S102 in Figures 3, 5, and 7.

[0099] In S151, the acquisition unit 101 receives questionnaire questions to determine the user's psychological state. The following is displayed. In S152, the acquisition unit 101 receives the user's response to the questionnaire. In S153, the psychological state determination unit 102 can determine the user's psychological state from the user's response to the questionnaire.

[0100] (others) The psychological state determination unit 102 may combine multiple modifications of the above modifications to determine the user's psychological state using multiple psychological determination information. In this case, the psychological state determination unit 102 can determine the psychological state with the highest confidence level among the psychological states determined using the corresponding trained model from each piece of psychological determination information as the user's psychological state.

[0101] In Embodiment 4, the information processing device 10 can more accurately determine the user's psychological state by acquiring various information (psychological determination information) from the user for determining the user's psychological state. The determination of the user's psychological state is performed, for example, before the work, after the work is presented, and after the work is performed. Note that the psychological state before the work may be determined each time the work content changes. The psychological state after the work is performed is not limited to immediately after the work is performed, but may be determined before the start of the work the next day. Furthermore, the psychological state after the work is performed is not limited to after the work is completed, but may be determined during the work.

[0102] Furthermore, the user's psychological state is not limited to being determined based on psychological assessment information acquired at a specific point in time; for example, it may be determined based on psychological assessment information acquired over a predetermined period (e.g., video footage taken during that period).

[0103] <Other> The embodiments and modifications described above are merely illustrative examples illustrating the configuration of the present invention. The configurations of the embodiments and modifications are not limited to the specific forms described above and can be appropriately combined and used within the scope of the technical idea of ​​the present invention. Furthermore, the present invention can be modified in various ways without departing from its technical idea.

[0104] <Note> (1) An acquisition unit (101) that acquires psychological assessment information, which is information for determining the user's psychological state, A determination unit (102) that determines the user's psychological state based on the aforementioned psychological determination information, A storage unit (103) that stores the user's past psychological state and work information including the user's work content and workload during that past psychological state, in association with each other. The system includes a determination unit (104) that acquires work information from the storage unit that corresponds to the user's current psychological state and the user's past psychological state, and determines the content of the work to be presented to the user based on the acquired work information. Information processing device (10).

[0105] (2) Computer (10) The acquisition step (S101) involves obtaining psychological assessment information, which is information used to determine the user's psychological state, A determination step (S102) is performed to determine the user's psychological state based on the aforementioned psychological assessment information, A storage unit stores the user's past psychological state and work information, including the user's work content and workload, in association with the user's past psychological state. From this storage unit, the user retrieves the work information associated with the user's current psychological state and the past psychological state, and based on the retrieved work information, the user determines the content of the work to be presented to the user (S103, S104). Information processing methods. [Explanation of Symbols]

[0106] 1: Information processing unit, 11: Processor, 12: Main memory unit, 13: Auxiliary memory unit, 14: Operation unit, 15: Display unit, 16: Camera, 17: Microphone, 18: Communication unit 101: Acquisition unit, 102: Psychological state determination unit, 103: Work information storage unit, 104: Work content determination unit 104, 105: Output unit

Claims

1. An acquisition unit that acquires psychological assessment information, which is information for determining the user's psychological state, A determination unit that determines the user's psychological state based on the aforementioned psychological determination information, A storage unit that stores the user's past psychological state and work information including the user's work content and workload during that past psychological state, in association with each other. The system includes: a storage unit that acquires work information from the storage unit that corresponds to the user's current psychological state and the user's past psychological state, and a determination unit that determines the content of the work to be presented to the user before assigning the work based on the acquired work information; The work information associated with the past psychological state further includes information about the user's psychological state after performing the work described in the work information, The decision unit determines the content of the presented task based on the task information, which is associated with the same past psychological state as the current psychological state, and in which the psychological state after the task is improved compared to before the task. Information processing device.

2. An acquisition unit that acquires psychological assessment information, which is information for determining the user's psychological state, A determination unit that determines the user's psychological state based on the aforementioned psychological determination information, A storage unit that stores the user's past psychological state and work information including the user's work content and workload during that past psychological state, in association with each other. The system includes a determination unit that acquires work information from the storage unit that corresponds to the user's current psychological state and the user's past psychological state, and determines the content of the work to be presented to the user based on the acquired work information, The work information associated with the past psychological state further includes information about the user's psychological state after performing the work described in the work information, The decision unit determines the content of the presented task based on the task information, which is associated with the same past psychological state as the current psychological state, and in which the psychological state after the task is improved compared to before the task. Information processing device.

3. The work information associated with the past psychological state further includes information about the user's psychological state after the work content of the work information was presented, The decision unit determines the content of the task to be presented based on the task information, which is associated with the same past psychological state as the current psychological state, and in which the psychological state after the task presentation is better than before the task presentation. The information processing apparatus according to claim 1 or 2.

4. Each psychological state of the user is assigned a score, and it is determined whether or not the user's psychological state has improved based on the change in the score. The information processing apparatus according to any one of claims 1 to 3.

5. If there are multiple pieces of work information corresponding to the same past psychological state as the current psychological state, the decision unit determines the content of the presented work based on at least one of the degree to which the user's psychological state has improved and the confidence level of the determination of the user's improved psychological state. The information processing apparatus according to any one of claims 1 to 4.

6. The aforementioned psychological assessment information includes an image of the user's face or a part of their face. The determination unit determines the user's psychological state using a trained model that has been trained using captured images of the face or part of the face and the user's psychological state at the time of shooting as training data. The information processing apparatus according to any one of claims 1 to 5.

7. The determination unit identifies the user by facial recognition and determines the user's psychological state using a trained model that has been trained using captured images of the user's face or a part of the user's face and the user's psychological state at the time of capture as training data. The information processing apparatus according to any one of claims 1 to 6.

8. The aforementioned psychological assessment information includes information indicating the positional relationship between the user's face and hands. The determination unit determines the user's psychological state using a trained model that has been trained using information indicating the positional relationship between the user's face and hands, and the user's psychological state at the time of shooting, as training data. The information processing apparatus according to any one of claims 1 to 7.

9. The determination unit detects the user's face from the captured image taken when the user aligns their face within the frame displayed on the display unit. The information processing apparatus according to any one of claims 6 to 8.

10. The aforementioned psychological assessment information includes the voice data of the user's voice, The determination unit determines the user's psychological state using a trained model that has been trained using the user's voice data and their psychological state at the time of recording as training data. The information processing apparatus according to any one of claims 1 to 9.

11. The aforementioned psychological assessment information includes the user's responses to a questionnaire for determining the user's psychological state. The determination unit determines the user's psychological state based on the user's responses to the questionnaire. The information processing apparatus according to any one of claims 1 to 10.

12. When the determination unit determines the user's psychological state using a plurality of psychological determination pieces of information, it determines the psychological state with the highest reliability among the psychological states determined from each of the psychological determination pieces of information as the user's psychological state. The information processing apparatus according to any one of claims 1 to 11.

13. The system further includes a presentation unit that presents the content of the presentation work determined by the decision unit to the user. The information processing apparatus according to any one of claims 1 to 12.

14. The system further includes an output unit that stores in the storage unit the psychological state of the user determined by the determination unit, the work information including the content of the presented work determined by the decision unit and the amount of work performed by the user, in association with each other. The information processing apparatus according to any one of claims 1 to 13.

15. Computers An acquisition step to obtain psychological assessment information, which is information used to determine the user's psychological state, A determination step in which the user's psychological state is determined based on the aforementioned psychological assessment information, A decision step is performed in which a storage unit stores the user's past psychological state and work information including the user's work content and workload in the past psychological state in association with the user's past psychological state, retrieves the work information associated with the user's current psychological state and the past psychological state, and determines the content of the work to be presented to the user before assigning the work based on the retrieved work information. The work information associated with the past psychological state further includes information about the user's psychological state after performing the work described in the work information, The decision step determines the content of the proposed work based on the work information, which is associated with the same past psychological state as the current psychological state, and in which the psychological state after the work is improved compared to before the work. Information processing methods.

16. On the computer, An acquisition step to obtain psychological assessment information, which is information used to determine the user's psychological state, A determination step in which the user's psychological state is determined based on the aforementioned psychological assessment information, A decision step is performed in which a storage unit stores the user's past psychological state and work information, including the user's work content and workload, in association with the user's past psychological state, retrieves the work information associated with the user's current psychological state and the past psychological state, and determines the content of the work to be presented to the user before assigning the work based on the retrieved work information. The work information associated with the past psychological state further includes information about the user's psychological state after performing the work described in the work information, The decision step determines the content of the proposed work based on the work information, which is associated with the same past psychological state as the current psychological state, and in which the psychological state after the work is improved compared to before the work. program.

17. Computers An acquisition step to obtain psychological assessment information, which is information used to determine the user's psychological state, A determination step in which the user's psychological state is determined based on the aforementioned psychological assessment information, A decision step is performed in which a storage unit stores the user's past psychological state and work information including the user's work content and workload in the past psychological state in association with the user's past psychological state, retrieves the work information associated with the past psychological state that is the same as the user's current psychological state, and determines the content of the work to be presented to the user based on the retrieved work information. 、 The work information associated with the past psychological state further includes information about the user's psychological state after performing the work described in the work information, The decision step determines the content of the proposed work based on the work information, which is associated with the same past psychological state as the current psychological state, and in which the psychological state after the work is improved compared to before the work. Information processing methods.

18. On the computer, An acquisition step to obtain psychological assessment information, which is information used to determine the user's psychological state, A determination step in which the user's psychological state is determined based on the aforementioned psychological assessment information, A decision step is performed in which a storage unit stores the user's past psychological state and work information including the user's work content and workload in the past psychological state in association with the user's past psychological state, retrieves the work information associated with the past psychological state that is the same as the user's current psychological state, and determines the content of the work to be presented to the user based on the retrieved work information. The work information associated with the past psychological state further includes information about the user's psychological state after performing the work described in the work information, The decision step determines the content of the proposed work based on the work information, which is associated with the same past psychological state as the current psychological state, and in which the psychological state after the work is improved compared to before the work. program.

Images