Information processing method, information processing device, and information processing program

Abstract

This information processing method is executed by a computer and includes: acquiring the amount of fluctuation in the movement of a wrist of a user while they perform a task; judging the divergent thinking state of the user on the basis of the acquired amount of fluctuation; and outputting the results of judging the divergent thinking state.

Description

Information processing method, information processing device, and information processing program

[0001] The present disclosure relates to techniques for determining a user's state of mind.

[0002] It has been known that thinking and physical movement influence each other. For example, Non-Patent Document 1 reports that performing smooth arm movements beforehand improves performance on a subsequent creativity task.

[0003] Michael L. Slepian, Nalini Ambady, "Fluid Movement and Creativity", Journal of Experimental Psychology: General, 2012, Vol. 141, No. 4, p. 625-629

[0004] However, while the above-mentioned conventional technology discloses that the user's arm movements before performing a task affect the user's subsequent mental state, it does not determine the user's divergent thinking state from wrist movements during the task, and further improvement is needed.

[0005] The present disclosure aims to provide a technology that can easily determine a divergent thinking state from easily detectable wrist movements of a user while working.

[0006] An information processing method according to the present disclosure is an information processing method executed by a computer, and includes acquiring a variation in a user's wrist movement during work, and determining the user's distracted thinking state based on the variation. This comprehensive or specific aspect may be realized by an apparatus, a system, an integrated circuit, a computer program, or a computer-readable recording medium, or may be realized by any combination of an apparatus, a system, a method, an integrated circuit, a computer program, and a computer-readable recording medium. The computer-readable recording medium includes, for example, a non-volatile recording medium such as a CD-ROM (Compact Disc-Read Only Memory).

[0007] According to the present disclosure, a divergent thinking state can be easily determined from the easily detectable wrist movements of a user while working.

[0008] 1 is a diagram showing the results of a questionnaire administered after a convergent thinking task or a divergent thinking task in an experiment. FIG. 2 is a diagram showing the average amount of variation in wrist movement when a convergent thinking task or a divergent thinking task is being performed. FIG. 3 is a diagram showing a correlation coefficient between the amount of variation in wrist movement during a divergent thinking task and the questionnaire results. FIG. 4 is a diagram showing an example of a workspace to which an information processing method according to an embodiment of the present disclosure is applied. FIG. 5 is a block diagram showing an example of a configuration of a mental state assessment system according to an embodiment of the present disclosure. FIG. 6 is a diagram showing an example of position information and time information of a right wrist and a left wrist stored in a camera according to the present embodiment. FIG. 7 is a diagram showing an example of the amount of variation in the distance from an origin to the wrist of a non-dominant hand, an activity determination flag, and time information stored in a server according to the present embodiment. FIG. 8 is a schematic diagram for explaining a method of determining a first threshold and a second threshold according to the present embodiment. FIG. 9 is a sequence diagram for explaining the flow of processing by a camera, a server, and a terminal device according to the present embodiment. FIG. 10 is a flowchart for explaining dominant hand determination processing by a server according to the present embodiment. FIG. 11 is a flowchart for explaining activity determination processing by a server according to the present embodiment. FIG. 12 is a schematic diagram for explaining reference value determination processing according to the present embodiment. FIG. 13 is a flowchart for explaining reference value determination processing according to the present embodiment. 1 is a flowchart for explaining the thinking state determination process by the server in this embodiment; FIG. 2 is a first flowchart for explaining the thinking state determination process by the server in a modified example of this embodiment; FIG. 3 is a second flowchart for explaining the thinking state determination process by the server in a modified example of this embodiment; and FIG. 4 is a flowchart for explaining the questionnaire process by the server in this embodiment.

[0009] (Findings that form the basis of this disclosure) It is known that intellectual tasks performed by users include tasks that require convergent thinking and tasks that require divergent thinking. Convergent thinking is defined as thinking to arrive at a single answer by logically thinking and reasoning based on existing information. For example, convergent thinking is thinking about a question with a definite correct answer, or thinking to combine multiple ideas and reach one or more conclusions. Convergent thinking is also known as critical thinking.

[0010] Divergent thinking is the process of generating new ideas by thinking about known information in various ways. Specifically, divergent thinking is the way we generate new ideas that have no correct answer. Divergent thinking also includes the thinking used in brainstorming and other idea generation processes. Divergent thinking is also known as creative thinking.

[0011] A divergent thinking state refers to a state in which a person is engaged in divergent thinking. Here, a divergent thinking state may include a state in which a person is demonstrating creativity.

[0012] The above-mentioned conventional techniques disclose that the movement of a user's arm before performing a task affects the user's state of mind thereafter, but do not disclose how to determine the user's state of divergent thinking from wrist movement during a task.

[0013] In this disclosure, the following techniques are disclosed.

[0014] (1) An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, and includes acquiring a fluctuation amount of a user's wrist movement while working, and determining the user's divergent thinking state based on the fluctuation amount.

[0015] According to this configuration, there is a correlation between the user's wrist movements while working and the user's divergent thinking state while working, so the user's divergent thinking state can be easily determined from the user's wrist movements while working, which can be easily detected.

[0016] Furthermore, with this configuration, since the amount of variation in wrist movement is used, there is no need to invasively acquire biometric information, which reduces the burden on the user.

[0017] Furthermore, with this configuration, the amount of variation in wrist movement is used, so the user's divergent thinking state can be determined with a small amount of data.

[0018] (2) In the information processing method described in (1) above, the determination of the divergent thinking state may include determining that the user is in the divergent thinking state if the amount of variation is equal to or greater than a threshold.

[0019] According to this configuration, by comparing the amount of variation with the threshold value, it is possible to easily determine whether the user is in a divergent thinking state.

[0020] (3) The information processing method described in (2) above may further include acquiring evaluation information indicating the user's evaluation of the assessment result of the divergent thinking state, and changing the threshold value based on the evaluation information.

[0021] According to this configuration, the criteria for determining whether a state of divergent thinking is present can be changed by increasing or decreasing the threshold value, thereby enabling a more accurate determination of whether a state of divergent thinking is present.

[0022] (4) In the information processing method described in any one of (1) to (3) above, the method may further include determining the degree of the divergent thinking state of the user depending on the magnitude of the amount of variation.

[0023] According to this configuration, for example, if the amount of variation is large, it is determined that the degree of divergent thinking is high, and if the amount of variation is small, it is determined that the degree of divergent thinking is low, so it is possible to determine to what extent the user was in a divergent thinking state while working.

[0024] (5) In the information processing method described in any one of (1) to (4) above, the acquisition of the amount of variation may include calculating the amount of variation based on a change in the position of the wrist in a moving image of the wrist of the user.

[0025] According to this configuration, the amount of variation in the movement of the user's wrist can be easily calculated using a moving image of the user's wrist.

[0026] (6) The information processing method described in (5) above may further include acquiring positional information of a work medium used by the user in the work based on the moving image, and determining, based on the positional information, whether to use the amount of movement of either the user's right wrist or left wrist to determine the divergent thinking state.

[0027] With this configuration, the divergent thinking state can be determined using the amount of movement of the user's right wrist or left wrist, whichever is most suitable for determining the divergent thinking state.

[0028] (7) The information processing method described in (5) above may further include determining the user's dominant hand based on the moving image, and the determination of the divergent thinking state may include determining the divergent thinking state based on the amount of variation in the wrist movement of the hand other than the dominant hand.

[0029] With this configuration, there is a correlation between the wrist movement of the user's non-dominant hand while working and the user's divergent thinking state while working, so the divergent thinking state can be determined with higher accuracy from the wrist movement of the user's non-dominant hand while working.

[0030] (8) In the information processing method described in any one of (1) to (4) above, the acquisition of the amount of change may include calculating the amount of change based on a physical quantity including at least one of angular velocity and acceleration detected by a sensor worn on the wrist of the user.

[0031] With this configuration, the amount of fluctuation in the movement of the user's wrist can be easily calculated using physical quantities including at least one of angular velocity and acceleration detected by a sensor worn on the user's wrist.

[0032] (9) In the information processing method described in (8) above, the sensor may be worn on the wrist of the user's non-dominant hand.

[0033] With this configuration, since there is a correlation between the wrist movement of the user's non-dominant hand while working and the user's divergent thinking state while working, the divergent thinking state can be determined with higher accuracy using physical quantities detected by a sensor attached to the wrist of the user's non-dominant hand while working.

[0034] (10) In the information processing method described in any one of (1) to (9) above, the method may further include determining whether the user is engaged in the task, and the determination of the divergent thinking state may include determining the divergent thinking state of the user based on the amount of fluctuation during a task period in which the user is engaged in the task.

[0035] According to this configuration, it is determined whether the user is engaged in a task, and therefore it is possible to determine the user's divergent thinking state during the task period when the user is engaged in a task.

[0036] (11) In the information processing method described in (10) above, the determination of whether the user is working on the task may include determining whether the user is working on the task based on at least one of gaze information indicating the user's gaze direction, face position information indicating the position of the user's face on the image, wrist position information indicating the position of the user's wrist on the image, seating pressure information indicating the user's seating pressure, and input information from the user acquired by a work medium used by the user in the task.

[0037] With this configuration, it is possible to easily determine whether a user is working based on at least one of gaze information indicating the direction of the user's gaze, face position information indicating the position of the user's face on the image, wrist position information indicating the position of the user's wrist on the image, seating pressure information indicating the user's seating pressure, and input information from the user acquired by the work medium used by the user in the work.

[0038] Furthermore, the present disclosure can be realized not only as an information processing method that executes the characteristic processes described above, but also as an information processing device having a characteristic configuration corresponding to the characteristic processes executed by the information processing method. Furthermore, the present disclosure can also be realized as a computer program that causes a computer to execute the characteristic processes included in such an information processing method. Therefore, the same effects as those of the above information processing method can also be achieved in the following other aspects.

[0039] (12) Another aspect of the present disclosure provides an information processing device including a processor, wherein the processor acquires a fluctuation amount of a user's wrist movement while working, and determines the user's divergent thinking state based on the fluctuation amount.

[0040] (13) An information processing program according to another aspect of the present disclosure causes a computer to acquire a fluctuation amount of a user's wrist movement while working, and determine the user's divergent thinking state based on the fluctuation amount.

[0041] (14) A computer-readable recording medium according to another aspect of the present disclosure records an information processing program, and the information processing program causes a computer to acquire a fluctuation amount of a user's wrist movement while working, and to determine the user's divergent thinking state based on the fluctuation amount.

[0042] (Experiment) Here, the process leading to one aspect of the present disclosure will be described.

[0043] The inventors conducted an experiment to verify the possibility of estimating divergent thinking. In the experiment, subjects were asked to perform tasks that could induce convergent or divergent thinking.

[0044] Because it was thought that switching between each thinking state in a short period of time would be difficult, subjects were asked to perform a task that would induce them into one of the thinking states for a certain period of time. In addition, breaks were set up before and after the task, and the task was designed to allow for measurement of changes from baseline. Each subject performed one divergent thinking task and one convergent thinking task. A short five-minute break was given immediately after each task, and subjects were given a questionnaire after each task.

[0045] The experiment was conducted on 18 healthy adult male and female subjects. After receiving an explanation of the experiment in the laboratory, the subjects performed the tasks in a room equipped with a camera.

[0046] The experiment consisted of five minutes of rest, 18 minutes of task performance (convergent thinking task or divergent thinking task), 3 minutes of questionnaire response, 5 minutes of rest, 18 minutes of task performance (convergent thinking task or divergent thinking task), 3 minutes of questionnaire response, and 5 minutes of rest. The subjects were informed that the convergent thinking task was a meeting-taking task and the divergent thinking task was a brainstorming task, and were not informed that the tasks they were performing were convergent thinking or divergent thinking tasks.

[0047] The convergent thinking task to measure non-divergent thinking involved taking minutes of a public lecture at a university.

[0048] As a divergent thinking task to measure divergent thinking, participants were asked to generate many ideas for a given task.

[0049] Immediately after completing the task, participants were asked to answer a questionnaire on a 7-point scale, asking, "When performing the task, did you try to think of something that no one else would think of?"

[0050] FIG. 1 shows the results of a questionnaire administered after a convergent thinking task or a divergent thinking task in an experiment.

[0051] In Figure 1, the graph on the left shows the average survey results after the convergent thinking task was performed, and the graph on the right shows the average survey results after the divergent thinking task was performed. The experimental results showed a significant difference in subjective evaluation in the survey between the divergent thinking task and the convergent thinking task. It can be said that the divergent thinking task and the convergent thinking task were able to induce divergent thinking states and non-divergent thinking states (convergent thinking states).

[0052] During the experiment, the subject's upper body was photographed using a camera placed approximately one meter away from the subject. The video images of the subject's upper body were analyzed using a skeletal detection algorithm. Specifically, the skeletal detection algorithm estimated the positions of the subject's upper body joints from the image data constituting the video images and detected the joint movements in real time. The detection results were stored as quantified joint feature data. The facial images and upper body images were discarded immediately. In other words, the facial images and upper body images were not stored as detection results.

[0053] In the analysis, the feature data for the first minute immediately after the start of the task was removed, and the feature data for the remaining 17 minutes was used.

[0054] FIG. 2 shows the average amount of variability in left and right wrist movements while a convergent thinking task or a divergent thinking task is being performed.

[0055] In the experiment, in the convergent thinking task or the divergent thinking task, the average amount of variation in the x-coordinate on the image of the left wrist, the average amount of variation in the y-coordinate on the image of the left wrist, the average amount of variation in the distance from the origin on the image to the left wrist, the average amount of variation in the x-coordinate on the image of the right wrist, the average amount of variation in the y-coordinate on the image of the right wrist, and the average amount of variation in the distance from the origin on the image to the right wrist were calculated.

[0056] FIG. 2 shows the average amount of fluctuation of the left and right wrists of 18 subjects during the convergent thinking task or the divergent thinking task.

[0057] There were significant differences between the two tasks for four of the six indices related to wrist features, and particularly for the left wrist, there were significant differences between the two tasks for three indices: x-coordinate, y-coordinate, and distance. For the indices with significant differences, the amount of variation in the divergent thinking task was greater than the amount of variation in the convergent thinking task. The amount of variation in the subject's wrist movement when the subject was in a divergent thinking state was greater than the amount of variation in the subject's wrist movement when the subject was in a convergent thinking state. Furthermore, the difference in variation between the two tasks was particularly pronounced for the non-dominant wrist (left wrist) than for the dominant wrist (right wrist).

[0058] The experiment showed that when a subject is in a divergent thinking state, there is a large amount of variation in the subject's wrist movement. Therefore, it is believed that it may be possible to estimate whether a user is in a divergent thinking state from the amount of variation in the user's wrist movement.

[0059] FIG. 3 is a diagram showing the correlation coefficient between the amount of wrist movement in the divergent thinking task and the questionnaire results.

[0060] Figure 3 shows the strength of the correlation between the amount of wrist movement during the divergent thinking task and the questionnaire results as a correlation coefficient. A correlation coefficient of 0 to 0.20 indicates almost no correlation, a correlation coefficient of 0.20 to 0.40 indicates a somewhat weak correlation, a correlation coefficient of 0.40 to 0.70 indicates a relatively strong correlation, and a correlation coefficient of 0.70 to 1 indicates a very strong correlation.

[0061] When the correlation coefficient between wrist movement and the evaluation scores obtained from the questionnaire was examined, it was found that in the divergent thinking task, the correlation between the evaluation score from the questionnaire asking whether the subject was in a divergent thinking state and the amount of movement of the left wrist was higher than the correlation between the evaluation score from the questionnaire asking whether the subject was in a divergent thinking state and the amount of movement of the right wrist.

[0062] The amount of movement of the left wrist was found to be more strongly correlated with divergent thinking than the amount of movement of the right wrist. In this experiment, the subjects were right-handed. In other words, the amount of movement of the wrist of the non-dominant hand was found to be more strongly correlated with divergent thinking than the amount of movement of the wrist of the dominant hand.

[0063] When a subject inputs information into a personal computer, the subject switches between keyboard and mouse operation with their dominant hand, which may result in large fluctuations in the wrist of the dominant hand due to movements resulting from tasks unrelated to thinking.

[0064] Furthermore, when subjects use writing implements, they change pencils, erasers, colored pens, etc. Therefore, in this case too, there is a risk that the amount of fluctuation in the wrist of the dominant hand will increase due to movements resulting from tasks that are not related to thought.

[0065] For these reasons, the inventors decided to focus on the movement of the wrist of the non-dominant hand.

[0066] (Embodiments) Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concept are described as optional components. Furthermore, in all embodiments, the respective contents can be combined.

[0067] 4 is a diagram showing an example of a work space to which the information processing method according to the embodiment of the present disclosure is applied. The work space is a space in which a user performs work using a work medium 4. Examples of the work space include a room of a user who works from home, a room in an office, or a space separated by a partition in an office.

[0068] The workspace is equipped with at least a camera 1 for photographing the user within the workspace, and a terminal device 3 for displaying the assessment results of the thinking state and accepting input from the user in a questionnaire regarding the assessment results of the thinking state.

[0069] The camera 1 is composed of a camera device that captures images of a user working in the workspace at a predetermined frame rate, for example. The camera 1 is installed above the wall of the workspace so that its optical axis faces the work medium 4 and the user using the work medium 4. The camera 1 is used to observe the user and the user's work medium 4. Note that if it is difficult to observe all information with a single camera, multiple cameras may be installed in the workspace.

[0070] Furthermore, camera 1 may be installed at any position within the work space as long as it is a position where the work medium 4 and the user can take pictures. For example, if the work medium 4 is a personal computer (PC), camera 1 may be a camera attached to the personal computer. Camera 1 may also be installed on or near a table while being supported by a support such as a tripod or a fixing arm. Camera 1 may also be a camera attached to terminal device 3.

[0071] The work medium 4 is an object on which the user performs work. Examples of the work medium 4 include a personal computer, a notebook, a book, writing implements, a tablet computer, and a smartphone. The personal computer may be a desktop computer or a notebook computer. The work performed by the user may be, for example, intellectual work, office work, desk work, work using a PC, or a meeting. The work may be desk work using a PC, or may be desk work using writing implements instead of a PC. The work includes tasks that require divergent thinking and tasks that require convergent thinking.

[0072] A divergent thinking state refers to a state in which a user is thinking divergently. Here, the divergent thinking state may include a state in which the user is demonstrating creativity. Furthermore, a convergent thinking state refers to a state in which a user is thinking convergently. Here, the convergent thinking state may include a state in which the user is concentrating.

[0073] Camera 1 captures the movement of the user's wrist. Based on the image captured by camera 1, any of information about the user's dominant hand, information about the user's posture, information about an object held by the user, and information about the arrangement of objects on a desk may be acquired.

[0074] Furthermore, when a user works using a PC, either information regarding whether or not input is being made using the keyboard or information regarding whether or not the user is gazing at the PC screen may be acquired.

[0075] The terminal device 3 is, for example, a personal computer, a tablet computer, or a smartphone. The terminal device 3 includes a display unit that displays images and an input unit that accepts input from a user. The terminal device 3 may also include a speaker that outputs sound.

[0076] 5 is a block diagram showing an example of the configuration of a mental state assessment system according to an embodiment of the present disclosure. The mental state assessment system includes a camera 1, a server 2, and a terminal device 3.

[0077] The camera 1 includes an imaging unit 11, a signal processing unit 12, a storage unit 13, and a communication unit 14.

[0078] The image capturing unit 11 captures an image of the upper body of the user while working. The image capturing unit 11 captures an image of the upper body of the user while working every 0.1 seconds.

[0079] The signal processing unit 12 performs signal processing on the image obtained by the imaging unit 11. The signal processing unit 12 detects skeletal information of the user from the image obtained by the imaging unit 11. The signal processing unit 12 detects the positions of multiple joints of the user. The skeletal information includes multiple feature points on the user's upper body and multiple skeletal lines connecting the feature points. For example, the multiple feature points include a feature point indicating the face, a feature point indicating the center of both shoulders, a feature point indicating the right shoulder joint, a feature point indicating the right elbow joint, a feature point indicating the right wrist joint, a feature point indicating the left shoulder joint, a feature point indicating the left elbow joint, and a feature point indicating the left wrist joint. The signal processing unit 12 detects position information of the feature points indicating the right wrist joint and the feature points indicating the left wrist joint on the image. The position information includes x coordinates and y coordinates on the image. The signal processing unit 12 detects position information of the user's right wrist and left wrist every 0.1 seconds. The signal processing unit 12 associates the position information of the user's right wrist and left wrist with time information and stores it in the storage unit 13.

[0080] Note that technology for identifying the position of joints such as the wrist is disclosed in, for example, Japanese Patent Application Laid-Open No. 2022-187215.

[0081] The storage unit 13 temporarily stores the images obtained by the imaging unit 11. The storage unit 13 also stores position information of the right wrist and left wrist detected by the signal processing unit 12 in association with time information.

[0082] FIG. 6 is a diagram showing an example of position information and time information of the right and left wrists stored in the camera 1 in this embodiment.

[0083] 6 , the storage unit 13 stores a user ID for identifying a user, time information (year, month, day, hour, minute, second) when the image was acquired, the y coordinate of the face, the x coordinate of the left wrist, the y coordinate of the left wrist, the x coordinate of the right wrist, and the y coordinate of the right wrist, in association with each other. The face position information and the position information of the right and left wrists are stored every 0.1 seconds. The y coordinate of the face is stored for use in task determination processing that determines whether the user is working on a task. If the y coordinate of the face is not used for task processing determination, the y coordinate of the face does not need to be stored.

[0084] The communication unit 14 transmits an image of the user to the server 2 every 0.1 seconds. That is, the communication unit 14 transmits a moving image of the user to the server 2. The communication unit 14 also transmits position information of the right wrist and left wrist detected by the signal processing unit 12 to the server 2 every 0.1 seconds.

[0085] The server 2 is an example of an information processing device. The server 2 is connected to the camera 1 and the terminal device 3 via a network so that they can communicate with each other. The network is, for example, the Internet.

[0086] The server 2 includes a communication unit 21 , a processor 22 , and a storage unit 23 .

[0087] The processor 22 is, for example, a central processing unit (CPU). The processor 22 implements a dominant hand determination unit 221, an activity determination unit 222, a reference value determination unit 223, a variation amount calculation unit 224, a thinking state determination unit 225, and a threshold value change unit 226.

[0088] The storage unit 23 is a storage device capable of storing various types of information, such as a random access memory (RAM), a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The storage unit 23 stores various types of information.

[0089] The communication unit 21 receives the moving images transmitted by the camera 1. That is, the communication unit 21 receives images captured by the camera 1 every 0.1 seconds. The communication unit 21 also receives position information of the user's right wrist and left wrist transmitted every 0.1 seconds by the camera 1. The communication unit 21 stores the position information of the user's right wrist and left wrist acquired every 0.1 seconds in the storage unit 23 in association with time information.

[0090] The storage unit 23 temporarily stores the images received by the communication unit 21. The storage unit 23 also stores position information of the user's right and left wrists in association with time information.

[0091] The handedness determination unit 221 acquires position information of the work medium 4 used by the user during work based on the moving image. Based on the acquired position information, the handedness determination unit 221 determines whether to use the amount of movement of the user's right wrist or left wrist to determine the divergent thinking state. Based on the moving image, the handedness determination unit 221 determines the handedness of the user.

[0092] The handedness determination unit 221 determines the user's handedness based on the moving image received by the communication unit 21. The handedness determination unit 221 detects the pen or mouse and the user's hand in the image and determines the hand operating the pen or mouse as the handedness. For example, if the user is operating the pen or mouse with their right hand, the handedness determination unit 221 determines that the user's handedness is the right hand. The handedness determination unit 221 may also detect a pen, cup, or mouse in the image and determine the handedness based on the position of the pen, cup, or mouse on the desk. For example, if the pen, cup, or mouse is on the user's right hand side, the handedness determination unit 221 determines that the user's handedness is the right hand. The handedness determination unit 221 stores non-handedness information in the storage unit 23, which indicates the hand other than the user's handedness, i.e., the user's non-handed hand.

[0093] The storage unit 23 may store user information 231 related to the user. The user information includes a user ID, the user's name, or a facial image of the user. The user information is, for example, input by the user using the terminal device 3 and transmitted to the server 2. The storage unit 23 of the server 2 stores the user information 231 input by the user. The terminal device 3 may accept an input from the user regarding which hand is the user's non-dominant hand. The user information 231 may include non-dominant hand information indicating which hand is the user's non-dominant hand.

[0094] The work determination unit 222 determines whether the user is working on a task. The work determination unit 222 may determine whether the user's posture is one in which the user is working on a task, based on an image acquired from the camera 1. The work determination unit 222 may also determine whether the user is working on a task based on at least one of gaze information indicating the user's gaze direction, face position information indicating the position of the user's face on the image, wrist position information indicating the position of the user's wrist on the image, seating pressure information indicating the user's seating pressure, and input information from the user acquired by the work medium 4 used by the user during the task.

[0095] The work determination unit 222 estimates the user's gaze direction using an image including the user. Since techniques for estimating the user's gaze direction from an image are publicly known, the work determination unit 222 may estimate the user's gaze direction using this publicly known method. For example, the work determination unit 222 may detect the user's eyeball position, pupil position, facial direction, etc. from an image including the user, and estimate the user's gaze direction from the detected eyeball position, pupil position, facial direction, etc. The work determination unit 222 may determine whether the user is working based on whether an extension line of a vector indicating the user's gaze direction intersects with the work medium 4.

[0096] The work determination unit 222 may acquire face position information indicating the coordinates on the image of feature points indicating the face included in the skeletal information from the camera 1. When an image of the upper body of a user working is captured, the user's face is located within a predetermined area in the image. Therefore, the work determination unit 222 may acquire the y coordinates of feature points indicating the user's face, and determine that the user is working if the acquired y coordinate of the face is within a predetermined range.

[0097] The work determination unit 222 may also acquire wrist position information from the camera 1, which indicates the coordinates on the image of a feature point indicating the wrist included in the skeletal information. When an image of a user's upper body is captured while the user is working, the user's wrist is located within a predetermined area in the image. Therefore, the work determination unit 222 may acquire the x-coordinate or y-coordinate of a feature point indicating the user's right wrist or left wrist, and determine that the user is working if the acquired x-coordinate or y-coordinate of the right wrist or left wrist is within a predetermined range. The work determination unit 222 may also acquire the y-coordinate of a feature point indicating the user's face and the x-coordinate or y-coordinate of a feature point indicating the user's right wrist or left wrist, and determine that the user is working if the acquired y-coordinate of the face is within a predetermined first range and the acquired x-coordinate or y-coordinate of the right wrist or left wrist is within a predetermined second range.

[0098] A seat-shaped seat pressure sensor may be disposed on the chair on which the user sits. The seat pressure sensor is used to determine whether the user is seated. The work determination unit 222 may acquire seat pressure information indicating the user's seat pressure from the seat pressure sensor. The work determination unit 222 may determine that the user is working when pressure is detected by the seat pressure sensor, or may determine that the user is not working when pressure is no longer detected by the seat pressure sensor. The work determination unit 222 may determine that the user is working when the pressure detected by the seat pressure sensor is equal to or greater than a predetermined threshold, or may determine that the user is not working when the pressure detected by the seat pressure sensor is less than the predetermined threshold.

[0099] The data from the seat pressure sensor may be used to determine whether to start or stop the camera 1. That is, when pressure is detected by the seat pressure sensor, shooting by the camera 1 may be started. Also, when pressure is no longer detected by the seat pressure sensor, shooting by the camera 1 may be stopped.

[0100] The work determination unit 222 may acquire input information input by the user while working from the work medium 4. If the work medium 4 is a PC, the user performs the work using the PC. Therefore, the work determination unit 222 may acquire input information input by the user from the work medium 4, and may determine that the user is working when the input information is acquired.

[0101] Furthermore, if the work medium 4 is a PC, the work determination unit 222 may acquire information indicating whether the PC is powered on or off. If the PC is powered on, the work determination unit 222 may determine that the user is working, and if the PC is powered off, the work determination unit 222 may determine that the user is not working.

[0102] Furthermore, the activity determination unit 222 may determine, from the acquired image, at least one of whether the user is holding a writing implement in their hand, whether the user has their hands on a keyboard or a mouse, whether the user is on a call, and whether the user is holding a cup, chopsticks, a spoon, or the like. The activity determination unit 222 may determine that the user is engaged in an activity if the user is holding a writing implement in their hand, if the user has their hands on a keyboard or a mouse, if the user is not on a call, or if the user is not holding a cup, chopsticks, a spoon, or the like.

[0103] The work determination unit 222 associates work determination result information indicating whether the user is working on a task with time information and stores the information in the storage unit 23. The time information indicates the time at which the image was obtained. The storage unit 23 stores the work determination result information indicating whether the user is working on a task with the time information in association with each other.

[0104] The reference value determination unit 223 determines a reference value used when determining the thinking state. The reference value determination unit 223 acquires non-dominant hand information indicating which hand is the user's non-dominant hand from the storage unit 23. The reference value determination unit 223 also acquires determination result information indicating whether the user is working on a task from the storage unit 23, and identifies a task period during which the user is working on a task. The task period may be equal to or longer than a second period described below.

[0105] The reference value determination unit 223 acquires position information of the wrist of the non-dominant hand during the work period, and calculates the amount of variation in the movement of the wrist of the non-dominant hand during each first period. The first period is, for example, one minute. The amount of variation is the standard deviation of the distance from the origin on the image to the wrist of the non-dominant hand during the first period. The origin is, for example, the upper left of the image. The distance from the origin to the wrist of the non-dominant hand is calculated based on the x-coordinate and y-coordinate of the wrist of the non-dominant hand (x ２ +y ２ ) １／２ The amount of variation may be the standard deviation of the x-coordinate of the wrist of the non-dominant hand during the first period. The amount of variation may also be the standard deviation of the y-coordinate of the wrist of the non-dominant hand during the first period. The reference value determination unit 223 stores the amount of variation in the movement of the wrist of the non-dominant hand during the first period, calculated for each first period, in the storage unit 23.

[0106] The reference value determination unit 223 determines the smallest amount of fluctuation among the multiple amounts of fluctuation calculated during the second period as the reference value. The second period is, for example, 15 minutes.

[0107] The mental state may be determined without determining a reference value. The process of determining the reference value may be performed when the mental state is determined, or may be performed using location information acquired on another day.

[0108] The fluctuation amount calculation unit 224 acquires a fluctuation amount of the wrist movement of the user while working. The fluctuation amount calculation unit 224 is an example of an acquisition unit. The fluctuation amount calculation unit 224 calculates a fluctuation amount of the wrist movement of the user while working. The fluctuation amount calculation unit 224 calculates the fluctuation amount based on a change in the position of the wrist in a video image of the user's wrist. The fluctuation amount calculation unit 224 acquires non-dominant hand information indicating which hand is the user's non-dominant hand from the storage unit 23, and identifies which hand is the user's non-dominant hand. In addition, the fluctuation amount calculation unit 224 acquires determination result information indicating whether the user is working on a task from the storage unit 23, and identifies a work period during which the user is working on a task.

[0109] The variation calculation unit 224 acquires position information of the wrist of the non-dominant hand during the work period, and calculates the variation of the movement of the wrist of the non-dominant hand during a first period. The first period is, for example, one minute. The variation is the standard deviation of the distance from the origin on the image of the wrist of the non-dominant hand during the first period. The distance from the origin is calculated based on the x-coordinate and y-coordinate of the wrist of the non-dominant hand, as follows: (x ２ +y ２ ) １／２ The amount of variation may be the standard deviation of the x-coordinate of the wrist of the non-dominant hand during the first period. The amount of variation may also be the standard deviation of the y-coordinate of the wrist of the non-dominant hand during the first period.

[0110] FIG. 7 is a diagram showing an example of the user ID, time information, task determination flag, and amount of variation in the distance from the origin to the wrist of the non-dominant hand, all of which are stored in the server 2 in this embodiment.

[0111] 7 , the storage unit 23 may store, in association with one another, a user ID for identifying a user, time information (year, month, day, hour, minute) at which the amount of variation was calculated, an activity determination flag indicating whether the user is engaged in an activity, and the amount of variation in the distance from the origin on the image to the wrist of the non-dominant hand during the first period. The activity determination flag is set to a value of “1” if it is determined that the user is continuously engaged in an activity during the first period (e.g., one minute), and is set to a value of “0” if it is determined that the user is not engaged in an activity even once during the first period (e.g., one minute).

[0112] The thinking state determination unit 225 determines the user's divergent thinking state based on the amount of variation calculated by the variation amount calculation unit 224. The thinking state determination unit 225 is an example of a determination unit. The thinking state determination unit 225 determines the divergent thinking state based on the amount of variation in the wrist movement of the non-dominant hand. The thinking state determination unit 225 also determines the user's divergent thinking state based on the amount of variation during a task period in which the user is working on a task.

[0113] Here, the thinking state determination section 225 determines whether the user's thinking state is a divergent thinking state or not, based on the amount of variation calculated by the variation amount calculation section 224. Note that the thinking state determination section 225 may determine whether the user's thinking state is a divergent thinking state or a convergent thinking state, based on the amount of variation calculated by the variation amount calculation section 224.

[0114] 8 is a schematic diagram for explaining a method for determining the first and second thresholds in this embodiment, in which the horizontal axis represents time and the vertical axis represents the amount of fluctuation.

[0115] The thinking state assessment unit 225 determines that the user is in a divergent thinking state when the amount of variation is equal to or greater than the first threshold. Furthermore, the thinking state assessment unit 225 determines that the user is in a convergent thinking state when the amount of variation is smaller than the first threshold. The thinking state assessment unit 225 determines the first threshold based on the reference value determined by the reference value determination unit 223. In FIG. 8 , the reference value is, for example, 1. The thinking state assessment unit 225 determines the first threshold as a value obtained by multiplying the reference value by a first coefficient. The first coefficient is, for example, 3. For example, the thinking state assessment unit 225 may determine the first threshold as a value obtained by tripling the reference value. In FIG. 8 , the first threshold is, for example, 3. Note that if the reference value is 0 or if a reference value has not been determined, the thinking state assessment unit 225 may determine a predetermined value as the first threshold. The predetermined value is, for example, 4.

[0116] Furthermore, the thinking state determination unit 225 determines the degree of the user's divergent thinking state according to the magnitude of the amount of variation. For example, when the amount of variation is equal to or greater than a second threshold, the thinking state determination unit 225 determines that the user is in a highly divergent thinking state, where the degree of divergent thinking is high, and when the amount of variation is smaller than the second threshold, the thinking state determination unit 225 determines that the user is in a low divergent thinking state, where the degree of divergent thinking is low.

[0117] Here, the second threshold is a value higher than the first threshold. The thinking state assessment unit 225 determines the second threshold based on the reference value determined by the reference value determination unit 223. The thinking state assessment unit 225 determines the second threshold as a value obtained by multiplying the reference value by a second coefficient. The second coefficient is, for example, 14. For example, the thinking state assessment unit 225 may determine the second threshold as a value obtained by multiplying the reference value by 14. In FIG. 8 , the second threshold is, for example, 14. Note that if the reference value is 0 or if a reference value has not been determined, the thinking state assessment unit 225 may determine a predetermined value as the second threshold. The predetermined value is, for example, 14.

[0118] The thought state determination section 225 may quantify the degree of the user's distracted thought state according to the magnitude of the amount of variation.

[0119] The communication unit 21 is an example of an output unit. The communication unit 21 outputs the determination result of the divergent thinking state made by the thinking state determination unit 225.

[0120] The communication unit 21 transmits the divergent thinking state assessment result made by the thinking state assessment unit 225 to the terminal device 3. The terminal device 3 presents the divergent thinking state assessment result to the user. After presenting the divergent thinking state assessment result to the user, the terminal device 3 also presents the user with a questionnaire about the divergent thinking state assessment result and accepts the user's input to the questionnaire. For example, the questionnaire presents three options: "I felt the divergent thinking state assessment result was appropriate," "I felt the divergent thinking state assessment criteria were strict," and "I felt the divergent thinking state assessment criteria were lenient." The user selects one of the three options. The terminal device 3 transmits questionnaire result information indicating the user's questionnaire results to the server 2. The questionnaire result information includes evaluation information indicating the user's evaluation of the divergent thinking state assessment result. The communication unit 21 receives the questionnaire result information transmitted by the terminal device 3.

[0121] The threshold changing unit 226 acquires evaluation information indicating the user's evaluation of the divergent thinking state assessment result. The evaluation information includes an evaluation that the divergent thinking state assessment result is appropriate, an evaluation that the divergent thinking state assessment criteria are strict, or an evaluation that the divergent thinking state assessment criteria are lenient. The threshold changing unit 226 changes the first threshold based on the evaluation information.

[0122] If the user's evaluation indicates that the divergent thinking state assessment result is appropriate, the threshold change unit 226 maintains the current first threshold without changing it. On the other hand, if the user's evaluation indicates that the divergent thinking state assessment criteria are strict, the threshold change unit 226 decreases the current first threshold. In this case, the threshold change unit 226 may subtract 10% of the current first threshold from the current first threshold. Alternatively, the threshold change unit 226 may subtract a predetermined value from the current first threshold. Alternatively, if the user's evaluation indicates that the divergent thinking state assessment criteria are lenient, the threshold change unit 226 increases the current first threshold. In this case, the threshold change unit 226 may add 10% of the current first threshold to the current first threshold. Alternatively, the threshold change unit 226 may add a predetermined value to the current first threshold.

[0123] The storage unit 23 may store a first threshold value in addition to the reference value. The storage unit 23 may also store an increase amount of the first threshold value or a decrease amount of the first threshold value in addition to the reference value.

[0124] The server 2 may include the signal processing unit 12 and the storage unit 13 of the camera 1. Also, a PC connected to the camera 1 may include the signal processing unit 12, the storage unit 13, and the communication unit 14 of the camera 1.

[0125] In addition, in this embodiment, the mental state assessment system includes a terminal device 3 that includes a display unit and an input unit, but the present disclosure is not particularly limited to this. The mental state assessment system may also include a display unit and an input unit separately. The display unit may be a projector.

[0126] FIG. 9 is a sequence diagram for explaining the flow of processing by the camera 1, the server 2, and the terminal device 3 in this embodiment.

[0127] First, the imaging unit 11 of the camera 1 captures an image of the user. At this time, the camera 1 captures an image of the upper body of the user, including at least the right and left wrists of the user. The communication unit 14 of the camera 1 transmits moving images of the user to the server 2. The imaging unit 11 captures an image of the user every 0.1 seconds, and the communication unit 14 sequentially transmits the images obtained every 0.1 seconds to the server 2.

[0128] Next, the communication unit 21 of the server 2 receives the video images transmitted by the camera 1. The communication unit 21 sequentially receives the images captured every 0.1 seconds. The handedness determination unit 221 of the server 2 performs a handedness determination process to determine the handedness of the user based on the received video images. Note that the handedness determination unit 221 may determine the handedness of the user based on one image of the received video images.

[0129] Next, the work determination unit 222 of the server 2 performs work determination processing to determine whether or not the user is working on a task based on the received video images. The work determination unit 222 determines whether or not the user is working on a task for each of the multiple images captured every 0.1 seconds. As a result, whether or not the user is working on a task is determined every 0.1 seconds. The work determination unit 222 stores the determination result as to whether or not the user is working on a task in the storage unit 23 together with time information.

[0130] Next, the signal processing unit 12 of the camera 1 detects position information of the user's right wrist and left wrist in the image captured every 0.1 seconds. The communication unit 14 of the camera 1 sequentially transmits the position information of the user's right wrist and left wrist detected every 0.1 seconds to the server 2.

[0131] Next, the communication unit 21 of the server 2 sequentially receives the position information of the user's right wrist and left wrist transmitted by the camera 1. Next, the reference value determination unit 223 of the server 2 performs a reference value determination process to determine a reference value to be used when judging the thinking state, based on the received position information.

[0132] Next, the variation calculation unit 224 of the server 2 performs a variation calculation process to calculate the variation in the movement of the wrist of the user's non-dominant hand during the work period, based on the received position information.

[0133] Next, the thinking state determination unit 225 of the server 2 performs a thinking state determination process to determine the user's distracted thinking state based on the amount of variation calculated by the variation amount calculation unit 224. The thinking state determination unit 225 determines the user's distracted thinking state during the work period.

[0134] Next, the communication unit 21 of the server 2 transmits the result of the determination of the divergent thinking state by the thinking state determination unit 225 to the terminal device 3 .

[0135] Next, the terminal device 3 receives the assessment result of the divergent thinking state transmitted by the server 2. Next, the terminal device 3 presents the assessment result of the divergent thinking state to the user. Next, the terminal device 3 presents a questionnaire regarding the assessment result of the divergent thinking state to the user and accepts input from the user to the questionnaire. Next, the terminal device 3 transmits the survey result to the server 2.

[0136] Next, the communication unit 21 of the server 2 receives the survey results sent by the terminal device 3. Next, the threshold change unit 226 performs a threshold change process to change the first threshold based on the survey results.

[0137] Next, the dominant hand determination process performed by the server 2 in this embodiment will be described.

[0138] FIG. 10 is a flowchart for explaining the dominant hand determination process by the server 2 in this embodiment.

[0139] First, in step S1 , the communication unit 21 acquires the moving image transmitted by the camera 1 .

[0140] Next, in step S2, the dominant hand determination unit 221 determines which hand is the dominant hand of the user based on the moving image received by the communication unit 21.

[0141] Next, in step S3, handedness determination unit 221 stores non-handedness information indicating which hand is the user's non-dominant hand in storage unit 23. In the present embodiment, handedness determination unit 221 stores non-dominant hand information in storage unit 23, but the present disclosure is not particularly limited to this, and handedness information indicating which hand is the user's dominant hand may also be stored in storage unit 23.

[0142] Next, the task determination process performed by the server 2 in this embodiment will be described.

[0143] FIG. 11 is a flowchart for explaining the task determination process by the server 2 in this embodiment.

[0144] First, in step S11, the work determination unit 222 acquires a moving image. The moving image is made up of a plurality of images captured at a predetermined frame rate over a predetermined period of time.

[0145] Next, in step S12, the work determination unit 222 selects one image from the moving image.

[0146] Next, in step S13, the work determination unit 222 determines whether the user's posture is a posture of working on a task based on the selected image. If it is determined that the user's posture is a posture of working on a task (YES in step S13), in step S14, the work determination unit 222 determines that the user is working on a task.

[0147] On the other hand, if it is determined that the user's posture is not a posture of working (NO in step S13), then in step S15, the work determination unit 222 determines that the user is not working.

[0148] Next, in step S16, the work determination unit 222 stores work determination result information indicating whether the user is working on the work in the storage unit 23 in association with time information indicating the time the image was obtained.

[0149] Next, in step S17, the work determination unit 222 determines whether or not the determination of all images constituting the acquired moving image has been completed. If it is determined that the determination of all images has not been completed (NO in step S17), the process returns to step S12, where the work determination unit 222 selects one image from the moving image that has not yet been determined. On the other hand, if it is determined that the determination of all images has been completed (YES in step S17), the process ends.

[0150] Next, the reference value determination process performed by the server 2 in this embodiment will be described.

[0151] FIG. 12 is a flowchart for explaining the reference value determination process by the server 2 in this embodiment.

[0152] First, in step S21, the reference value determination unit 223 acquires position information of the wrist of the non-dominant hand during a second period within a work period in which the user is working, from the storage unit 23. The second period is, for example, 15 minutes. The position information is detected every 0.1 seconds, for example.

[0153] Next, in step S22, the reference value determination unit 223 calculates the amount of variation in the movement of the wrist of the non-dominant hand for each first period. The first period is, for example, one minute. Here, the amount of variation is the standard deviation of the distance from the reference point (origin) on the image to the wrist of the non-dominant hand during the first period. The reference value determination unit 223 calculates the standard deviation of the distance from the reference point (origin) on the image to the wrist of the non-dominant hand during the first period.

[0154] Next, in step S23, the reference value determination unit 223 temporarily stores the amount of fluctuation for each first period in the storage unit 23.

[0155] Next, in step S24, the reference value determination unit 223 determines the smallest amount of fluctuation within the second period as the reference value.

[0156] Next, in step S25, the reference value determination unit 223 stores the determined reference value in the storage unit 23.

[0157] FIG. 13 is a schematic diagram for explaining the reference value determination process in this embodiment.

[0158] 13 shows time-series data of the user's minute-by-minute fluctuation during a task period. The reference value determination unit 223 determines the smallest fluctuation among the fluctuations during periods A6 to A20 in the second period from 5 to 20 minutes as the reference value. In FIG. 13, the fluctuation during period A12 is the smallest, so the fluctuation during period A12, 0.014991, is determined as the reference value.

[0159] Next, the imaging process by the camera 1 in this embodiment will be described.

[0160] FIG. 14 is a flowchart for explaining the imaging process by the camera 1 in this embodiment.

[0161] First, in step S31, the imaging unit 11 captures an image of the user. The imaging unit 11 captures an image of the user, for example, every 0.1 seconds. Note that the imaging unit 11 may start capturing images when the user presses a start button provided on the camera 1.

[0162] Next, in step S32, the signal processing unit 12 detects position information of the wrist from the image obtained by the imaging unit 11. At this time, the signal processing unit 12 detects the x-coordinates and y-coordinates of the user's right wrist and left wrist as position information.

[0163] Next, in step S33, the signal processing unit 12 stores the detected wrist position information in the storage unit 13 in association with time information.

[0164] Next, in step S34, the communication unit 14 transmits the wrist position information together with time information to the server 2. Note that the communication unit 14 may transmit the wrist position information to the server 2 together with the time information and a user ID for identifying the user. The communication unit 21 of the server 2 receives the user's wrist position information. Then, the communication unit 21 stores the received user's wrist position information in the storage unit 23 in association with the time information. At this time, the communication unit 21 may (i) store the position information of the user's left wrist and right wrist in the storage unit 23, or (ii) store the wrist position information of the user's non-dominant hand in the storage unit 23, and may not store the wrist position information of the user's dominant hand in the storage unit 23. The storage unit 23 stores the wrist position information of the user during a predetermined period when the moving image was acquired.

[0165] Next, in step S35, the imaging unit 11 determines whether or not to end imaging. Note that if the user presses an end button provided on the camera 1, the imaging unit 11 may end imaging. Here, if it is determined that imaging should not be ended (NO in step S35), the process returns to step S31. On the other hand, if it is determined that imaging should be ended (YES in step S35), the process ends.

[0166] Next, the thought state determination process performed by the server 2 in this embodiment will be described.

[0167] 15 is a flowchart for explaining the mental state determination process by the server 2 in this embodiment. Note that the mental state determination process in this embodiment is carried out after video images are acquired for a predetermined period of time.

[0168] First, in step S41, the mental state assessment unit 225 identifies the task period during which the user is working on the task, based on the task assessment result information stored in the storage unit 23. When the user presses a start button provided on the terminal device 3, the mental state assessment unit 225 may start the mental state assessment process and identify the task period.

[0169] Next, in step S42, the fluctuation amount calculation unit 224 acquires position information of the wrist of the user's non-dominant hand during the work period. Note that the fluctuation amount calculation unit 224 identifies the user's non-dominant hand based on the non-dominant hand information stored in the storage unit 23.

[0170] Next, in step S43, the variation calculation unit 224 calculates the variation of the wrist movement of the user's non-dominant hand during a first period. The first period is, for example, one minute. Here, the variation is the standard deviation of the distance from the reference point (origin) on the image to the wrist of the non-dominant hand during the first period. The distance from the reference point (origin) on the image to the wrist of the non-dominant hand is calculated as (x ２ +y ２ ) １／２ The variation calculation unit 224 calculates the standard deviation of the distance from the reference point (origin) to the wrist of the non-dominant hand during the first period.

[0171] Next, in step S44, the thinking state assessment unit 225 determines whether the variation calculated by the variation calculation unit 224 is equal to or greater than a first threshold. Here, the first threshold is a value obtained by multiplying a reference value stored in the storage unit 23 by a first coefficient. The first coefficient is, for example, 3. Note that in this embodiment, the first threshold is determined based on the reference value, but the present disclosure is not particularly limited thereto. If the reference value is 0 or if a reference value has not been determined, the thinking state assessment unit 225 may determine a predetermined value as the first threshold. The predetermined value is, for example, 4.

[0172] If it is determined that the amount of variation is not equal to or greater than the first threshold (NO in step S44), then in step S45 the thinking state determination unit 225 determines that the user's thinking state is a convergent thinking state. Note that if it is determined that the amount of variation is not equal to or greater than the first threshold, the thinking state determination unit 225 may determine that the user's thinking state is not a divergent thinking state. After the processing of step S45 is performed, the process proceeds to step S50.

[0173] On the other hand, if it is determined that the amount of variation is equal to or greater than the first threshold value (YES in step S44), then in step S46 the thought state determination section 225 determines that the user's thought state is a divergent thought state.

[0174] Next, in step S47, the thinking state assessment unit 225 determines whether the variation calculated by the variation calculation unit 224 is equal to or greater than a second threshold. Here, the second threshold is a value greater than the first threshold and is a value obtained by multiplying the reference value stored in the storage unit 23 by a second coefficient. The second coefficient is, for example, 14. Note that in this embodiment, the second threshold is determined based on the reference value, but the present disclosure is not limited to this. If the reference value is 0 or if no reference value has been determined, the thinking state assessment unit 225 may determine a predetermined value as the second threshold. The predetermined value is, for example, 14.

[0175] If it is determined that the amount of variation is equal to or greater than the second threshold (YES in step S47), then in step S48 the thought state determination section 225 determines that the user's thought state is a highly divergent thought state.

[0176] On the other hand, if it is determined that the amount of variation is not equal to or greater than the second threshold (NO in step S47), then in step S49, the thought state determination section 225 determines that the user's thought state is a low-diffusive thought state.

[0177] In this embodiment, the degree of divergent thinking is determined, but the present disclosure is not limited to this, and the degree of divergent thinking does not necessarily have to be determined. In this case, the processes of steps S47 to S49 are not performed.

[0178] Next, in step S50, the thought state determination section 225 stores the determination result of the divergent thought state in the storage section 23 in association with time information.

[0179] Next, in step S51, the thinking state determination unit 225 determines whether or not the divergent thinking state has been determined for all task periods. If it is determined that the divergent thinking state has not been determined for all task periods (NO in step S51), the process returns to step S43, and the variation calculation unit 224 calculates the variation in the movement of the wrist of the non-dominant hand for the next first period.

[0180] On the other hand, if it is determined that the divergent thinking state has been determined for all task periods (YES in step S51), then in step S52 the communication unit 21 transmits the determination result of the divergent thinking state to the terminal device 3. For example, the communication unit 21 transmits time-series data of the determination result of the divergent thinking state to the terminal device 3.

[0181] The terminal device 3 receives the assessment result of the divergent thinking state transmitted by the server 2. The terminal device 3 presents the received assessment result of the divergent thinking state to the user. The display unit of the terminal device 3 displays the received assessment result of the divergent thinking state. For example, the terminal device 3 displays time-series data of the assessment result of the divergent thinking state.

[0182] In this way, there is a correlation between the user's wrist movements while working and the user's state of divergent thinking while working, so the user's state of divergent thinking can be easily determined from the user's wrist movements while working, which can be easily detected.

[0183] In this embodiment, wrist position information for a predetermined period including a work period is acquired, a variation in wrist movement is calculated based on the wrist position information accumulated over the predetermined period, and the user's distracted thinking state for the predetermined period is determined based on the variation, but the present disclosure is not particularly limited to this. In a variation of this embodiment, while acquiring images and wrist position information in real time, it may be determined whether the user is working on a task and the variation in wrist movement is calculated, and the user's distracted thinking state may be determined based on the variation.

[0184] FIG. 16 is a first flowchart for explaining the thinking state determination process by the server 2 in a modified example of this embodiment, and FIG. 17 is a second flowchart for explaining the thinking state determination process by the server 2 in a modified example of this embodiment.

[0185] First, in step S61, the communication unit 21 acquires an image transmitted by the camera 1. At this time, the camera 1 captures an image of the user, for example, every 0.1 seconds, and transmits the obtained image to the server 2. The communication unit 21 receives the image transmitted by the camera 1.

[0186] Next, in step S62, the task determination unit 222 determines whether the user's posture is a posture for working on a task based on the acquired image. If it is determined that the user's posture is not a posture for working on a task (NO in step S62), the process returns to step S61.

[0187] On the other hand, if it is determined that the user's posture is one in which the user is working (YES in step S62), then in step S63, the communication unit 21 acquires wrist position information. At this time, the camera 1 transmits the position information of the user's right and left wrists to the server 2, for example, every 0.1 seconds. The communication unit 21 receives the position information of the right and left wrists transmitted by the camera 1.

[0188] Next, in step S64, the communication unit 21 stores the acquired wrist position information in the storage unit 23 in association with time information.

[0189] Next, in step S65, the fluctuation amount calculation unit 224 determines whether the first period has elapsed. The first period is, for example, one minute. If it is determined that the first period has not elapsed (NO in step S65), the process returns to step S61.

[0190] On the other hand, if it is determined that the first period has elapsed (YES in step S65), in step S66, the fluctuation amount calculation unit 224 calculates the amount of fluctuation in the movement of the wrist of the non-dominant hand during the first period. Note that the processing in steps S66 to S73 is the same as the processing in steps S43 to S50 in Figure 15, and therefore description thereof will be omitted.

[0191] Next, in step S74, the communication unit 21 transmits the determination result of the divergent thinking state to the terminal device 3. For example, the communication unit 21 transmits the determination result of the divergent thinking state for the first period to the terminal device 3.

[0192] The terminal device 3 receives the assessment result of the divergent thinking state transmitted by the server 2. The terminal device 3 presents the received assessment result of the divergent thinking state to the user. The display unit of the terminal device 3 displays the received assessment result of the divergent thinking state. For example, the terminal device 3 displays the assessment result of the divergent thinking state for a first period for each first period.

[0193] Next, in step S75, the thinking state assessment unit 225 determines whether or not to end the process of assessing the divergent thinking state. Note that the thinking state assessment unit 225 may end the process of assessing the divergent thinking state if the user presses an end button provided on the terminal device 3. Here, if it is determined that the process of assessing the divergent thinking state should not be ended (NO in step S75), the process returns to step S61.

[0194] On the other hand, if it is determined that the process for determining the divergent thinking state is to be ended (YES in step S75), the process ends.

[0195] Next, the questionnaire processing by the server 2 in this embodiment will be described.

[0196] FIG. 18 is a flowchart for explaining the questionnaire processing by the server 2 in this embodiment.

[0197] First, in step S81, the communication unit 21 receives, from the terminal device 3, survey result information indicating the results of a survey about the divergent thinking state assessment result presented to the user. After presenting the divergent thinking state assessment result to the user, the terminal device 3 presents the user with a survey about the divergent thinking state assessment result and accepts the user's input to the survey. The survey presents three options: "I felt the divergent thinking state assessment result was appropriate," "I felt the divergent thinking state assessment criteria were strict," and "I felt the divergent thinking state assessment criteria were lenient." The user selects one of the three options.

[0198] The user checks the presented assessment result of their divergent thinking state, and if they feel that the assessment result of their divergent thinking state is appropriate, they select the option "I felt that the assessment result of their divergent thinking state was appropriate." Furthermore, if the user checks the presented assessment result of their divergent thinking state and feels that they were in a divergent thinking state, but the assessment result indicates that they were not in a divergent thinking state, they select the option "I felt that the assessment criteria for divergent thinking states were strict." Furthermore, if the user checks the presented assessment result of their divergent thinking state and feels that they were not in a divergent thinking state, but the assessment result indicates that they were in a divergent thinking state, they select the option "I felt that the assessment criteria for divergent thinking states were lenient."

[0199] The terminal device 3 transmits questionnaire result information indicating the results of the user's questionnaire to the server 2. The questionnaire result information includes evaluation information indicating the user's evaluation of the divergent thinking state assessment result. The evaluation information includes an evaluation that the divergent thinking state assessment result is appropriate, an evaluation that the divergent thinking state assessment criteria are strict, or an evaluation that the divergent thinking state assessment criteria are lenient.

[0200] Next, in step S82, the threshold change unit 226 determines whether the user's questionnaire result indicates that the assessment result of the divergent thinking state is appropriate. If it is determined that the user's questionnaire result indicates that the assessment result of the divergent thinking state is appropriate (YES in step S82), the threshold change unit 226 ends the questionnaire process without changing the current first threshold.

[0201] On the other hand, if it is determined that the user survey result does not indicate that the assessment result of the divergent thinking state is appropriate (NO in step S82), then in step S83, the threshold change unit 226 determines whether the user survey result indicates that the assessment criteria for the divergent thinking state are lenient. Here, if it is determined that the user survey result indicates that the assessment criteria for the divergent thinking state are lenient (YES in step S83), then in step S84, the threshold change unit 226 increases the first threshold. In this case, the threshold change unit 226 may add 10% of the current first threshold to the current first threshold. Alternatively, the threshold change unit 226 may add a predetermined value to the current first threshold.

[0202] On the other hand, if the user survey result is determined not to be an evaluation that the criteria for determining a divergent thinking state are lenient, i.e., if the user survey result is determined to be an evaluation that the criteria for determining a divergent thinking state are strict (NO in step S83), the threshold changing unit 226 decreases the first threshold in step S85. In this case, the threshold changing unit 226 may subtract 10% of the current first threshold from the current first threshold. Alternatively, the threshold changing unit 226 may subtract a predetermined value from the current first threshold.

[0203] Next, in step S86, the threshold value changing unit 226 stores the changed first threshold value in the storage unit 23.

[0204] The terminal device 3 may first confirm whether the user felt that the assessment result of the divergent thinking state was appropriate. If the user answers that the assessment result of the divergent thinking state was appropriate, the threshold change unit 226 may end the questionnaire process. If the user answers that the assessment result of the divergent thinking state was inappropriate, the terminal device 3 may allow the user to select either "I felt that the assessment criteria for the divergent thinking state were strict" or "I felt that the assessment criteria for the divergent thinking state were lenient."

[0205] Furthermore, if the judgment result is a convergent thinking state, the terminal device 3 may confirm whether or not the user felt the judgment result of the thinking state was appropriate. If the user answers that they felt the judgment result of the thinking state was appropriate, the threshold change unit 226 may end the questionnaire process. If the user answers that they felt the judgment result of the thinking state was inappropriate, the threshold change unit 226 may decrease the first threshold.

[0206] In the present embodiment, the mental state assessment system includes a camera 1. However, the present disclosure is not limited to this. Instead of the camera 1, a wearable sensor attached to the user's wrist may be included. The wearable sensor is an example of a sensor and detects acceleration or angular velocity. The wearable sensor is, for example, a gyro sensor or an acceleration sensor. The wearable sensor is attached to the wrist of the user's non-dominant hand. The fluctuation amount calculation unit 224 may calculate the fluctuation amount based on a physical quantity including at least one of angular velocity and acceleration detected by the wearable sensor attached to the user's wrist. The fluctuation amount calculation unit 224 may calculate the standard deviation of the x-axis acceleration, the standard deviation of the y-axis acceleration, and the standard deviation of the z-axis acceleration of the wrist of the non-dominant hand during the first period. Furthermore, the fluctuation amount calculation unit 224 may calculate the standard deviation of the x-axis angular velocity, the standard deviation of the y-axis angular velocity, and the standard deviation of the z-axis angular velocity of the wrist of the non-dominant hand during the first period.

[0207] In this embodiment, the image captured by camera 1 may be a color image or a monochrome image. By using a monochrome image, the amount of data to be transmitted to server 2 can be reduced, and the load on image processing, such as detecting wrist position information, can be reduced.

[0208] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Furthermore, the program may be executed by another independent computer system by recording the program on a recording medium and transferring it, or by transferring the program via a network.

[0209] Some or all of the functions of the device according to the embodiment of the present disclosure are typically realized as an LSI (Large Scale Integration), which is an integrated circuit. These may be individually integrated into a single chip, or some or all of them may be integrated into a single chip. Furthermore, the integrated circuit is not limited to an LSI, and may be realized using a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array), which can be programmed after LSI manufacturing, or a reconfigurable processor, which can reconfigure the connections and settings of circuit cells within an LSI.

[0210] Furthermore, some or all of the functions of the device according to the embodiment of the present disclosure may be realized by a processor such as a CPU executing a program.

[0211] Furthermore, all the numbers used above are merely examples to specifically explain the present disclosure, and the present disclosure is not limited to the numbers used as examples.

[0212] The order in which the steps are executed in the above flowchart is merely an example for specifically explaining the present disclosure, and other orders may be used as long as similar effects are obtained. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0213] The technology disclosed herein is useful as a technology for determining a user's thought state because it can easily determine the user's thought state from the user's wrist movements, which are easily detectable while the user is working.

[0214] REFERENCE SIGNS LIST 1 camera 2 server 3 terminal device 4 work medium 11 imaging unit 12 signal processing unit 13 memory unit 14 communication unit 21 communication unit 22 processor 23 memory unit 221 handedness determination unit 222 work determination unit 223 reference value determination unit 224 variation amount calculation unit 225 thinking state determination unit 226 threshold change unit 231 user information

Claims

1. An information processing method executed by a computer, comprising: acquiring an amount of variation in wrist movement of a user while working; and determining a distracted thinking state of the user based on the amount of variation.

2. The information processing method according to claim 1, wherein said determining whether said user is in a divergent thinking state includes determining that said user is in said divergent thinking state if said amount of variation is equal to or greater than a threshold value.

3. The information processing method according to claim 2, further comprising: acquiring evaluation information indicating the user's evaluation of the result of the divergent thinking state assessment; and changing the threshold value based on the evaluation information.

4. The information processing method according to any one of claims 1 to 3, further comprising determining the degree of the divergent thinking state of the user according to the magnitude of the amount of variation.

5. An information processing method according to any one of claims 1 to 3, wherein the acquisition of the amount of variation includes calculating the amount of variation based on a change in the position of the wrist in a video image of the wrist of the user.

6. The information processing method of claim 5, further comprising: acquiring position information of a work medium used by the user in the work based on the moving image; and determining whether to use the amount of movement of either the user's right wrist or left wrist to judge the divergent thinking state based on the position information.

7. The information processing method of claim 5, further comprising determining a dominant hand of the user based on the video image, and wherein determining the divergent thinking state comprises determining the divergent thinking state based on the amount of variation in wrist movement of the hand other than the dominant hand.

8. An information processing method according to any one of claims 1 to 3, wherein the acquisition of the amount of variation includes calculating the amount of variation based on a physical quantity including at least one of angular velocity and acceleration detected by a sensor worn on the wrist of the user.

9. The information processing method according to claim 8, wherein the sensor is attached to the wrist of the user's non-dominant hand.

10. An information processing method according to any one of claims 1 to 3, further comprising determining whether the user is engaged in the task, and wherein the determination of the divergent thinking state comprises determining the divergent thinking state of the user based on the amount of variation during a task period in which the user is engaged in the task.

11. The information processing method of claim 10, wherein the determination of whether the user is working on the task includes determining whether the user is working on the task based on at least one of gaze information indicating the user's gaze direction, face position information indicating the position of the user's face on the image, wrist position information indicating the position of the user's wrist on the image, seating pressure information indicating the user's seating pressure, and input information from the user acquired by a work medium used by the user in the task.

12. An information processing device including a processor, wherein the processor acquires an amount of variation in wrist movement of a user while working, and determines a divergent thinking state of the user based on the amount of variation.

13. An information processing program that causes a computer to function in such a way that it acquires an amount of variation in the movement of a user's wrist while working, and determines the user's state of divergent thinking based on the amount of variation.

Images