Information processing method and information processing system

Abstract

An information processing method includes: acquiring, using a processor, at least one index selected from an index group including an amount of variation in a position of a user's face, an amount of variation in an orientation of the face, an amount of variation in an orientation of a user's line of sight, and an eye closing rate of the user's eyes (step S101); and determining whether or not the user is in a divergent thinking state on the basis of the acquired at least one index (step S106).

Description

Technical Field

[0001] The present disclosure relates to an information processing method and an information processing system. Background Art

[0002] Human creative thinking includes: convergent thinking, which is to enter into logical thinking based on known information and come up with a single solution; and divergent thinking, which is to generate new ideas through repeated thinking based on known information.

[0003] Prior art literature

[0004] Patent Document 1: Japanese Patent Application Publication No. 2021-23492

[0005] Patent Document 2: Japanese Patent Application Publication No. 2016-224142

[0006] Patent Document 3: Japanese Patent Application Publication No. 2021-35499

[0007] Patent Document 4: Japanese Patent Application Publication No. 2020-8278

[0008] Non-patent literature

[0009] Non-patent literature 1: S. Mednick.: The associative basis of the creative process. Psychol. Rev. 69, 220-232, (1962).

[0010] Non-patent document 2: R. Orita, M. Hattori, Y. Nishida.: Development of a Japanese Remote Associates Task as insight problems. Japanese J. Psychol. 89, 376-386, (2018).

[0011] Non-patent literature 3: & R. Raven, JC, Court, JH: Raven's progressive matrices.; Oxford Psychologists Press, 1998.

[0012] Non-patent literature 4: LEMatzen, ZOBenz, KRDixon, J.Posey, JKKroger, AESpeed.: Recreating Raven's: Software for systematically generating large numbers of Raven-like matrix problems with normedproperties. Behav.Res.Methods 42, 525-541, (2010).

[0013] Non-patent literature 5: MA Wallach, Nathan Kogan.: Modes of thinking in young children; A study of the creativity-intelligence distinction; Holt, Rinehart and Winston, 1965.

[0014] Non-Patent Document 6: Koichiro Tomimoto, Hiroshi Hashiura, Kozo Tsuji, Kenki Kimura, "Office Lighting System for Improving Workers' Creativity - Study on Lighting Conditions Suitable for Creative Office Scenes", Proceedings of the Conference of the Kansai Branch of the Japanese Ergonomics Society 2009, pp. 171-174, December 5, 2009 Summary of the invention

[0015] The present disclosure provides an information processing method and the like that can objectively evaluate a subject's creative thinking state.

[0016] In order to solve the above-mentioned problems, an information processing method of one embodiment of the present invention includes: using a processor to obtain at least one indicator selected from an indicator group including a change in the position of a user's face, a change in the orientation of the face, a change in the orientation of the user's line of sight, and a rate of eye closure of the user's eyes; and determining whether the user is in a divergent thinking state based on the at least one indicator obtained.

[0017] In addition, these general or specific methods can be implemented through systems, devices, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or can be implemented through any combination of systems, devices, integrated circuits, computer programs, and recording media.

[0018] The information processing method disclosed in the present invention can objectively evaluate the creative thinking state of the subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is an explanatory diagram showing the amount of change in the position of the subject's face, etc., according to each thought state.

[0020] Figure 2 It is an explanatory diagram schematically showing an example of the appearance and usage of the living body measurement device in the embodiment.

[0021] Figure 3 This is a block diagram showing an example of the configuration of a living body measurement device according to an embodiment.

[0022] Figure 4 This is a flowchart showing a method of determining a thought state based on biometric data of a user, which is executed by the biometric measurement device in the embodiment.

[0023] Figure 5 It is an explanatory diagram showing an example of a mental state determined by the living body measurement device in the embodiment.

[0024] Fig. 6A This is an explanatory diagram of a first example of presentation of determination results by the living body measurement apparatus according to the embodiment.

[0025] Figure 6B It is an explanatory diagram of a second example of display of determination results in the embodiment.

[0026] Figure 6C It is an explanatory diagram of a third example of the display of the determination result in the embodiment.

[0027] Fig.6D It is an explanatory diagram of a fourth example of the display of the determination result in the embodiment.

[0028] Fig. 6E It is an explanatory diagram of a fifth example of the display of the determination result in the embodiment.

[0029] Fig. 6F It is an explanatory diagram of a sixth example of the display of the determination result in the embodiment.

[0030] Figure 6G It is an explanatory diagram of a seventh example of the display of the determination result in the embodiment.

[0031] Figure 7 This is a flowchart showing an example of a process of controlling lighting according to a determination result in the embodiment.

[0032] Figure 8 It is an explanatory diagram showing the relationship between the illuminance and color temperature of lighting and the user's mental state in lighting control according to the embodiment.

[0033] Fig. 9 This is a flowchart showing processing related to updating of threshold information in the embodiment. DETAILED DESCRIPTION

[0034] (Insights forming the basis of this disclosure)

[0035] Before describing the embodiments of the present disclosure, the findings found by the inventors of the present application will be described.

[0036] The government is promoting work style reforms in the context of a labor shortage in an aging society with a low birth rate. Companies are working to build various mechanisms to improve employees' intellectual productivity. Employees' willingness to work is closely related to office space, and environmental design or space control that improves intellectual productivity is being developed.

[0037] In order to improve intellectual productivity, it is important not only for employees to be able to focus on their work, but also to be able to think more creatively. Creative thinking includes: convergent thinking, which is to enter logical thinking based on known information and come up with a single solution; and divergent thinking, which is to think repeatedly based on known information to generate new ideas.

[0038] Although methods for evaluating the impact of the surrounding environment on thinking state have been studied in the past, no standardized objective quantitative indicators have been established.

[0039] Specifically, a method of estimating the concentration level of a subject based on data obtained from a sensor that monitors the subject is proposed in Patent Documents 1 to 4. However, Patent Documents 1 to 4 do not disclose a mechanism for objectively evaluating the mental state of the subject.

[0040] The present disclosure provides an information processing method and the like capable of objectively evaluating a person's creative thinking state.

[0041] Hereinafter, examples of biological information when a subject is asked to perform a task requiring convergent thinking and a task requiring divergent thinking will be described.

[0042] Figure 1 This is a graph showing an example of biological information of 50 subjects when they performed tasks requiring convergent thinking and tasks requiring divergent thinking.

[0043] about Figure 1 The biological information of the subject is described in detail below.

[0044] As the topics requiring convergent thinking, the Japanese version of Remote Association Task (RAT) was used as a language-based topic, and the Raven Task (Raven) was used as a non-language-based topic.

[0045] For topics that require divergent thinking, the Unusual Uses Test (UUT) was used for language-based topics, and the Figural Divergent Task (FDT) was used for non-language-based topics.

[0046] The RAT was developed by Mednick (see Non-Patent Document 1). Mednick presented three words that seemingly had no commonality at first glance (e.g., "Pure", "Blue", and "Fall") to the experimental participants and asked them to find a common word ("Water") associated with each word.

[0047] The RAT is creating topics based on languages other than English and revising the topic itself. In this experiment, the questions prepared by Oda et al. were used (see Non-Patent Document 2). The guidance for the RAT topic for the subjects was as follows. "Please orally answer the Chinese character of the word that is related to all three Chinese characters. When answering, please orally answer the word related to the left Chinese character. For example, when the three Chinese characters are '異, 口, 序', the correct answer is '論' because it becomes '異論, 口論, 序論'. In this case, when answering, please answer '论 (lun)'. Please answer as quickly and accurately as possible. After answering, proceed to the next question. If no answer is given after 45 seconds, it will also proceed to the next question. One point is given for a correct answer, and -1 point is given for an incorrect answer or timeout."

[0048] Raven’s Progressive Matrices is widely used as a test for evaluating intelligence and reasoning ability (see Non-Patent Document 3). Matzen et al. analyzed the types of relationships that appear in the Standard Progressive Matrices (SPM) of Raven’s Progressive Matrices and created a software tool that can generate a very large number of matrices by combining the same types of relationships according to the parameters selected by the experimenter (see Non-Patent Document 4). In this experiment, the Raven topic was created using the publicly available software. The guidance for the Raven topic for the subjects was as follows. "There are regular geometric figures arranged here. Please orally answer the figure that is selected from 8 options and filled in the blank part. When answering, please orally answer the numbers 1 to 8 attached to each option. In this case, '1' is the correct answer. Please answer as quickly and accurately as possible. After answering, proceed to the next question. If no answer is given after 45 seconds, it will also proceed to the next question. One point is given for a correct answer, and -1 point is given for an incorrect answer or timeout."

[0049] UUT is a topic that lists many unique uses of things and measures the degree of divergent thinking. The instructions for the UUT topic for the subjects are as follows. "Please say as many uses of 'something' as possible that are different from the common usage within 3 minutes. There is no correct answer, so please say whatever comes to mind. The answer should be specific and the voice should be clear. For example, 'something' is 'socks.' In this case, 'roll the socks into a ball and make a small cloth pocket' is a usage that is different from the common usage. On the other hand, 'wear socks' is a common usage. The score will be based on the number and originality of the answers. Please give as many original answers as possible. There are no wrong answers and no points will be deducted." In addition, in Figure 1 In the measurement of the recorded biological information, the subjects were required to answer as many uses of the word "ping-pong" as possible that were different from the normal uses within 3 minutes.

[0050] FDT is a task to answer as many questions as possible about what a figure may represent (see non-patent document 5). The instructions for the FDT task for the subjects are as follows. "Please say as many things as possible about what the figure may represent within 3 minutes. Since there is no correct answer, please say whatever comes to mind. The answer should be specific and the voice should be clear. For example, the figure is a '□' (i.e., a hollow square). In this case, the figure may represent 'bread', and the score will be based on the number and originality of the answers. Please answer as many original answers as possible. There are no wrong answers and no points will be deducted." In this experiment, the subjects were required to answer as many uses of "+" (i.e., a figure composed of a horizontal line segment and a line segment perpendicular to the line segment) as possible within 3 minutes that are different from the common usage.

[0051] The average and standard deviation of the subject's face position, face orientation, and line of sight orientation were calculated for each subject for 1 minute when the subject was not performing the subject (calm state) and 3 minutes when the subject was performing the subject (execution state). In addition, the average of the subject's eye closure rate for 1 minute in the calm state and the eye closure rate for 3 minutes in the execution state was calculated. The eye closure rate is the proportion of time per minute that the eyes are closed. The 3 minutes of the execution state are divided into 1 minute each, and 3 values ​​are calculated as the eye closure rate in the execution state. The average of these 3 values ​​is the eye closure rate in the execution state.

[0052] Next, the difference of the above standard deviation of the performing state relative to the calm state is calculated for each subject for the subject's face position, face orientation, and line of sight orientation. The standard deviation is called a variation as a value indicating the degree of deviation of the data, and can be used to compare the variation of the value in the calm state with the variation of the value in the performing state. In addition, the difference of the eye-closing rate in the performing state relative to the average value of the eye-closing rate in the calm state is calculated for each subject. Finally, the average value of the above difference in the face position of all subjects, the average value of the above difference in the face orientation of all subjects, the average value of the above difference in the line of sight orientation of all subjects, and the average value of the above difference in the eye-closing rate of all subjects are compared when a topic requiring convergent thinking is being performed and when a topic requiring divergent thinking is being performed.

[0053] In addition, as the variation amount, in addition to the above-mentioned standard deviation of the position of the subject's face, etc., an index (also referred to as a deviation index) indicating the variation of the position of the subject's face, etc. may be used. The deviation index is, for example, variance.

[0054] In addition, as the variation amount, the difference between the deviation index of the position of the face of the subject in the calm state and the deviation index of the face of the subject in the execution state can be used. Specifically, for example, the value obtained by subtracting the deviation index of the position of the face of the subject in the calm state from the deviation index of the position of the face of the subject measured can be used as the variation amount. For example, the deviation index of the position of the face of the subject in the calm state can be measured in advance and stored in the storage device.

[0055] Figure 1 (a) shows the average value of the difference in the amount of change in the face position of all subjects in each state of the execution state relative to the calm state, the convergent thinking state (abbreviated as "convergence" in the figure) and the divergent thinking state (abbreviated as "divergence" in the figure). The amount of change in the face position shows how much the face has moved from the reference position in the image of the subject's face. Here, as the amount of change in the face position, a value obtained by standardizing the amount of change in the face position is used. The unit is "point". However, as the amount of change in the face position, a value standardized using a personal feature such as the width of the face, the number of pixels on the acquired image, or a length calculated based on the specifications of the camera lens (in millimeters) can be used.

[0056] Figure 1(b) shows the average value of the difference in the amount of change in the face orientation of all subjects in each state of the convergent thinking state (abbreviated as "convergence" in the figure) and the divergent thinking state (abbreviated as "divergence" in the figure) in the execution state relative to the calm state. The amount of change in the face orientation is the amount of change in the three-dimensional orientation of the face obtained by analyzing the image of the subject's face taken, and shows how much the face orientation is tilted from the reference direction. Here, as the amount of change in the face orientation, the angle of the face orientation relative to the reference direction is used.

[0057] It can be seen that the subject's face position and face orientation change more in the execution state (i.e., the state of convergent thinking or divergent thinking) than in the calm state.

[0058] Generally speaking, it is believed that when a person is concentrating on a simple task, the position and orientation of the face change less than when the person is calm because the person is focusing on the position of the task display and suppressing useless movements. On the other hand, it is known that when the subject is engaged in creative thinking, that is, convergent thinking and divergent thinking, there is a tendency different from when the subject is concentrating on a simple task.

[0059] In addition, it is known that when the subject is engaged in divergent thinking, the amount of change in the position and orientation of the face is greater than when the subject is engaged in convergent thinking. Therefore, when the amount of change in the position and orientation of the face of the subject increases compared to the calm state, it can be estimated that the subject is engaged in creative thinking, and when the amount of change in the position and orientation of the face of the subject further increases, it can be estimated that the subject is approaching divergent thinking or the degree of divergent thinking is increasing.

[0060] Figure 1 (c) shows the average value of the difference in the amount of change in the direction of the line of sight of all subjects in each state of the convergent thinking state (abbreviated as "convergence" in the figure) and the divergent thinking state (abbreviated as "divergence" in the figure) in the execution state relative to the calm state. The amount of change in the direction of the line of sight is the amount of change in the direction of the line of sight obtained by analyzing the image of the face of the subject, and shows how much the direction of the line of sight has changed from the reference direction. Here, as the amount of change in the direction of the line of sight, the angle of the direction of the line of sight relative to the reference direction is used.

[0061] Figure 1 (d) shows the average value of the difference in eye closure rate of all subjects in each state of convergent thinking state (abbreviated as "convergent" in the figure) and divergent thinking state (abbreviated as "divergent" in the figure) in the execution state relative to the calm state. The eye closure rate is the proportion of time the eyes are closed per unit time (for example, 1 minute).

[0062] It is known that in the execution state (i.e., the state of performing convergent thinking or divergent thinking), the change in line of sight is large, and the rate of eye closure is also large compared to the calm state. This is also a tendency opposite to the state of a person who is generally concentrating on a simple task, and can be said to be a characteristic of creative thinking. Moreover, when performing divergent thinking, the change in the direction of the line of sight and the rate of eye closure are larger than when performing convergent thinking. When the change in the direction of the line of sight or the rate of eye closure is larger than that in the calm state, it can be estimated that the person is performing creative thinking, and it can also be estimated that the person is approaching divergent thinking or the degree of divergent thinking is increasing.

[0063] For example, when the amount of change in the face orientation and the eye closing rate increase compared to a calm state, it can be estimated that the subject is approaching divergent thinking. For example, when only the eye closing rate is increasing among the amount of change in the face orientation and the eye closing rate, the subject may not be approaching a divergent thinking state but may feel sleepy. Therefore, by recombining at least one of the biological information of the position of the face, the orientation of the face, or the amount of change in the direction of the sight line, it is possible to improve the estimation accuracy of the thinking state.

[0064] Hereinafter, the technology obtained based on the disclosure of this specification is exemplified, and the effects etc. obtained by the technology are described.

[0065] (1) An information processing method, comprising: using a processor to obtain at least one indicator selected from an indicator group including a change in the position of a user's face, a change in the orientation of the face, a change in the orientation of the user's line of sight, and a rate of closure of the user's eyes; and determining whether the user is in a divergent thinking state based on the at least one indicator obtained.

[0066] According to the above-mentioned method, it is possible to determine whether the user is in a divergent thinking state using at least one indicator selected from an indicator group including the amount of change in the position of the face of the user as the subject, the amount of change in the orientation of the face, the amount of change in the orientation of the user's line of sight, and the rate of closure of the user's eyes. In the past, there was no known technology for determining whether the user is in a divergent thinking state based on the amount of change in the position of the face of the user, the amount of change in the orientation of the face, the amount of change in the orientation of the user's line of sight, or the rate of closure of the user's eyes. According to the above-mentioned information processing method, it is possible to determine whether the user is in a divergent thinking state with higher accuracy based on the amount of change in the position of the face of the user, the amount of change in the orientation of the face, the amount of change in the orientation of the user's line of sight, or the rate of closure of the user's eyes. In this way, according to the above-mentioned information processing method, the creative thinking state of the subject can be objectively evaluated.

[0067] (2) An information processing method according to (1), wherein the determination includes: comparing the at least one indicator with a first threshold value corresponding to the at least one indicator; and when it is determined that the at least one indicator is above the first threshold value, determining that the user is in a divergent thinking state.

[0068] According to the above aspect, by determining the difference between the index and the first threshold, it is possible to easily determine whether the user is in a divergent thinking state. Thus, according to the above information processing method, it is possible to more easily and objectively evaluate the creative thinking state of the subject.

[0069] (3) An information processing method according to (2), wherein the judgment includes: comparing the at least one indicator with a second threshold corresponding to the at least one indicator and different from the first threshold; and when it is judged that the at least one indicator is less than the first threshold and greater than the second threshold, judging that the user is in a convergent thinking state.

[0070] According to the above aspect, by determining the difference between the index and the first threshold and the second threshold, it is possible to easily determine whether the user is in a convergent thinking state. Thus, according to the above information processing method, it is possible to more easily and objectively evaluate the creative thinking state of the subject.

[0071] (4) The information processing method according to (2) or (3), wherein the determination includes determining, for each of a plurality of indicators selected from the indicator group, whether the indicator is equal to or greater than the first threshold value corresponding to the indicator.

[0072] According to the above method, since multiple indicators are used as at least one indicator selected from the indicator group to determine whether the user is in a divergent thinking state, it is helpful to improve the appropriateness of the determination and improve the accuracy of the determination result. In this way, according to the above information processing method, the creative thinking state of the subject can be objectively and accurately evaluated.

[0073] (5) An information processing method according to (4), wherein the judgment includes: when it is judged that the change in the orientation of the face is greater than the first threshold value corresponding to the change in the orientation of the face, and the change in the orientation of the line of sight is greater than the first threshold value corresponding to the change in the orientation of the line of sight, it is judged that the user is in the divergent thinking state.

[0074] According to the above method, the amount of change in the orientation of the face and the amount of change in the orientation of the line of sight are used as at least one indicator selected from the indicator group to determine whether the user is in a divergent thinking state. Through experiments conducted by the inventors of the present application and others, it has been confirmed that the amount of change in the orientation of the face and the amount of change in the orientation of the line of sight have a greater difference between the values ​​in a calm state and the values ​​in a divergent thinking state than other indicators. Therefore, by using the amount of change in the orientation of the face and the amount of change in the orientation of the line of sight as at least one indicator, it is helpful to improve the appropriateness of the determination and improve the accuracy of the determination result. In this way, according to the above information processing method, the creative thinking state of the subject can be objectively and accurately evaluated.

[0075] (6) An information processing method according to (4), wherein the determination includes determining the degree of the divergent thinking state of the user in the unit period based on the number of indicators above the first threshold value corresponding to the indicator among the multiple indicators obtained in the unit period.

[0076] According to the above method, multiple indicators are used as at least one indicator selected from the indicator group to determine whether the user is in a divergent thinking state, and the degree of the divergent thinking state is determined. Thus, the creative thinking state of the subject can be evaluated based on the degree of the divergent thinking state. Therefore, according to the above information processing method, the creative thinking state of the subject can be objectively and accurately evaluated.

[0077] (7) An information processing method according to (3), wherein the determination includes: using the at least one indicator obtained in a plurality of unit periods to determine whether the user is in a divergent thinking state and a convergent thinking state in the plurality of unit periods; generating a graph showing the ratio of the period during which the user is determined to be in the convergent thinking state to the period during which the user is determined to be in the divergent thinking state; and displaying the graph.

[0078] According to the above aspect, since a graph showing the ratio of the user's thinking state in each of the multiple periods is displayed, it is helpful for the viewer of the graph to centrally grasp the user's thinking state throughout the multiple periods. Therefore, according to the above information processing method, the creative thinking state of the subject throughout the multiple periods can be objectively evaluated, and the viewer can easily grasp the subject's thinking state.

[0079] (8) An information processing method according to any one of (2) to (7), wherein the information processing method further includes: obtaining identification information for identifying the user; and obtaining threshold information corresponding to the identification information, wherein the first threshold is determined based on the threshold information.

[0080] According to the above method, since the first threshold used in the index size determination is determined based on the threshold information, the first threshold can be made more appropriate, and it can be appropriately determined whether the user's thinking state is in a divergent thinking state. Therefore, according to the above information processing method, the creative thinking state of the subject can be objectively and accurately evaluated.

[0081] (9) An information processing method according to (8), wherein the information processing method further includes: obtaining evaluation information that evaluates the user's thinking state through a method different from the determination processing; and updating the threshold information based on a determination result of the user's thinking state and the evaluation information.

[0082] According to the above method, since the first threshold is updated based on the subjective evaluation information of the user's thinking-related state evaluated by a method different from the above determination processing, it helps to make the subsequent determination result of the user's thinking state closer to the user's evaluation. Therefore, according to the above information processing method, it is possible to make the determination result of the subject's thinking state close to the subject's subjective evaluation and objectively evaluate the subject's creative thinking state.

[0083] (10) An information processing method according to (3), wherein, when it is determined that the at least one indicator is less than the first threshold and greater than the second threshold, the lighting device illuminating the vicinity of the user is controlled to reduce the color temperature of the lighting light and increase the illuminance.

[0084] According to the above method, when it is determined that the user's thinking-related state is a convergent thinking state, the lighting device can be used to appropriately guide the user to a divergent thinking state. Therefore, according to the above information processing method, the creative thinking state of the subject can be objectively evaluated, and the user determined to be in a convergent thinking state can be guided to a divergent thinking state.

[0085] (11) An information processing method according to any one of (2) to (9), comprising: when it is determined that the at least one indicator is greater than the first threshold, controlling the lighting device that illuminates the vicinity of the user to increase the color temperature of the lighting light and increase the illuminance.

[0086] According to the above method, when it is determined that the user's thinking state is a divergent thinking state, the lighting device can be used to appropriately guide the user to a convergent thinking state. Therefore, according to the above information processing method, the creative thinking state of the subject can be objectively evaluated, and the user determined to be in a divergent thinking state can be guided to a convergent thinking state.

[0087] (12) An information processing method according to (3), comprising: when it is determined that the at least one indicator is less than the second threshold value, controlling the lighting device that illuminates the vicinity of the user to (i) lower the color temperature of the lighting light and increase the illuminance or (ii) increase the color temperature of the lighting light and increase the illuminance.

[0088] According to the above method, when it is determined that the user's thinking-related state is neither a divergent thinking state nor a convergent thinking state, the lighting device can be used to appropriately guide the user to a divergent thinking state or a convergent thinking state. Therefore, according to the above information processing method, it is possible to objectively evaluate the creative thinking state of the subject, and guide the user who is determined to be neither a divergent thinking state nor a convergent thinking state to a divergent thinking state or a convergent thinking state.

[0089] (13) An information processing method according to any one of (1) to (12), wherein the obtaining includes: obtaining an image generated by a photographing device that photographs the user; and obtaining the at least one indicator by performing an analysis process on the obtained image.

[0090] According to the above aspect, since the index is acquired by analyzing the image in which the user is projected, the process of acquiring the index can be easily realized. Therefore, according to the above information processing method, the creative thinking state of the subject can be more easily objectively evaluated.

[0091] (14) An information processing method, comprising: using a processor to obtain target information indicating that a user is in a state set as a target, namely, a target state; obtaining at least one indicator selected from a group of indicators including a change in the position of the user's face, a change in the orientation of the face, a change in the orientation of the user's line of sight, and a rate of eye closure of the user's eyes; determining the user's state of mind based on the at least one indicator; comparing the target state indicated by the target information with the determined state of mind of the user; and, if the result of the comparison is that the target state is different from the determined state of mind of the user, controlling a change in the characteristics of the illumination light irradiated by the illumination device illuminating the vicinity of the user.

[0092] According to the above method, when the user's thinking state determined based on the amount of change in the position of the user's face is different from the target state, the user's thinking state can be appropriately guided to the target state using the lighting device. Therefore, according to the above information processing method, the creative thinking state of the subject can be objectively evaluated, and the user's thinking state can be guided to the target state.

[0093] (15) An information processing system, comprising: a camera that captures a user and generates an image; a processing circuit; a storage device; and a display device, wherein the processing circuit performs analysis processing on the image generated by the camera to obtain at least one indicator selected from a group of indicators including a change in the position of the user's face, a change in the orientation of the face, a change in the orientation of the user's line of sight, and a rate of eye closure of the user during a unit period, obtains threshold information representing a first threshold corresponding to the at least one indicator from the storage device, and performs determination processing to determine whether the user is in a divergent thinking state during the unit period based on a result of determining whether the at least one indicator during the unit period is above the first threshold, and displays an image corresponding to the result of the determination processing on the display device.

[0094] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0095] In addition, these general or specific methods can be implemented by systems, devices, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, devices, integrated circuits, computer programs, or recording media.

[0096] Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0097] In addition, the embodiments described below all represent general or specific examples. The numerical values, shapes, materials, constituent elements, configuration positions and connection methods of constituent elements, steps, and the order of steps shown in the following embodiments are all examples, and their purport is not to limit the present disclosure. In addition, among the constituent elements of the following embodiments, constituent elements not recorded in the independent technical solutions representing the highest concept are described as arbitrary constituent elements.

[0098] (Implementation Method)

[0099] In the present embodiment, an information processing method and the like that can objectively evaluate the creative thinking state of a subject will be described.

[0100] Figure 2 It is an explanatory diagram schematically showing an example of the appearance and usage of the biological measurement device 1 according to the present embodiment.

[0101] The biological measurement device 1 is an information processing device that measures biological information of the user 100 .

[0102] The biometric measurement device 1 includes a camera 10 and a display device 20. When the biometric measurement device 1 measures biometric information, the camera 10 is provided separately from the user 100 so as to reflect the head (more specifically, the face) of the user 100 as a subject. While the user 100 is solving a problem, the camera 10 captures the face of the user 100 and acquires a signal of image information of the face of the user 100. The camera 10 may repeatedly capture the face of the user 100. The acquisition of the image signal may be performed in real time or sequentially each time the image is captured.

[0103] Figure 3 It is a block diagram showing a configuration example of the living body measurement device 1 according to the present embodiment.

[0104] like Figure 3 As shown, the living body measurement device 1 includes a camera 10, an acquisition unit 11, an analysis unit 12, a storage unit 13, a determination unit 14, an output unit 15, and a display device 20. The analysis unit 12, the storage unit 13, the determination unit 14, and the output unit 15 can be implemented by a processor provided in the living body measurement device 1 executing a predetermined program.

[0105] The acquisition unit 11 acquires an image by converting a signal of image information acquired by the camera 10 into an image.

[0106] The analyzing unit 12 extracts the feature points of the contour, eyes or mouth of the user 100's face, etc. by performing an analyzing process (e.g., face recognition process) on the image obtained by the obtaining unit 11 and showing the face of the user 100. The analyzing unit 12 uses the extracted feature points to determine at least one of the position of the user 100's face, the orientation of the face, the orientation of the sight line, and the rate of eye closure. Furthermore, the analyzing unit 12 calculates at least one index selected from the amount of change in the position of the user 100's face, the amount of change in the orientation of the face, the amount of change in the orientation of the sight line, and the rate of eye closure, and stores the calculated index in the storage unit 13. The above-mentioned index is also referred to as biometric information. The amount of change in the position of the face, the amount of change in the orientation of the face, the amount of change in the orientation of the sight line, and the rate of eye closure are also referred to as an index group. The analyzing unit 12 obtains at least one index selected from the above-mentioned index group as described above.

[0107] The amount of change in the position of the face of the user 100 corresponding to time t0 and time t1 can be determined based on the position of the face of the user 100 in a calm state at time t0 and the position of the face of the user 100 at time t1.

[0108] The amount of change in the orientation of the face of the user 100 corresponding to time t0 and time t1 can be determined based on the orientation of the face of the user 100 in a calm state at time t0 and the orientation of the face of the user 100 at time t1.

[0109] The amount of change in the direction of the sight line of the user 100 corresponding to time t0 and time t1 can be determined based on the direction of the sight line of the user 100 in a calm state at time t0 and the direction of the sight line of the user 100 at time t1. It is also possible to obtain multiple indicators based on a moving image obtained by a single camera, and determine the state of mind based on these indicators. In this way, compared with a complex structure using multiple devices, the state of mind can be determined simply and with high accuracy.

[0110] Based on the moving image acquired by a single camera, the amount of change in the orientation of the face and the amount of change in the orientation of the sight line can be derived without deriving the position of the face and the rate of eye closure. As a result, while making a high-precision determination using the amount of change in the orientation of the face and the amount of change in the orientation of the sight line, the derivation process of other indicators can be omitted, thereby reducing the processing load.

[0111] The storage unit 13 is a storage device for storing the indicators calculated by the analysis unit 12. In addition, threshold values ​​(specifically, a first threshold value and a second threshold value) corresponding to the indicators of the change amount of the position of the face, the change amount of the orientation of the face, the change amount of the orientation of the line of sight, and the closing rate of the eyes are pre-stored in the storage unit 13. The threshold value corresponding to the indicator is used for comparison with the indicator as a basis for determining whether the user 100 is in a divergent thinking state. The indicators and threshold values ​​stored in the storage unit 13 are read by the determination unit 14. The processor may also prevent information that is not used in the estimation process (for example, one of the position of the face, the orientation of the face, the orientation of the line of sight, and the closing rate of the eyes) from being stored in the storage unit 13. In this way, unnecessary consumption of the storage unit 13 can be reduced.

[0112] The determination unit 14 determines the thinking state of the user 100 based on the index recorded in the storage unit 13, that is, at least one index selected from the amount of change in the position of the face, the amount of change in the orientation of the face, the amount of change in the orientation of the line of sight, and the rate of closing the eyes. The determination of the thinking state at least includes the determination of whether the user 100 is in a divergent thinking state. The determination unit 14 provides the determination result to the output unit 15.

[0113] For example, the determination includes comparing the at least one indicator with a first threshold value corresponding to the at least one indicator. In the determination, if the at least one indicator is determined to be greater than the first threshold value, the determination unit 14 determines that the user 100 is in a divergent thinking state.

[0114] In addition, the above-mentioned judgment may include: comparing the above-mentioned at least one indicator with a second threshold corresponding to the above-mentioned at least one indicator and different from the above-mentioned first threshold; and when it is determined that the above-mentioned at least one indicator is less than the above-mentioned first threshold and above the second threshold, it is determined that the user 100 is in a convergent thinking state.

[0115] In addition, the at least one indicator may be a plurality of indicators. That is, the determination may include: determining whether each of the plurality of indicators selected from the indicator group is equal to or greater than a first threshold value corresponding to the indicator.

[0116] More specifically, the at least one indicator may be a change in the orientation of the face and a change in the orientation of the sight line. That is, the determination may include: if it is determined that the change in the orientation of the face is greater than a first threshold value corresponding to the change in the orientation of the face and the change in the orientation of the sight line is greater than a first threshold value corresponding to the change in the orientation of the sight line, it is determined that the user 100 is in a divergent thinking state.

[0117] The output unit 15 obtains the determination result of the determination unit 14 and converts the obtained determination result into an outputtable format. For example, when the determination result is output by the display device 20 , the output unit 15 generates an image showing the determination result and supplies it to the display device 20 .

[0118] The display device 20 displays an image indicating the determination result of the determination unit 14 .

[0119] In addition, the biometric device 1 may include other output devices (for example, a speaker or a communication interface) instead of the display device 20, or may include other output devices together with the display device 20. When the biometric device 1 includes a speaker, the output unit 15 generates voice data indicating the determination result and supplies it to the speaker, and the speaker outputs the voice. When the biometric device 1 includes a communication interface, the output unit 15 generates communication data including information indicating the determination result, supplies the communication data to the communication interface, and supplies it to other communication devices via the communication interface.

[0120] Figure 4 1 is a flowchart showing a method of determining a thought state based on the biometric data of the user 100 , which is executed by the biometric measurement device 1 in the present embodiment.

[0121] In step S101, the acquisition unit 11 acquires an image showing the face of the user 100 using the camera 10. In addition, the analysis unit 12 calculates at least one index based on the image acquired by the acquisition unit 11 and stores it in the storage unit 13. Here, the case where four indexes, namely, the position of the face, the direction of the face, the direction of the line of sight, and the closing rate of the eyes, are calculated as at least one index is described as an example.

[0122] The analyzing unit 12 may calculate at least one index after reducing the resolution of the image. This can reduce the amount of processing by the processor.

[0123] In step S102, the determination unit 14 reads the position of the face stored in the storage unit 13 and calculates the amount of change in the position of the face. In addition, the determination unit 14 evaluates the calculated amount of change in the position of the face. More specifically, the determination unit 14 determines whether the calculated amount of change in the position of the face is greater than a first threshold, and when it is determined that the amount of change is less than the first threshold, determines whether the amount of change is greater than a second threshold.

[0124] In step S103, the determination unit 14 reads the face orientation stored in the storage unit 13 and calculates the amount of change in the face orientation. In addition, the determination unit 14 evaluates the calculated amount of change in the face orientation. More specifically, the determination unit 14 determines whether the calculated amount of change in the face orientation is greater than a first threshold value, and when it is determined that the amount of change is less than the first threshold value, determines whether the amount of change is greater than a second threshold value.

[0125] In step S104, the determination unit 14 reads the direction of the sight line stored in the storage unit 13, and calculates the amount of change in the direction of the sight line. In addition, the determination unit 14 evaluates the calculated amount of change in the direction of the sight line. More specifically, the determination unit 14 determines whether the calculated amount of change in the direction of the sight line is greater than a first threshold, and when it is determined that the amount of change is less than the first threshold, determines whether the amount of change is greater than a second threshold.

[0126] In step S105, the determination unit 14 reads the eye closure rate stored in the storage unit 13 and calculates the eye closure rate. In addition, the determination unit 14 evaluates the calculated eye closure rate. More specifically, the determination unit 14 determines whether the calculated eye closure rate is greater than a first threshold value, and if it is determined that the eye closure rate is less than the first threshold value, determines whether the eye closure rate is greater than a second threshold value.

[0127] In step S106, the determination unit 14 uses the evaluation results in steps S102 to S105 to determine the thinking state of the user 100. Specifically, when at least one of the indicators in the determinations in steps S102 to S105 is greater than the first threshold, the determination unit 14 determines that the user 100 is in a divergent thinking state. In addition, when all the indicators in the determinations in steps S102 to S105 are less than the first threshold and at least one indicator is greater than the second threshold, the determination unit 14 determines that the user 100 is in a convergent thinking state.

[0128] In step S107, the determination unit 14 stores the determination result in step S106 in the storage unit 13. In addition, the output unit 15 outputs the determination result in step S106 via the display device 20.

[0129] In addition, regarding the above steps S102 to S105, at least one step only needs to be executed. If there is an unexecuted step among steps S102 to S105, "steps S102 to S105" in step S106 is read as "steps other than the unexecuted steps among steps S102 to S105".

[0130] Figure 5 1 is an explanatory diagram showing an example of a mental state determined by the biological measurement device 1 in the present embodiment.

[0131] use Figure 5 To illustrate Figure 4 Here, a specific example of the determination performed by the determination unit 14 is shown in the flowchart of Figure 5 The time intervals T1, T2 and T3 are shown. Figure 5 In the description of , all the indicators that are the determination targets of steps S102 to S105 are set to be equal to or greater than the second threshold value.

[0132] In the case of time interval T1, the determination unit 14 determines in step S102 that the amount of change in the position of the face is greater than the first threshold, in step S103 that the amount of change in the orientation of the face is greater than the first threshold, in step S104 that the amount of change in the orientation of the line of sight is greater than the first threshold, and in step S105 that the rate of eye closure is greater than the first threshold. Then, in step S106, the determination unit 14 determines that the thinking state of the user 100 is a divergent thinking state based on the determination results of steps S102 to S105, in other words, based on the fact that the indicator is greater than the first threshold in at least one determination of steps S102 to S105.

[0133] In the case of time interval T2, the determination unit 14 determines in step S102 that the amount of change in the position of the face is less than the first threshold, in step S103 that the amount of change in the orientation of the face is less than the first threshold, in step S104 that the amount of change in the orientation of the line of sight is greater than the first threshold, and in step S105 that the rate of eye closure is less than the first threshold. Then, in step S106, the determination unit 14 determines that the thinking state of the user 100 is a divergent thinking state based on the determination results of steps S102 to S105, in other words, based on the fact that the indicator is greater than the first threshold in at least one determination of steps S102 to S105, and records the result in step S107.

[0134] In this way, when it is determined that at least one indicator is equal to or greater than the first threshold value, the biological measurement device 1 determines that the person is in a divergent thinking state.

[0135] In addition, when comparing the time interval T1 with the time interval T2, it can be said that the degree of the divergent thinking state of the user 100 is higher in the time interval T1 in which more indicators are determined to be above the first threshold. In this way, the determination unit 14 can determine the degree of the divergent thinking state based on the number of indicators determined to be above the first threshold, and store it in the storage unit 13. In addition, the output unit 15 can also output the stored degree of the divergent thinking state.

[0136] In addition, together with or in place of the above, the determination unit 14 can evaluate that the degree of the divergent thinking state increases as the difference between the index and the first threshold increases. In addition, the determination unit 14 can evaluate that the degree of the divergent thinking state increases as the time determined as the divergent thinking state increases.

[0137] In the case of time interval T3, the determination unit 14 determines in step S102 that the change amount of the face position is less than the first threshold, in step S103 that the change amount of the face orientation is less than the first threshold, in step S104 that the change amount of the sight line orientation is less than the first threshold, and in step S105 that the eye closing rate is less than the first threshold. Then, in step S106, the determination unit 14 determines that the thinking state of the user 100 is a convergent thinking state based on the determination results of steps S102 to S105, in other words, based on the fact that the index is less than the first threshold in all the determinations of steps S102 to S105.

[0138] In this way, when it is evaluated that all the indicators are smaller than the first threshold value, the biological measurement device 1 determines that the biological measurement device is in the convergent thinking state.

[0139] Hereinafter, the display of information related to the status of the user 100 is described. The information presented includes Figure 4 The determination result stored in the storage unit 13 in step S107 is presented to the user 100 using the display device 20 .

[0140] The display device 20 presents the thought state of the user 100 determined by the biological measurement device 1 , in other words, visualizes (so-called visualization) the thought state of the user 100 .

[0141] The diagram (e.g., graph) showing the determination result presented by the display device 20 may be a diagram showing the thinking state of the user 100 (i.e., the convergent thinking state or the divergent thinking state) in time series ( Fig.6D ), or a graph showing a ratio or a proportion of the thinking state of the user 100 ( Figure 6C or Fig. 6E ). In addition, only the thinking state of the user 100 may be shown ( Fig. 6A , Figure 6C or Fig.6D ), the evaluation value of the judgment may also be displayed together with the thinking state of the user 100 ( Figure 6B or Fig. 6E ). The evaluation value may be a value indicating the degree of appropriateness of the thinking state of the user 100, a value indicating the degree of stability of thinking based on the time that a certain thinking state can be maintained continuously, or a value indicating the proportion of the time in a certain thinking state in the task time. When the evaluation value is displayed, it may be displayed as a score in the form of x points for a convergent thinking state or y points for a divergent thinking state ( Figure 6B ). As a display method of the evaluation value, an illustration that abstracts the mental state can also be used ( Fig. 6F ). In addition, a more general expression can be used to express the convergent thinking state as "concentration" and the divergent thinking state as "creativity". The judgment result can be the result of evaluating the entire business time, or the result of judging the state of a specific time period.

[0142] After the determination unit 14 determines the thought state of the user 100, when the determined thought state is scored, the score may be determined based on the amount of change to the biological information in the execution state, based on the biological information of the user 100 in the calm state obtained in advance. In addition, the score may be calculated by comparing the average value of the biological information measured in the execution state of the user 100 with the value of the average feature quantity of data of multiple people obtained in advance through experiments. Furthermore, the score may be calculated by comparing the value of the amount of change to the biological information in the execution state calculated based on the biological information of the user 100 in the calm state obtained in advance with the value of the average change of feature quantity data of multiple people obtained in advance through experiments.

[0143] FIG. 6A to FIG. 6G An example of displaying the determination result in this embodiment is shown.

[0144] Fig. 6A It is an explanatory diagram of a first example of display of determination results according to the present embodiment. Fig. 6A This is an example of using language to indicate the tendency of the user 100's state of mind during business. Fig. 6A The "divergent thinking" in is the abbreviation of "divergent thinking state". In addition, the abbreviation of "convergent thinking state" can be "convergent thinking". The same is true in the following. In addition, Fig. 6A The display is not limited to the display on the screen, and the sound prompt can also be used.

[0145] Figure 6B It is an explanatory diagram of a second example of display of determination results according to the present embodiment. Figure 6BThis is an example of displaying the result of the determination of the state of mind of the user 100 during the work as a score. The displayed score may be a score only for a specific state of mind or a score for multiple states of mind. The upper limit of the score may be 100 points or 5 points.

[0146] Figure 6C It is an explanatory diagram of a third example of display of determination results according to the present embodiment. Figure 6C This is an example of using a pie chart to represent the ratio of the thinking state (ie, convergent thinking state or divergent thinking state) of the user 100 during a task.

[0147] Fig.6D It is an explanatory diagram of a fourth example of display of determination results according to the present embodiment. Fig.6D This is an example of representing the thinking state of the user 100 during the business in time series. The horizontal axis represents time, and the vertical axis represents the thinking state of the user 100 (i.e., convergent thinking state or divergent thinking state). In addition, the time shown on the horizontal axis can represent a time point or a few minutes.

[0148] Fig. 6E It is an explanatory diagram of a fifth example of the display of the determination result according to the present embodiment. Fig. 6E This is an example of representing the score of the divergent thinking state (also called divergent thinking degree) of the user 100 during the business in time series. The horizontal axis represents time, and the vertical axis represents the score of the divergent thinking state of the user 100. In addition, the vertical axis can be set to the score of the convergent thinking state. In addition, the results of multiple thinking states can be integrated and displayed.

[0149] Fig. 6F It is an explanatory diagram of a sixth example of display of determination results according to the present embodiment. Fig. 6F This is an example of showing the state of mind of the user 100 during work as an abstract illustration. For example, when the user 100 is in a calm state such as when starting work, an ellipse with a focus out of focus is displayed (see Fig. 6F (a)).

[0150] When the thinking state of the user 100 is determined by the biometric measurement device 1 to be a convergent thinking state, the convergent thinking state is displayed as a circle or a sphere with a fixed focus (see Fig. 6F (b)). It can be said that this is an illustration of an abstract situation in which a person’s thoughts are summarized or an answer is obtained through logical thinking.

[0151] In addition, the size of the displayed circle may increase as the degree of divergent thinking state of the user 100 increases. The expansion of thinking is manifested as an increase in the size of the circle.

[0152] When the biological measurement device 1 determines that the thinking state of the user 100 is an improvement in the divergent thinking state (in other words, the degree of the divergent thinking state is increasing), or determines that the thinking state of the user 100 is a divergent thinking state, a graphic of a plurality of partially overlapping circles is displayed as a divergent thinking state (see Fig. 6F (c)). It can be said that this is an illustration that abstracts the situation where multiple bubbles are generated and diverged, or a large number of ideas are generated and emerge through abstract thinking. In addition, the tone or chroma of the color can also be used to represent the state of thinking, such as using cool colors to represent the illustration of the convergent state of thinking, or using warm colors to represent the illustration of the divergent state of thinking. The shape or color of the illustration is not limited to the content described here.

[0153] Figure 6G It is an explanatory diagram of a seventh example of display of determination results according to the present embodiment. Figure 6G 1 is a diagram showing an example of presentation of information other than the determination result. 2 is a diagram showing a display image G1 displayed on the display device 20 as an example of presentation information. The display image G1 includes a graph G11, an input field G12, and message display fields G13 and G14.

[0154] Figure G11 and Figure 6C The diagrams shown are the same.

[0155] The input bar G12 is a bar where the user selects and inputs whether to save the determination result. In the input bar G12, a message "Do you want to save the result?", a "Yes" button, and a "No" button are displayed. The "Yes" button is a button for the user to choose to save the determination result. The "No" button is a button for the user to choose not to save the determination result. When the user selects the "Yes" button, the display device 20 accepts the operation and changes the color of the "Yes" button. When the user selects the "No" button, the display device 20 accepts the operation and changes the color of the "No" button. In this way, the user can easily confirm which button is selected.

[0156] When the user selects the “Yes” button, the display device 20 accepts the operation, and determines whether the result can be saved or not, and changes to “Allow”, saving the determination result about the thought state of the user 100. When the user selects the “No” button, the display device 20 accepts the operation, and determines whether the result can be saved or not, and does not save the determination result about the thought state of the user 100.

[0157] The message display field G13 includes a QR code (registered trademark). In addition, the message display field G13 includes a message for notifying the user that the user can confirm the result of this time on his / her own portable terminal by reading the QR code. Figure 6GIn the example, the message display field G13 includes a message such as "If you want to confirm the result on your smartphone, please read the QR code here" and a QR code. In addition, although the QR code is shown in the example, the user's email address can also be input. In addition, when a card reader is installed on the display device 20 and a card such as an employee ID card is read by the card reader, the determination result can be sent to the user 100 and his supervisor.

[0158] The message display field G14 includes messages that take into account the biological information of the user 100 during the business hours, the weather of the day, the time period, etc. Figure 6G In the example, it is assumed that the user 100 is in a specific state of mind for a long time during the early morning period, and a positive message “Your state of mind is stable. If you can think deeply, you should be able to achieve better results. I wish you a good day.” is displayed.

[0159] In this embodiment, the living body measurement device 1 can control the lighting device so that the color temperature or illuminance of the lighting light illuminating the vicinity of the user 100 is changed according to the determination result. For example, when the user 100 inputs in advance to the living body measurement device 1 that he wants to perform a task in a certain state of mind and start the task under a certain lighting state, the illuminance or color temperature of the room can be changed at regular intervals during the task according to the determination result of the state of mind of the user 100. In addition, when the user 100 feels comfortable with the lighting at a certain time, the user 100 can store the lighting setting from the operation terminal connected to the living body measurement device 1. The next time when the user 100 wants to be in the same state of mind, the task can be started from the stored lighting setting.

[0160] Figure 7 This is a flowchart showing an example of a process of controlling lighting according to a determination result in this embodiment.

[0161] In step S201 , the acquisition unit 11 acquires information on the target mental state of the user 100 .

[0162] The processing from step S202 to step S203 is the same as Figure 3 The processing of steps S101 to S107 is the same as that of steps S101 to S107.

[0163] In step S204, the determination unit 14 determines whether the target mental state acquired by the acquisition unit 11 in step S201 is consistent with the mental state determined in step S203. If it is determined that the two states are consistent (yes in step S204), the process proceeds to step S206, and if it is determined that the two states are inconsistent (no in step S204), the process proceeds to step S205.

[0164] In step S205, the output unit 15 controls at least one of the illuminance and the color temperature of the lighting. The details of the control will be described later.

[0165] In step S206, the determination unit 14 determines whether to end Figure 7 For example, the determination unit 14 may determine whether the acquisition of new biological information (step S202) is successful, and if it is determined to be unsuccessful, it is determined to be finished. Figure 7 In addition, the determination unit 14 can determine whether a preset time has passed, and when it is determined that the above time has passed, it is determined that the process is finished. Figure 7 The series of determination processes shown in the figure. By ending the determination process, the amount of processing by the processor can be reduced.

[0166] Figure 8 This is a diagram showing the relationship between the illuminance and color temperature of lighting suitable for guiding the user 100 to a certain state of thinking (i.e., convergent state of thinking or divergent state of thinking). This relationship between the illuminance and color temperature of lighting is described in Non-Patent Document 6.

[0167] Therefore, as an example, the biological measurement device 1 can guide the user 100 to a divergent thinking state or a convergent thinking state by controlling the lighting device.

[0168] (1) Control to guide the user 100 to a divergent thinking state

[0169] For example, Non-Patent Document 6 describes the use of lighting with high illuminance and low color temperature as a condition suitable for work such as brainstorming, that is, a condition suitable for a divergent thinking state.

[0170] On the other hand, when the user 100 is intended to be guided to a divergent thinking state, the thinking state of the user 100 is different from the divergent thinking state and is estimated to be a convergent thinking state or a neutral state.

[0171] At this time, if the lighting device is controlled to lower the color temperature of the lighting light and increase the illuminance, the biological measurement device 1 may be able to guide the user 100 to a divergent thinking state (see Figure 8 ).

[0172] Therefore, when it is determined that at least one of the change amount of the position of the face of the user 100, the change amount of the face orientation, the change amount of the sight line orientation, and the eye closure rate is less than the first threshold, and when it is determined that at least one of the above-mentioned indicators is greater than the second threshold (that is, when it is determined to be a convergent thinking state), if the lighting device is controlled to lower the color temperature of the lighting light and increase the illuminance, the biometric measurement device 1 may be able to guide the user 100 to a divergent thinking state.

[0173] In addition, when it is determined that at least one of the change amount of the position of the face of the user 100, the change amount of the face orientation, the change amount of the sight line orientation, and the eye closure rate is less than the second threshold value (that is, when it is determined to be a neutral state), if the lighting device is controlled to lower the color temperature of the lighting light and increase the illuminance, the biometric measurement device 1 may be able to guide the user into a divergent thinking state.

[0174] (2) Control of guiding the user 100 to a convergent thinking state

[0175] For example, Non-Patent Document 6 describes the use of lighting with a high color temperature and a high illuminance as a condition suitable for a convergent thinking state.

[0176] On the other hand, when the user 100 is intended to be guided to a convergent thinking state, the thinking state of the user 100 is different from the convergent thinking state and is estimated to be a divergent thinking state or a neutral state.

[0177] At this time, if the lighting device is controlled to increase the color temperature of the lighting light and the illuminance, the biological measurement device 1 may be able to guide the user 100 to a convergent thinking state (see Figure 8 ).

[0178] Therefore, when the biometric measurement device 1 determines that at least one of the change amount of the face position, the change amount of the face orientation, the change amount of the line of sight orientation, and the eye closure rate of the user 100 is greater than the first threshold value (that is, when determined to be in a divergent thinking state), if the biometric measurement device 1 controls the lighting device to increase the color temperature of the lighting light and the illuminance, it is possible that the user 100 can be guided to a convergent thinking state.

[0179] Furthermore, when the biometric measurement device 1 determines that at least one of the amount of change in the position of the face, the amount of change in the orientation of the face, the amount of change in the orientation of the line of sight, and the rate of eye closure of the user 100 is less than the second threshold value (i.e., when the user 100 is determined to be in a neutral state), if the biometric measurement device 1 controls the lighting device to increase the color temperature of the lighting light and the illuminance, it is possible that the user 100 can be guided to a convergent thinking state (see Figure 8 ).

[0180] In addition, the biometric measurement device 1 can also determine the state of mind according to the type of problem being solved. In this case, the biometric measurement device 1 inputs information on the status of the user 100 solving the problem, calculates the average value based on the biometric information in the time domain required for each problem, and determines the state of mind when each problem is solved.

[0181] exist Figure 4In the determination processing shown in the above, threshold information for determining the value to which the threshold is set may be stored in a storage device, and the threshold set using the threshold information obtained from the storage device may be used.

[0182] The threshold information may be stored in association with an identification ID for identifying the individual of the user 100 . In this case, the thought state can be determined using a threshold suitable for each individual of the user 100 , thereby improving the determination accuracy.

[0183] The threshold information corresponding to the identification ID may also be a value that takes into account the amount of change in the position of the face, the amount of change in the orientation of the face, the amount of change in the orientation of the sight line, and the rate of eye closure in the calm state of the user 100. The amount of change in the position of the face, the amount of change in the orientation of the face, the amount of change in the orientation of the sight line, and the rate of eye closure in the calm state of each user 100 may be stored in the storage device in correspondence with the identification ID, and the threshold information of each user 100 may be generated. The calm state may be a neutral state (in other words, a neutral state from the perspective of thinking) induced by having the user 100 listen to white noise or say meaningless words, or a state in which the task is not being solved. The threshold information may be updated based on the result of the mental state determination, etc.

[0184] Fig. 9 This is a flowchart showing an example of processing related to updating of threshold information according to the present embodiment. Fig. 9 The process of making a determination using the threshold information associated with the identification information of the user 100 is shown.

[0185] In step S301, the determination unit 14 obtains identification information of the user 100. The identification information is information that can uniquely identify the user 100. The identification information may be characters or symbols input by the user 100 using an input device such as a keyboard, or may be a facial image of the user 100 captured by the camera 10.

[0186] In step S302 , the determination unit 14 acquires threshold information corresponding to the identification information acquired in step S301 from the storage device.

[0187] In step S303, the determination unit 14 determines a threshold value based on the threshold value information obtained in step S302. The determined threshold value is a first threshold value corresponding to at least one of the amount of change in the position of the face, the amount of change in the orientation of the face, the amount of change in the orientation of the sight line, and the rate of eye closure. In addition, the determined threshold value may be a second threshold value corresponding to at least one of the amount of change in the position of the face, the amount of change in the orientation of the face, the amount of change in the orientation of the sight line, and the rate of eye closure.

[0188] In step S304, the determination unit 14 performs determination processing based on the biometric information of the user 100. The determination processing performed in step S304 is equivalent to Figure 4 Processing shown.

[0189] In step S305, the determination unit 14 obtains subjective evaluation information related to the state of the user 100. The subjective evaluation information is the thinking state of the user 100 evaluated by the user 100 himself. For example, the determination unit 14 uses a GUI (Graphical User Interface) for accepting the answer of the user 100 to prompt a message including "Please conduct a self-evaluation of the state in the task." to inquire about the thinking state of the user 100 during the task, and options for the state of the user 100 when performing the task (specifically, "divergent thinking state" and "convergent thinking state"). In the case where the user 100 answers "divergent thinking state" using the GUI, the information that the user 100 is in a divergent thinking state when performing the task is obtained as the subjective evaluation information. In addition, the information obtained in step S305 is not limited to the subjective evaluation evaluated by the subject himself, and the judgment result determined by a method different from the judgment of the thinking state based on the heartbeat can also be obtained as the evaluation information.

[0190] In step S306, the determination unit 14 determines whether the state of the user 100 determined in step S304 is consistent with the state of the user 100 indicated by the subjective evaluation information obtained in step S305. For example, when the state of the user 100 determined based on the biometric information is a convergent thinking state, and the state of the user 100 indicated by the subjective evaluation information is a convergent thinking state, it is determined that the two states are consistent. On the other hand, for example, when the state of the user 100 determined based on the biometric information is a convergent thinking state, and the state of the user 100 indicated by the subjective evaluation information is a divergent thinking state, it is determined that the two states are inconsistent.

[0191] If the two states are consistent (yes in step S306), end Fig. 9 In other cases (No in step S306), the process proceeds to step S307.

[0192] In step S307, the determination unit 14 updates the threshold information corresponding to the user 100 (i.e., the identification information) based on the determined state of the user 100 and the state of the user 100 indicated by the subjective evaluation information. For example, when the state of the user 100 determined based on the biometric information is a convergent thinking state, and the state of the user 100 indicated by the subjective evaluation information is a divergent thinking state, the threshold information is updated so that the first threshold of the amount of change in the position of the face indicated by the threshold information is lowered. This is because by lowering the first threshold of the amount of change in the position of the face, the range of the amount of change in the position of the face determined to be above the first threshold (step S102) is expanded, which helps to determine the state of the user 100 as a divergent thinking state.

[0193] As described above, the biometrics measuring device 1 can improve the accuracy of the determination of the mental state by updating the threshold information using the result of the subjective evaluation.

[0194] In addition, in the above-mentioned embodiments, each component is composed of dedicated hardware, but it can also be realized by executing a software program suitable for each component. Each component can be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the information processing device and the like of the above-mentioned embodiments is the following program.

[0195] That is, the program is a program that uses a processor to cause a computer to execute an information processing method, the information processing method comprising: obtaining at least one indicator selected from an indicator group including a change in the position of the user's face, a change in the orientation of the face, a change in the orientation of the user's line of sight, and a rate of eye closure of the user's eyes; and determining whether the user is in a divergent thinking state based on the at least one indicator obtained.

[0196] Furthermore, any one of the CPU, the processor, and the storage device that performs the mental state determination process may be a configuration included in a server that communicates with a sensor that detects biological information of a user via a network.

[0197] The above describes one or more information processing methods based on the embodiments, but the present disclosure is not limited to the embodiments. As long as it does not deviate from the main purpose of the present disclosure, various modifications that a person skilled in the art can think of and apply to the embodiments, and the methods constructed by combining the constituent elements in different embodiments can also be included in the scope of one or more embodiments.

[0198] (other)

[0199] Modifications of the embodiments of the present disclosure may be as follows.

[0200] (Item 1)

[0201] A method executed by a processor, wherein the method comprises:

[0202] Get the value V included in the first index FP , the value of the second index V FD , the value of the third index V ED and the fourth index V EC One or more values ​​of ;

[0203] The first index indicates a change in the position of the user's face, the second index indicates a change in the orientation of the user's face, the third index indicates a change in the direction of the user's line of sight, and the fourth index indicates a proportion of the user's eyes being closed; and

[0204] Based on the obtained one or more values, it is determined whether the user is in a divergent thinking state.

[0205] (Item 2)

[0206] The method described in item 1, wherein the method further comprises:

[0207] Acquire a plurality of images I1(1) to I1(n), wherein the plurality of images I1(1) to I1(n) are a plurality of images obtained by photographing the face of the user at a first predetermined interval during a first period when the user is not performing a task, i=1 to n, and the image I1(i) is photographed at time t1i,

[0208] Based on the image I1(i), the value fp1(i) indicating the position of the face of the user at the time t1i, the value fd1(i) indicating the orientation of the face of the user at the time t1i, the value ed1(i) indicating the direction of the gaze of the user at the time t1i, and the value ec1(i) indicating whether the user has his eyes closed at the time t1i are determined, thereby:

[0209] Determine fp1(1) to fp1(n),

[0210] fd1(1)~fd1(n),

[0211] ed1(1)~ed1(n), and

[0212] ec1(1)~ec1(n),

[0213] When the user closes both eyes at time t1i, ec1(i)=1,

[0214] When the user does not close his eyes at the time t1i, ec1(i)=0,

[0215] FP1 is determined based on fp1(1) to fp1(n).

[0216] FD1 is determined based on fd1(1) to fd1(n).

[0217] ED1 is determined based on ed1(1) to ed1(n).

[0218] EC1 is determined based on ec1(1) to ec1(n).

[0219] Acquire a plurality of images I2(1) to I2(m), wherein the plurality of images I2(1) to I2(m) are a plurality of images obtained by photographing the face of the user at a second predetermined interval during a second period in which the user is performing the task, j=1 to m, and the image I2(j) is photographed at time t2j,

[0220] Based on the image I2(j), the value fp2(j) representing the position of the face of the user at the moment t2j, the value fd2(j) representing the orientation of the face of the user at the moment t2j, the value ed2(j) representing the direction of the line of sight of the user at the moment t2j, and the value ec2(j) representing whether the user has his eyes closed at the moment t2j are determined, thereby,

[0221] Determine fp2(1) to fp2(m),

[0222] fd2(1)~fd2(m),

[0223] ed2(1)~ed2(m) and

[0224] ec2(1)~ec2(m),

[0225] When the user closes both eyes at time t2j, ec2(j)=1,

[0226] When the user does not close his eyes at time t2j, ec2(j)=0,

[0227] FP2 is determined based on fp2(1) to fp2(m).

[0228] FD2 is determined based on fd2(1) to fd2(m).

[0229] ED2 is determined based on ed2(1) to ed2(m).

[0230] EC2 is determined based on the ec2(1) to ec2(m).

[0231] VFP = (FP2 - FP1),

[0232] V FD = (FD2 - FD1),

[0233] V ED = (ED2 - ED1),

[0234] V EC = (EC2 - EC1).

[0235] The first period may be less than the second period. It may be n < m.

[0236] When n < m, the amount of processing executed by the processor is less than that when n = m. When processing is executed under the condition of n < m, the memory consumption is less than that when processing is executed under the condition of n = m.

[0237] The time required to identify changes in the value of the position of the user's face, changes in the value of the orientation of the user's face, changes in the value of the user's line of sight direction, and changes in the value indicating whether the user has both eyes closed during the first period in which the user is not performing a task

[0238] Only needs to be shorter than the time required to identify changes in the value of the position of the user's face, changes in the value of the orientation of the user's face, changes in the value of the user's line of sight direction, and changes in the value indicating whether the user has both eyes closed during the second period in which the user is performing a task.

[0239] The first period may be 1 minute, n may be the number of images taken during the first period, the first specified interval may be 0.1 second, and n may be 600. It may be (1 / 20) second ≤ the first specified interval ≤ (1 / 10) second.

[0240] The second period may be 3 minutes, m may be the number of images taken during the second period, the second specified interval may be 0.1 second, and m may be 1800. It may be (1 / 20) second ≤ the second specified interval ≤ (1 / 10) second.

[0241] (Item 3)

[0242] The method according to item 2, wherein,

[0243] The FP1 is the standard deviation of the fp1(1) to the fp1(n),

[0244] The FD1 is the standard deviation of the fd1(1) to the fd1(n),

[0245] The ED1 is the standard deviation of ed1(1) to ed1(n),

[0246] The EC1 is (ec1(1)+~+ec1(n)) / n,

[0247] The FP2 is the standard deviation of fp2(1) to fp2(m),

[0248] FD2 is the standard deviation of fd2(1) to fd2(m),

[0249] ED2 is the standard deviation of ed2(1) to ed2(m),

[0250] The EC2 is (ec2(1)+~+ec2(m)) / m.

[0251] (Item 4)

[0252] The method described in item 2, wherein:

[0253] FP1 is the variance of fp1(1) to fp1(n),

[0254] FD1 is the variance of fd1(1) to fd1(n),

[0255] ED1 is the variance of ed1(1) to ed1(n),

[0256] The EC1 is (ec1(1)+~+ec1(n)) / n,

[0257] FP2 is the variance of fp2(1) to fp2(n),

[0258] FD2 is the variance of fd2(1) to fd2(n),

[0259] ED2 is the variance of ed2(1) to ed2(m),

[0260] The EC2 is (ec2(1)+~+ec2(m)) / m.

[0261] (Item 5)

[0262] The method described in item 2, wherein:

[0263] The plurality of first thresholds include a threshold (11) for the first indicator, a threshold (12) for the second indicator, a threshold (13) for the third indicator, and a threshold (14) for the fourth indicator.

[0264] The one or more values ​​obtained include a value corresponding to the j-th index,

[0265] If the value is greater than or equal to the threshold value (1j), the processor determines that the user is in a divergent thinking state.

[0266] j=1, j=2, j=3, or j=4.

[0267] (Item 6)

[0268] The method described in item 2, wherein:

[0269] The plurality of first thresholds include a threshold (11) for the first indicator, a threshold (12) for the second indicator, a threshold (13) for the third indicator, and a threshold (14) for the fourth indicator.

[0270] The plurality of second thresholds include a threshold (21) for the first indicator, a threshold (22) for the second indicator, a threshold (23) for the third indicator, and a threshold (24) for the fourth indicator.

[0271] The one or more values ​​obtained are respectively included in the plurality of first thresholds and are smaller than the thresholds respectively corresponding to the one or more values ​​obtained,

[0272] The value included in the one or more values ​​obtained is a value corresponding to the k-th index,

[0273] If the value is greater than the threshold (2k), the processor determines that the user is in a convergent thinking state.

[0274] k=1, k=2, k=3, or k=4.

[0275] The threshold value (11) is greater than the threshold value (21),

[0276] The threshold value (12) is greater than the threshold value (22),

[0277] The threshold value (13) is greater than the threshold value (23),

[0278] The threshold value (14) is greater than the threshold value (24).

[0279] “The one or more values ​​obtained are respectively included in the multiple first thresholds and are smaller than the thresholds respectively corresponding to the one or more values ​​obtained” can be interpreted as the following situations (a) to (d).

[0280] (a) When one or more acquired values ​​are the first value, (first value)<(threshold value corresponding to the first value).

[0281] The first value is any one of FP1, FD1, ED1, and EC1.

[0282] In the case where the first value is FP1, (the threshold corresponding to the first value) is the threshold value (11).

[0283] In the case where the first value is FD1, (the threshold corresponding to the first value) is the threshold (12).

[0284] In the case where the first value is ED1, (the threshold corresponding to the first value) is the threshold (13).

[0285] In the case where the first value is EC1, (the threshold corresponding to the first value) is the threshold (14).

[0286] (b) When the acquired value or values ​​are the first value and the second value, (first value)<(threshold value corresponding to the first value) and (second value)<(threshold value corresponding to the second value).

[0287] The first value is one of FP1, FD1, ED1, and EC1. The second value is one of FP1, FD1, ED1, and EC1. However, the first value and the second value are different.

[0288] In the case where the first value is FP1, (the threshold corresponding to the first value) is the threshold value (11).

[0289] In the case where the first value is FD1, (the threshold corresponding to the first value) is the threshold (12).

[0290] In the case where the first value is ED1, (the threshold corresponding to the first value) is the threshold (13).

[0291] In the case where the first value is EC1, (the threshold corresponding to the first value) is the threshold (14).

[0292] In the case where the second value is FP1, (the threshold corresponding to the second value) is the threshold (11).

[0293] In the case where the second value is FD1, (the threshold corresponding to the second value) is the threshold (12).

[0294] In the case where the second value is ED1, (the threshold value corresponding to the second value) is the threshold value (13).

[0295] In the case where the second value is EC1, (the threshold corresponding to the second value) is the threshold (14).

[0296] (c) When the one or more values ​​obtained are a first value, a second value, and a third value, (first value) < (threshold value corresponding to the first value) and (second value) < (threshold value corresponding to the second value) and (third value) < (threshold value corresponding to the third value).

[0297] The first value is one of FP1, FD1, ED1, and EC1. The second value is one of FP1, FD1, ED1, and EC1. The third value is one of FP1, FD1, ED1, and EC1. However, the first value, the second value, and the third value are different from each other.

[0298] In the case where the first value is FP1, (the threshold corresponding to the first value) is the threshold value (11).

[0299] In the case where the first value is FD1, (the threshold corresponding to the first value) is the threshold (12).

[0300] In the case where the first value is ED1, (the threshold corresponding to the first value) is the threshold (13).

[0301] In the case where the first value is EC1, (the threshold corresponding to the first value) is the threshold (14).

[0302] In the case where the second value is FP1, (the threshold corresponding to the second value) is the threshold (11).

[0303] In the case where the second value is FD1, (the threshold corresponding to the second value) is the threshold (12).

[0304] In the case where the second value is ED1, (the threshold value corresponding to the second value) is the threshold value (13).

[0305] In the case where the second value is EC1, (the threshold corresponding to the second value) is the threshold (14).

[0306] In the case where the third value is FP1, (the threshold value corresponding to the third value) is the threshold value (11).

[0307] In the case where the third value is FD1, (the threshold value corresponding to the third value) is the threshold value (12).

[0308] In the case where the third value is ED1, (the threshold corresponding to the third value) is the threshold (13).

[0309] In the case where the third value is EC1, (the threshold corresponding to the third value) is the threshold (14).

[0310] (d) When the obtained value or values ​​are FP1, FD1, ED1, EC1,

[0311] FP1<threshold (11), FD1<threshold (12), ED1<threshold (13), EC1<threshold (14).

[0312] INDUSTRIAL APPLICABILITY The present disclosure can be used in a device for sensing a state of a person.

[0313] Description of Reference Numerals

[0314] 1Biological measurement device

[0315] 10. Camera

[0316] 11 Acquisition Department

[0317] 12Analysis Department

[0318] 13 Storage

[0319] 14 Judgment Department

[0320] 15 Output Department

[0321] 20Display device

[0322] 40, 41, 42, 43 first threshold

[0323] 100 users

[0324] G1 Display Image

[0325] G11 Chart

[0326] G12 Input Bar

[0327] G13, G14 message display bar

[0328] T1, T2, T3 time interval

Claims

1. An information processing method, wherein: include: Using the processor, obtaining at least one indicator selected from an indicator group including a change amount of a position of a user's face, a change amount of an orientation of the face, a change amount of an orientation of a line of sight of the user, and a closing rate of an eye of the user; as well as Whether the user is in a divergent thinking state is determined based on the at least one indicator obtained.

2. The information processing method according to claim 1, wherein: The determination includes: comparing the at least one indicator with a first threshold value corresponding to the at least one indicator; and When it is determined that the at least one indicator is equal to or greater than the first threshold, it is determined that the user is in the divergent thinking state.

3. The information processing method according to claim 2, wherein: The determination includes: comparing the at least one indicator with a second threshold value corresponding to the at least one indicator and different from the first threshold value; and When it is determined that the at least one indicator is smaller than the first threshold and equal to or greater than the second threshold, it is determined that the user is in a convergent thinking state.

4. The information processing method according to claim 2 or 3, wherein: The determination includes: For each of a plurality of indicators selected from the indicator group, it is determined whether the indicator is equal to or greater than the first threshold value corresponding to the indicator.

5. The information processing method according to claim 4, wherein: The determination includes: When it is determined that the change in the orientation of the face is greater than the first threshold corresponding to the change in the orientation of the face and the change in the orientation of the line of sight is greater than the first threshold corresponding to the change in the orientation of the line of sight, it is determined that the user is in the divergent thinking state.

6. The information processing method according to claim 4, wherein: The determination includes: The degree of the divergent thinking state of the user in the unit period is determined based on the number of indicators corresponding to the indicator that are greater than or equal to the first threshold value among the plurality of indicators acquired in the unit period.

7. The information processing method according to claim 3, wherein: The determination includes: Using the at least one indicator acquired in each of a plurality of unit periods, determining whether the user is in a divergent thinking state or a convergent thinking state in each of the plurality of unit periods; generating a graph indicating a ratio of a period during which the user is determined to be in the convergent thinking state to a period during which the user is determined to be in the divergent thinking state; and The chart is displayed.

8. The information processing method according to claim 2, wherein: The information processing method further includes: obtaining identification information for identifying the user; and obtaining threshold information corresponding to the identification information, The first threshold is determined based on the threshold information.

9. The information processing method according to claim 8, wherein: The information processing method further includes: acquiring evaluation information for evaluating the user's mental state by a method different from the determination process; and The threshold information is updated based on the determined result of the user's mental state and the evaluation information.

10. The information processing method according to claim 3, wherein: include: When it is determined that the at least one indicator is smaller than the first threshold and greater than the second threshold, the lighting device illuminating the vicinity of the user is controlled to lower the color temperature of the lighting light and increase the illuminance.

11. The information processing method according to claim 2, wherein: include: When it is determined that the at least one indicator is equal to or greater than the first threshold, control is performed on the lighting device that illuminates the vicinity of the user to increase the color temperature of the lighting light and thereby increase the illuminance.

12. The information processing method according to claim 3, wherein: include: When it is determined that the at least one indicator is less than the second threshold, the lighting device illuminating the vicinity of the user is (i) lowering the color temperature of the lighting and increasing the control of the illuminance, or (ii) Increasing the color temperature of the lighting and improving the control of illumination.

13. The information processing method according to claim 1, wherein: The acquisition includes: obtaining an image generated by a camera that photographs the user; and The at least one index is acquired by performing an analysis process on the acquired image.

14. An information processing method, wherein: include: Using the processor, Obtaining target information indicating that the user is in a state set as a target, i.e., a target state; obtaining at least one indicator selected from an indicator group including a change amount of the position of the user's face, a change amount of the orientation of the face, a change amount of the orientation of the user's line of sight, and a closing rate of the user's eyes; Determining the user's mental state based on the at least one indicator; comparing the target state indicated by the target information with the determined thinking state of the user; as well as When the result of the comparison is that the target state is different from the determined mental state of the user, control is performed to change the characteristics of the illumination light irradiated by the illumination device that illuminates the vicinity of the user.

15. An information processing system, wherein: have: A camera device for photographing a user and generating an image; Processing circuit; Storage device; as well as display device, the processing circuit, By performing analysis processing on the image generated by the camera, at least one indicator selected from an indicator group including a change in the position of the user's face, a change in the orientation of the face, a change in the orientation of the user's line of sight, and a closing rate of the user's eyes is obtained. acquiring threshold information from the storage device, the threshold information indicating a first threshold corresponding to the at least one indicator, Based on the result of determining whether the at least one indicator in the unit period is equal to or greater than the first threshold value, a determination process of determining whether the user is in a divergent thinking state in the unit period is performed, An image corresponding to the result of the determination process is displayed on the display device.

Images