Processing method, processing device, and program

Abstract

To stably determine a mental state of a user by restraining the influence of individual differences.SOLUTION: A processing method executed by a computer includes: acquiring biological information of a user; acquiring information related to at least one of an acceleration and an angular velocity; inferring biological information of the user at a current time, based on the information in the past, the biological information in the past, and the information at the current time, which are accumulated in a storage device; and determining a mental state of the user at the current time, based on the acquired biological information of the user at the current time and the inferred biological information of the user at the current time.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] The present disclosure relates to a processing method, a processing device, and a program. [Background technology]

[0002] Mental concentration is extremely important in daily work. Distracted work without attention not only reduces efficiency but also has the risk of leading to accidents. For example, lack of concentration during work leads to reduced work efficiency and reduced productivity. Furthermore, lack of concentration or drowsiness while driving a vehicle can be a direct cause of traffic accidents. If the concentration level can be constantly monitored and appropriate advice or assistance can be provided according to the concentration level or tension at any given time, it can contribute to improving work efficiency or safety. In fact, many such efforts have been proposed. For example, a technology has been proposed that determines the mental concentration level from various biological information and provides advice or assistance according to the concentration level. Patent documents 1 to 3 disclose examples of such technology. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2002-65650 A [Patent Document 2] Patent No. 6003782 [Patent Document 3] Patent No. 5119375 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional systems for assessing mental state related to task suitability had the problem that there were large individual differences in biometric information, and even for the same person, there was a large variability between measurements.

[0005] The present disclosure provides a method and system capable of suppressing the influence of individual differences in biometric information and stably determining the suitability of a worker's work. [Means for solving the problem]

[0006] A method according to one aspect of the present disclosure is a processing method executed by a computer, which includes acquiring biometric information of a driver who drives a vehicle, acquiring driving information related to driving operations by the driver, correlating the biometric information with the driving information while the driver is driving and storing them in a storage device, and determining the suitability of the current driving of the driver based on (1) a correlation between the past driving information and the biometric information stored in the storage device, (2) the current biometric information of the driver, and (3) the current driving information.

[0007] The general or specific aspects of the present disclosure may be realized by an element, an apparatus, a system, a method, or any combination thereof. Effect of the Invention

[0008] According to one aspect of the present disclosure, it is possible to suppress the influence of individual differences and stably determine the driving suitability. [Brief description of the drawings]

[0009] [Figure 1A] FIG. 1 is a diagram illustrating a configuration of a task suitability determination system according to an exemplary embodiment of the present disclosure. [Figure 1B] FIG. 1 is a diagram showing an example of the operation of a task suitability determination system according to an exemplary embodiment of the present disclosure. [Diagram 2] Diagram showing individual differences in biological responses to stress [Figure 3A] FIG. 1 is a diagram showing an imaging device using a light source that emits light having a uniform distribution. [Figure 3B] FIG. 1 is a diagram showing the concept of an imaging device using a light source that emits time-modulated light. [Figure 3C]FIG. 1 is a diagram showing the concept of an imaging device using a light source that emits spatially modulated light. [Figure 4A] FIG. 1 is a diagram showing an example of the layout of a task suitability determination system according to a first embodiment of the present disclosure; [Figure 4B] FIG. 1 is a schematic diagram illustrating a cerebral blood flow measuring device according to a first embodiment of the present disclosure. [Diagram 5] FIG. 1 is a diagram illustrating a configuration of a task suitability determination system according to a first embodiment of the present disclosure. [Figure 6] Diagram showing an example of calculation of driving workload [Figure 7] Diagram showing actual measured changes in cerebral blood flow during a driving task [Figure 8A] First figure to explain the method for estimating cerebral blood flow [Figure 8B] The second figure shows the method for estimating cerebral blood flow. [Figure 8C] The third figure illustrates the method for estimating cerebral blood flow. [Figure 9A] A diagram showing the estimated and measured changes in cerebral blood flow during normal driving. [Figure 9B] A diagram showing the estimated and measured changes in cerebral blood flow when driving is inappropriate [Figure 10A] FIG. 13 is a diagram showing an example of the arrangement of a system for measuring work stress according to a second embodiment of the present disclosure. [Figure 10B] FIG. 13 is a diagram showing an example of the configuration of a cerebral blood flow measuring device according to a second embodiment. [Figure 11A] FIG. 1 is a first diagram for explaining signal processing for acquiring biological information in the second embodiment; [Figure 11B] FIG. 2 is a second diagram for explaining signal processing for acquiring biological information in the second embodiment. [Figure 11C] FIG. 3 is a third diagram for explaining signal processing for acquiring biological information in the second embodiment. [Figure 12] FIG. 13 is a diagram showing an example of the relationship between a work situation and biological information during work. [Figure 13A] A diagram showing the distribution of vital signs and work status estimated based on long-term data. [Figure 13B] FIG. 1 is a diagram showing an example of actually measured biological information distribution. [Figure 13C] FIG. 13 is a diagram showing another example of actually measured biological information distribution. [Figure 14A] FIG. 1 is a first diagram for explaining signal processing for acquiring biological information in the third embodiment. [Figure 14B] FIG. 2 is a second diagram for explaining signal processing for acquiring biological information in the third embodiment. [Figure 14C] FIG. 3 is a third diagram for explaining signal processing for acquiring biological information in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The present disclosure includes a work suitability determination system, method, and computer program described in the following items. [Item 1] The work suitability determination system according to item 1 of the present disclosure is an authentication device that identifies a worker based on the worker's characteristics and outputs information that identifies the worker; a biosensing device that acquires bioinformation of the worker and outputs the bioinformation; a work sensing device that detects a work load of the worker and outputs work information indicating the detected work load; A storage device; A signal processing device; Equipped with. The signal processing device includes: While the worker is performing the work, the information for identifying the worker, the biometric information, the work information, and time information are stored in the storage device in association with each other; estimating the current biometric information of the worker based on a correlation between a time transition of the past work information and a time transition of the biometric information stored in the storage device; The suitability of the current work of the worker is determined by comparing the estimated current biometric information of the worker with the current biometric information of the worker acquired by the biometric sensing device. [Item 2] In the work suitability judgment system according to item 1, the authentication device includes an input device that allows the worker to input a personal authentication code before performing the task; The authentication device may identify the worker based on the input personal authentication code. [Item 3] In the work suitability judgment system according to item 1, The authentication device may include a biometric authentication device that identifies the worker using at least one selected from the group consisting of a fingerprint, a palm print, an iris, and a vein pattern. [Item 4] The work suitability assessment described in item 1 is an imaging device for capturing an image of the worker and acquiring data of the image; The authentication device may identify the worker based on the data of the acquired image. [Item 5] In the work suitability judgment system according to any one of items 1 to 4, The biosensing device includes: Located at a location away from the worker, an image pickup element for capturing an image including the head of the worker and acquiring data of the image; The biometric information of the worker may be acquired based on the data of the acquired image. [Item 6] In the work suitability judgment system according to any one of items 1 to 4, The biosensing device includes: a light source that emits near-infrared light that is spatially or temporally modulated; an image pickup element that captures an image including a face portion of the worker illuminated by the light source; may also include [Item 7] In the work suitability judgment system according to item 6, The near infrared light may be spatially modulated in a dot array pattern, a line and space pattern, or a checkerboard pattern. [Item 8] In the work suitability judgment system according to item 6, the near-infrared light is pulsed light, The imaging element may include at least one charge storage section that receives the pulsed light and stores a signal charge. [Item 9] In the work suitability judgment system according to any one of items 1 to 8, The signal processing device may estimate the current biometric information of the worker by analyzing the correlation stored in the storage device using multivariate analysis. [Item 10] In the work suitability judgment system according to any one of items 1 to 8, The signal processing device may learn the correlation stored in the storage device by machine learning, and estimate the current biometric information of the worker based on a learning result. [Item 11] In the work suitability judgment system according to any one of items 1 to 10, The worker is a driver who drives a vehicle, the task is driving the vehicle, The work information may be information regarding a driving operation of the vehicle. [Item 12] In the work suitability judgment system according to item 11, The driving operation may include at least one selected from the group consisting of an accelerator operation, a brake operation, and a steering operation. [Item 13] In the work suitability judgment system according to item 11 or 12, The work sensing device includes an acceleration sensor, The signal processing device may estimate information related to the driving operation based on information output from the acceleration sensor. [Item 14] In the work suitability judgment system according to any one of items 11 to 13, The vehicle has an automatic driving function including driving assistance, The signal processing device may determine content of the driving assistance in accordance with the determined suitability of the driver, and cause the vehicle to execute the driving assistance. [Item 15] In the work suitability judgment system according to any one of items 1 to 10, The work is an input work using a computer, The work information may be information regarding an operation to be input to the computer. [Item 16] In the work suitability judgment system according to item 15, The operation may include at least one selected from the group consisting of a keyboard input and a mouse operation. [Item 17] In the work suitability judgment system according to item 15 or 16, The signal processing device may cause the computer to output an image or a sound indicating advice regarding the input task, depending on the determined suitability of the worker. [Item 18] In the work suitability judgment system according to any one of items 1 to 10, The task is computer-assisted learning, The work information may be information regarding the content of the learning and the operation of the computer. [Item 19] In the work suitability judgment system according to item 18, The signal processing device may cause the computer to provide learning content according to the determined aptitude of the worker. [Item 20] The method for determining the suitability of a worker's work according to item 20 of the present disclosure includes: acquiring information for identifying the worker, biometric information of the worker performing the work, and work information indicating a load of the work; storing the information for identifying the worker, the biometric information, the work information, and time information in a storage device in association with each other; estimating the current biometric information of the worker based on a correlation between a time transition of the past work information and a time transition of the biometric information stored in the storage device; The suitability of the current task of the worker is determined by comparing the estimated current biometric information of the worker with the acquired current biometric information of the worker. [Item 21] The computer program according to item 21 of the present disclosure comprises: A computer program stored on a computer-readable recording medium, The computer includes: acquiring information for identifying a worker, biometric information of the worker performing the work, and work information indicating a load of the work; storing the information for identifying the worker, the biometric information, the work information, and time information in a storage device in association with each other; estimating the current biometric information of the worker based on a correlation between a time transition of the past work information and a time transition of the biometric information stored in the storage device; determining a suitability of the work of the worker at present by comparing the estimated biometric information of the worker at present with the acquired biometric information of the worker at present; Make the following happen.

[0011] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, and component arrangement positions shown in the following embodiments are merely examples and are not intended to limit the technology of the present disclosure. In addition, among the components in the following embodiments, components that are not described in the independent claims that indicate the highest concept are described as optional components.

[0012] In the present disclosure, all or part of a circuit, unit, device, member or part, or all or part of a functional block in a block diagram may be implemented by one or more electronic circuits including, for example, a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). The LSI or IC may be integrated into one chip, or may be configured by combining multiple chips. For example, functional blocks other than memory elements may be integrated into one chip. Here, although it is referred to as an LSI or IC, the name may change depending on the degree of integration, and may be referred to as a system LSI, a VLSI (very large scale integration), or the like. Ultra large scale integration, or ULSI A field programmable gate array (FPGA), which is programmed after the LSI is manufactured, or a reconfigurable logic device, which can reconfigure the connections inside the LSI or set up circuit sections inside the LSI, can also be used for the same purpose.

[0013] Furthermore, all or part of the functions or operations of a circuit, unit, device, member, or part can be executed by software processing. In this case, the software is recorded in one or more non-transitory recording media such as ROMs, optical disks, hard disk drives, etc., and when the software is executed by a processor, the functions specified in the software are executed by the processor and peripheral devices. The system or device may include one or more non-transitory recording media on which the software is recorded, a processor, and necessary hardware devices, such as interfaces.

[0014] (Findings on which this disclosure is based) Before describing the embodiments of the present disclosure, the findings on which the present disclosure is based will be described.

[0015] As mentioned above, whether a person's mental state during work is appropriate for that work is an extremely important factor that affects not only the efficiency of the work but also the safety of the work. For this reason, various efforts have been made to estimate the mental state. Various biological information has been detected to estimate mental states such as concentration or concentration. For example, biological information such as heart rate, heart rate fluctuation, respiratory rate, respiratory rate fluctuation, breathing depth, blood pressure, brain waves, cerebral blood flow, pupil diameter, nasal temperature, blinking, and eye movement have been used. However, there are three major problems with directly reading mental states from these biological information.

[0016] The first issue is individual differences in biological reactions. Even when the accuracy of the work is the same, or the level of arousal or concentration measured based on an interview with the person is the same, the way biological reactions appear varies greatly from person to person. Simply measuring biological information does not allow for accurate estimation of an individual's mental state.

[0017] The second issue is stability. Even for the same subject, the biological reactions may differ depending on the environment at the time of measurement, or the behavior or work history up to the time of measurement. Even if the subjects are limited to specific individuals, it is difficult to estimate the mental state of the individual with high accuracy based only on the biological reactions at the time of measurement.

[0018] The third issue is the testing method. A commonly used method involves providing some kind of stimulus to the subject and estimating the mental state from the biological reaction. In a laboratory, the environment is kept constant and specific tasks (e.g., calculation problems, quizzes, or direct stimulation of the body) are performed to measure the biological reaction. However, during actual work, the work environment and work content may change in various ways. In such a situation, it is not easy to estimate the mental state of a worker in real time.

[0019] FIG. 2 shows the individual differences and the variability of the changes in biological information when a task is performed. Here, a calculation task was given to the subject, and the degree of stress associated with the task was measured based on the fluctuations in heart rate fluctuations when relaxed and when performing the task. The stress at this time is sometimes called the degree of arousal. The heart rate is constantly fluctuating. It is said that the high-frequency components of the fluctuations in the heart rate fluctuations are affected by the activity of the parasympathetic nerves caused by breathing. On the other hand, it is said that the low-frequency components are affected by the activity of both the sympathetic nerves and the parasympathetic nerves. In this example, the range of the high-frequency components is 0.20 Hz or more and less than 0.35 Hz, and the range of the low-frequency components is 0.05 Hz or more and less than 0.20 Hz. The value indicating the magnitude of the high-frequency components of the fluctuations in the heart rate is denoted as "HF", and the value indicating the magnitude of the low-frequency components is denoted as "LF". LF / HF increases when one is under stress or tension. Therefore, this value can be used as an index of stress or tension.

[0020] ΔLF / HF on the vertical axis of Figure 2 represents the amount of change in LF / HF before and after performing the task. From the results of Figure 2, ΔLF / HF exceeds zero (0), with some exceptions. This shows that LF / HF tends to increase with stress or tension. However, this increase tends to decrease as the subject gets older.

[0021] The black circles in the figure show the results of multiple tests on the same person. It can be seen that there is a large variation between tests, even for the same subject.

[0022] The results in Figure 2 were obtained under limited conditions in which a specific task was given to each subject. However, the content of the work, i.e., the task, is not constant during actual work. When the content of the work changes, the biometric information changes accordingly. For this reason, it has been difficult to determine the work suitability of a worker during actual work based only on the measured values ​​of the biometric information. Here, "work suitability" refers to the degree of mental state that allows one to perform work efficiently without making mistakes. Work suitability refers to the degree of mental state, such as the degree of alertness, tension, attention, or concentration.

[0023] It is also possible to consider a system that interrupts a task during work, performs a certain judgment task, and judges the appropriateness of the work from the results. However, in reality, it is not realistic to incorporate such a task into the work. For this reason, such a system is not widely used.

[0024] The present disclosure provides a novel technique that enables a stable estimation of a mental state of a worker while working. According to an embodiment of the present disclosure, the workload of each worker is monitored, and the relationship between the workload and biological information is learned. By utilizing data accumulated through multiple learning sessions, it becomes possible to estimate changes in biological information associated with work.

[0025] 1A is a diagram showing the configuration of a task suitability determination system according to an exemplary embodiment of the present disclosure. The task suitability determination system 100 includes an authentication device 101, a task sensing device 102, a biometric sensing device 106, a signal processing device 108, a storage device 104, and a display device 107. The signal processing device 108 includes a biometric information estimation unit 103 and a task suitability determination unit 105.

[0026] In this embodiment, an individual is recognized when starting a task, and a database is constructed for each individual. In the database, information indicating the workload of each task (referred to as "task information" in this specification) and biometric information acquired during the task are recorded in association with each other. The task information may include the content of the task. By utilizing the recorded data, the influence of individual differences in biometric information can be reduced, and task suitability can be determined stably regardless of the worker.

[0027] The personal authentication is performed, for example, by a worker inputting a worker ID at the start of work, or by the authentication device 101 performing biometric authentication. The biometric authentication can be performed by a method such as face authentication, fingerprint authentication, iris authentication, or vein authentication. In a mode in which a worker inputs a personal authentication code such as a worker ID before work, the authentication device 101 can be a device equipped with an input device, such as a portable information terminal or a personal computer (PC). The input device can include, for example, at least one of a keyboard and a mouse. In a mode in which the authentication device 101 performs biometric authentication, the authentication device 101 is a device equipped with a biometric authentication function such as face authentication, fingerprint authentication, iris authentication, or vein authentication. In the latter mode, the authentication device 101 is equipped with one or more biometric authentication devices required for personal authentication, such as a camera or a fingerprint sensor.

[0028] The work sensing device 102 monitors the current work status of the worker and outputs work information indicating the work load. The configuration of the work sensing device 102 differs depending on the content of the work. For example, if the work is driving a vehicle, the work sensing device 102 may include a sensor such as an acceleration sensor or an angular velocity sensor. If the work is office work or learning using a computer, the work sensing device 102 may include the computer or an input device thereof.

[0029] The biosensing device 106 continuously or intermittently measures the bioinformation of the worker during work. The bioinformation acquired by the biosensing device 106 is actually measured bioinformation. For this reason, the bioinformation may be referred to as "actually measured bioinformation." The biosensing device 106 may acquire the bioinformation of the worker in a non-contact manner. When the biosensing device 106 acquires the bioinformation of the worker in a non-contact manner, the bioinformation can be acquired even if it is difficult to wear a contact-type device due to the work content. By using a non-contact type biosensing device 106, it is also possible to eliminate the feeling of restraint or discomfort caused by wearing a biosensor.

[0030] The storage device 104 stores a database of the acquired individual actual biometric information and task information. The database accumulates data for each individual and for each task. The storage device 104 may include any storage medium, such as a flash memory, a magnetic disk, or an optical disk.

[0031] The signal processing device 108 may be realized by, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), or a combination of a central processing unit (CPU) and a computer program. The signal processing device 108 may be a component of an external device such as a server provided at a remote location. In this case, the external device such as the server includes a communication means and transmits and receives data to and from the authentication device 101, the work sensing device 102, the biometric sensing device 106, the storage device 104, and the display device 107.

[0032] The signal processing device 108 includes a biometric information estimation unit 103 and an activity suitability determination unit 105. Each of these units can be realized by the signal processing device 108 executing a computer program stored in the storage device 104. Alternatively, each of these units may be an individual circuit configured to perform the operations described below.

[0033] The biometric information estimation unit 103 estimates the current biometric information of the worker based on the data stored in the storage device 104. The estimated biometric information is a value of the current biometric information estimated based on the worker's workload and biometric information acquired in the past. For this reason, this is sometimes referred to as "estimated biometric information." The work suitability determination unit 105 compares the estimated biometric information of the worker with the actually measured biometric information, and determines the suitability of the work based on the comparison result. Specific examples of the processes performed by the biometric information estimation unit 103 and the work suitability determination unit 105 will be described later.

[0034] If the worker has no abnormality, the estimated vital information based on past data will be almost the same as the current actual vital information. On the other hand, if the worker has some abnormality, a change occurs between the estimated vital information and the actual vital information that is greater than the variation value calculated from the past data. In that case, the work suitability determination unit 105 determines that the suitability of the work is insufficient. In that case, the work suitability determination unit 105 causes the display device 107 to execute display of advice or warning to the worker. Alternatively, the work suitability determination unit 105 may forcibly stop the work. The forcible stop of the work is, for example, a process of stopping the device or the system. Note that instead of the display device 107 or in addition to the display device 107, a speaker may be provided and the speaker may output a sound for calling attention or advice.

[0035] Next, an example of the overall operation of the task suitability determination system will be described with reference to FIG. 1B.

[0036] 1B is a flowchart showing an example of the operation of the task suitability determination system. This system executes the operation shown in FIG. 1B for each task.

[0037] First, in step S101, the authentication device 101 identifies the worker. As described above, this identification is performed by a method such as an input operation by the worker or biometric authentication.

[0038] In step S102, the biosensing device 106 acquires bioinformation of the worker. The bioinformation may be acquired, for example, by using a non-contact cerebral blood flow measuring device described later. The acquired bioinformation may be, for example, information indicating a time change in the worker's cerebral blood flow or information indicating a time change in the worker's heart rate.

[0039] In step S103, the work sensing device 102 detects the current work content of the worker and outputs information indicating the work content. A value indicative of the degree of strain may be calculated. This calculation of the strain of the task may be performed by a device other than the task sensing device 102, for example a processor of the signal processing device 108.

[0040] In step S104, the signal processing device 108 records the actually measured biological information and information indicating the workload in the storage device 104 for each worker in association with time information.

[0041] In step S106, the signal processing device 108 compares the estimated biometric information of the worker with the measured biometric information. Then, in step S107, the signal processing device 108 determines the worker's work suitability based on the comparison result. For example, if the difference between the estimated biometric information value and the measured biometric information value exceeds a predetermined threshold, it can be determined that the work is not suitable. Alternatively, the suitability of the work may be determined in stages according to the magnitude of the difference between the estimated biometric information value and the measured biometric information value.

[0042] If it is determined that the work is not appropriate, the signal processing device 108 proceeds to step S108. In step S108, the signal processing device 108 advises or warns the worker, or forcibly stops the work, via another device such as the display device 107, as described above.

[0043] The operations from step S102 to S108 may be repeatedly performed, for example, at a predetermined time interval, while the worker is performing the work. Furthermore, when a single task is completed and the worker performs the same task the next time, data recorded in the past is used in step S105. Using data accumulated over a relatively long period of time allows for more accurate estimation.

[0044] The particularly important points in the embodiment of the present disclosure are as follows. (1) Work information indicating the workload and biometric information are accumulated in a database for each individual for a relatively long period of time. For example, data can be accumulated repeatedly for each work task over a period of several days, a week, a month, several months, half a year, a year, or several years. (2) Estimate current biometric information based on accumulated data. (3) The estimated current vital signs are compared with the actually measured vital signs to determine the current suitability of the task.

[0045] In this embodiment, a mechanism is constructed that learns changes in the workload and changes in the biological information, and estimates the current biological information in real time, thereby making it possible to more accurately estimate the workload of a worker.

[0046] Conventionally, there have been systems that build databases of individual biometric information to correct individual differences and determine the worker's work aptitude by correcting the individual differences. Such systems have been shown to be effective in tests in which a certain task, i.e., a test task, is performed in a test environment and the reaction is observed. For example, a method is known in which a worker is given a test task such as a calculation problem or recalling related words, and the change in cerebral blood flow is measured. However, in an actual work environment, it is difficult to give such a task while working, studying, or driving. It is necessary to determine the mental state of a worker from changes in biometric information during work with various different loads. However, simple methods such as simple averaging of past data or averaging of data extracted from similar situations could not correct the influence of individual differences or work load.

[0047] In an embodiment of the present disclosure, a relationship between an individual's workload and a biological response is derived from past data of the workload of actual work and the measured values ​​of biological information. The current mental state or task aptitude of the worker is estimated from the relationship between the workload and the biological response. The correlation between an individual's workload and biological information is derived from past measurement data, and the current biological information is estimated based on the correlation. This is the biggest difference from the conventional system.

[0048] Next, an example of a method for measuring biological information will be described.

[0049] In this embodiment, a stable and highly accurate bio-measuring device is used to stably determine the mental state. If one tries to constantly measure bio-information during work, a sensor that contacts the body may interfere with the work. In addition, in a contact-type sensor, the contact state of the sensor may change due to body movements associated with work, and the measurement accuracy may decrease. In consideration of such problems, in the embodiment shown in FIG. 1A, a non-contact bio-sensing device 106 based on an imaging device is used. By using an imaging device, non-contact bio-sensing without a sense of restraint can be realized. In addition, the imaging device has many pixels arranged two-dimensionally, and can simultaneously obtain many pieces of bio-information. Therefore, high accuracy can be achieved by processing such as averaging the signals of multiple pixels. Furthermore, the imaging device obtains a two-dimensional image signal. Another major advantage is that information on the difference in bio-reactions of different parts of the body can be obtained from this two-dimensional image signal. For example, it is known that mental stress reduces nasal blood flow and nasal temperature. When an imaging device is used, nasal blood flow and forehead blood flow to be compared can be measured simultaneously. This makes it possible to stably measure changes in biological information.

[0050] As described above, the method of acquiring biological information using an imaging device has many advantages, but also has some problems. That is, it is difficult to acquire information about the inside of the body, which is necessary for determining the state of brain activity. This is because when an opaque subject such as a living body is photographed, the information of light reflected from the surface of the body accounts for the majority, and the information of light from inside the living body is buried. Near-infrared light has a higher transmittance into the human body than visible light, and is suitable for acquiring information from deeper parts of the living body. However, even when near-infrared light is used, the ratio of reflected light from the surface or epidermis of the living body is higher than that of reflected light from inside the living body. Therefore, there was a problem that the signal-to-noise ratio was reduced and stable and high-precision measurement could not be performed.

[0051] In order to increase the proportion of reflected light from inside the living body that contains biological information and to stably acquire biological information, it is conceivable to modulate the light emitted from the light source temporally or spatially. Such a configuration makes it possible to selectively acquire optical signals from deeper inside the living body.

[0052] Fig. 3A is a schematic diagram showing an imaging method using a light source that emits light having a uniform distribution, Fig. 3B is a schematic diagram showing an imaging method using a light source that emits time-modulated light, and Fig. 3C is a schematic diagram showing an imaging method using a light source that emits spatially modulated light.

[0053] In the example shown in Fig. 3A, the light source 601a emits light having a uniform distribution. When the light source 601a illuminates and images a living body 602, most of the signals acquired by the imaging device 603 are components of light reflected from the surface. Since the ratio of the light components from inside the living body is extremely small, sufficient accuracy cannot be obtained.

[0054] In contrast, in the example shown in FIG. 3B, the light source 601b emits light to which time modulation has been applied. The light source 601b is used to illuminate the living body 602, and the timing of imaging by the imaging device 603 is controlled in synchronization with the modulation of the light from the light source 601b. In this case, the time from light emission to detection can be changed by controlling the light emission time and the timing of imaging. Here, the time from light emission to detection depends on the propagation distance of light from the light source 601b through the living body 602 to the imaging device 603. By appropriately controlling the timing of light emission and detection, it becomes possible to detect reflected light having information of any depth. By using this method, it becomes possible to selectively detect reflected light having information of any depth of the living body. Therefore, it becomes possible to detect biological information with a high signal-to-noise ratio.

[0055] In the example shown in FIG. 3C, the light source 601c emits spatially modulated light. The light emitted from the light source 601c illuminates the living body 602, and the imaging device 603 acquires a signal from an area outside the illuminated area. When the position of the light source 601c is separated from the detection position, the light reaches the detection position via an arc-shaped light path called a banana shape. Compared to the case where the light source 601a emitting uniform light shown in FIG. 3A is used, the detected signal contains information of light that has penetrated deeper into the living body. For this reason, the method using the light source 601c is suitable for applications in which deep information of a living body is acquired.

[0056] In this way, by using a light source that emits temporally modulated or spatially modulated light and an imaging method corresponding to the light source, it is possible to obtain information about deeper parts of a living body.

[0057] In an embodiment of the present disclosure, the system is designed based on the following new findings. - In order to reduce the influence of individual differences in biological reactions and to construct a practical system for determining the suitability of work, it is effective to use biological information estimated from a database that records the relationship between biological information and workload. In order to improve the accuracy of acquiring biometric information, it is effective to use an imaging device that includes a light source that emits near-infrared light that is modulated temporally or spatially.

[0058] Here, near-infrared light, i.e., near-infrared light, refers to electromagnetic waves having a wavelength in a vacuum of approximately 700 nm or more and 2500 nm or less. The embodiments of the present disclosure are not limited to those using near-infrared light. However, by using near-infrared light, it is possible to obtain biosignals with higher accuracy.

[0059] Hereinafter, the embodiments of the present disclosure will be described in more detail. However, more detailed explanations than necessary may be omitted. For example, detailed explanations of already well-known matters and duplicate explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the inventor provides the accompanying drawings and the following explanation so that those skilled in the art can fully understand the present disclosure, and does not intend to limit the subject matter described in the claims by them.

[0060] (Embodiment 1) As a first embodiment, a task suitability determination system using a non-contact cerebral blood flow measuring device will be described. In this embodiment, task suitability is determined from information on changes in the worker's cerebral blood flow. This embodiment is an example in which the task suitability determination system is applied to driver monitoring. The target task is driving a vehicle, and the worker is a driver (also referred to as a "driver").

[0061] A cerebral blood flow measuring device that measures changes in cerebral blood flow using near-infrared light has been proposed. In the conventional method, a worker repeats a predetermined fixed task and a rest multiple times. The fixed task is, for example, calculation, memorization, association, or a quiz. The cerebral blood flow measuring device measures the change in cerebral blood flow during each period. The degree of brain activity can be estimated from the data obtained by the measurement. However, it is not realistic to repeatedly perform such a routine task while driving a vehicle.

[0062] In this embodiment, instead of a routine task, an indefinite task, such as driving operation, is used as the task. Then, changes in cerebral blood flow are measured during driving operation. During driving operation, the surrounding environment changes constantly. Therefore, driving operation may be a constantly changing task. In this embodiment, the workload is estimated based on information acquired by an acceleration sensor. In other words, this embodiment enables "taskless cerebral blood flow measurement."

[0063] FIG. 4A is a diagram showing a configuration example of a task suitability assessment system according to the first embodiment of the present disclosure. As shown in FIG. 4A, the task suitability assessment system includes a cerebral blood flow measuring device 401 and an acceleration sensor 403. The cerebral blood flow measuring device 401 is an example of the above-mentioned biosensing device. The acceleration sensor 403 is an example of the above-mentioned task sensing device. The cerebral blood flow measuring device 401 measures the cerebral blood flow of a driver 402. The cerebral blood flow measuring device 401 is disposed at a position on the upper part of the vehicle so as not to obstruct the forward view of the driver's seat side windshield in front of the driver 402. The acceleration sensor 403 is disposed adjacent to the cerebral blood flow measuring device 401. The acceleration sensor 403 monitors the status of the driving task.

[0064] The configuration of the cerebral blood flow measuring device 401 will be described with reference to Fig. 4B. Fig. 4B is a diagram showing a schematic configuration of the cerebral blood flow measuring device 401 according to the embodiment 1 of the present disclosure. The cerebral blood flow measuring device 401 includes a light source 601, an imaging device 603, and a control circuit 604. The control circuit 604 is connected to the light source 601 and the imaging device 603. The control circuit 604 controls the light source 601 and the imaging device 603.

[0065] The cerebral blood flow measuring device 401 uses a TOF (Time Of Flight) imaging device 603 to measure the cerebral blood flow of the driver 402 in a non-contact manner. The light source 601 emits time-modulated light in accordance with instructions from a control circuit 604. The imaging device 603 captures an image in synchronization with the emission of light from the light source 601 in accordance with instructions from the control circuit 604. The imaging device 603 captures an image of the forehead, which is the measurement location of the driver 402. The imaging device 603 includes at least one charge accumulation unit that receives light and accumulates a signal charge. Typically, the imaging device 603 includes multiple charge accumulation units arranged two-dimensionally. This allows a two-dimensional image signal to be acquired.

[0066] The light source 601 in this embodiment is a light source that emits near-infrared pulsed light with wavelengths of 750 nm and 850 nm. The light source 601 irradiates the forehead of the driver 402 with these two types of near-infrared light. The light source 601 can be, for example, a laser light source.

[0067] The imaging device 603 in this embodiment includes an image sensor having a plurality of light receiving elements arranged two-dimensionally. Each light receiving element is sensitive to infrared light of the above two wavelengths. Therefore, the imaging device 603 captures a two-dimensional image by the above two types of infrared light irradiated onto the forehead of the driver 402.

[0068] The control circuit 604 can measure the distance to the subject based on the time from when the light source 601 emits pulsed light to when the reflected light reaches the imaging device 603. The control circuit 604 resolves the light signal returning from the forehead in terms of time, and measures the intensity of the light component 404a reflected from the surface of the forehead and the intensity of the light component 404b that reaches the brain and returns. For example, the control circuit 604 controls the timing of the electronic shutter of the imaging device 603 to measure the intensity of each of the above two light components 404a and 404b. Based on the intensity of these two types of light, it is possible to measure the changes in the concentration of oxygenated hemoglobin (HbO2) and the concentration of deoxygenated hemoglobin (Hb) in the cerebral blood. In this way, the cerebral blood flow measuring device 401 measures the changes in the concentration of oxygenated hemoglobin and deoxygenated hemoglobin in the brain using the light source 601 that emits light of two wavelengths. This makes it possible to determine the task aptitude of the driver 402 when driving.

[0069] The main role of blood is to receive oxygen from the lungs and transport it to the tissues, and to pick up carbon dioxide from the tissues. The goal is to capture and circulate the hemoglobin in the lungs. There are about 15 g of hemoglobin in 100 ml of blood. Hemoglobin bound to oxygen is called oxygenated hemoglobin, and hemoglobin not bound to oxygen is called deoxygenated hemoglobin. Oxygenated hemoglobin and deoxygenated hemoglobin have different light absorption characteristics. Oxygenated hemoglobin absorbs near-infrared light with wavelengths longer than about 830 nm relatively well. On the other hand, deoxygenated hemoglobin absorbs red light and near-infrared light with wavelengths shorter than 830 nm relatively well. There is no difference in the absorption rate of the near-infrared light with a wavelength of 830 nm between the two. The ratio of the intensity of these infrared and red lights can be used to determine the ratio of the two types of hemoglobin or the oxygen saturation. Oxygen saturation is a value that indicates how much of the hemoglobin in the blood is bound to oxygen. Oxygen saturation is defined by the following formula. Oxygen saturation = C(HbO2) / [C(HbO2)+C(Hb)]×100(%) Here, C(Hb) represents the concentration of deoxygenated hemoglobin, and C(HbO2) represents the concentration of oxygenated hemoglobin.

[0070] In addition to blood, living organisms contain components that absorb light in the red to near-infrared wavelength range. However, the temporal variation in light absorption rate is mainly due to hemoglobin in arterial blood. Therefore, based on the variation in absorption rate, it is possible to measure the concentration change of two types of hemoglobin and / or blood oxygen saturation with high accuracy. Arterial blood pumped from the heart moves through the blood vessels as a pulse wave. On the other hand, venous blood does not have a pulse wave. Light that enters a living organism is absorbed by each layer of the organism, such as arteries, veins, and tissues other than blood, and passes through the organism. At this time, the thickness of tissues other than arteries does not change over time. Therefore, scattered light from within the organism shows a temporal change in intensity according to the change in the thickness of the arterial blood layer due to pulsation. This change reflects the change in the thickness of the arterial blood layer and does not include the influence of venous blood and tissue. Therefore, by focusing only on the variable component of scattered light, information on arterial blood can be obtained. The pulse rate can also be obtained by measuring the period of the component that changes over time.

[0071] It is not essential to use the light source 601 that emits light of two wavelengths. For example, when only the concentration of oxygenated hemoglobin is to be measured, a light source that emits near-infrared light of a single wavelength exceeding 830 nm may be used.

[0072] 5 is a diagram showing a configuration of the task suitability determination system 200 according to the first embodiment of the present disclosure. The task suitability determination system 200 includes an authentication device 201, a signal processing device 208, a storage device 204, and a display device 206 in addition to the acceleration sensor 403 and the cerebral blood flow measuring device 401 described above. The signal processing device 208 includes a biological information estimation unit 203 and a driving suitability determination unit 207. As described above, the acceleration sensor 403 functions as a task sensing device, and the cerebral blood flow measuring device 401 functions as a biological sensing device.

[0073] The authentication device 201 authenticates the driver who is the operator. The authentication device 201 includes an imaging device. When the driver starts driving, the imaging device captures an image of the driver's face. The driver is identified by comparing the face image with a face image previously recorded in the storage device 204.

[0074] The imaging device in the authentication device 201 may be the same device as the imaging device 603 in the cerebral blood flow measuring device 401. In that case, the cerebral blood flow measuring device 401 also identifies the driver by face authentication using the image used for cerebral blood flow measurement. The cerebral blood flow measuring device 205 also has a camera function. An image of the light component 404a reflected from the surface of the driver's face is similar to an image acquired by a normal camera. Therefore, personal authentication can be easily performed using the image output from the cerebral blood flow measuring device 401. In this way, one device may have both the function of the authentication device and the function of the biosensing device.

[0075] The acceleration sensor 403 monitors the current driving situation and measures the workload of the driver 402. The acceleration sensor 403 can monitor not only the degree of acceleration and deceleration during driving, but also the integral value of acceleration indicating the driving speed, the lateral acceleration indicating the cornering speed on a curve, and the periodic fluctuation of the lateral acceleration indicating the wobble of the vehicle. Therefore, the acceleration sensor 403 is effective in grasping the driving operation situation of the driver 402.

[0076] In order to grasp the driving situation, it is also possible to obtain driving information from the vehicle. Here, "driving information" means information on changes over time in driving operations such as accelerator, brake, and steering operation. Driving information is an example of the above-mentioned work information. The work suitability determination system 200 may be equipped with a communication circuit to enable data transmission and reception between the vehicle. In such a system, a computer in the vehicle functions as a work sensing device, rather than the acceleration sensor 403. If the acceleration sensor 403 is used as in this embodiment, a system can be constructed at low cost. In addition, the system can be constructed simply without integrating different information for each vehicle model. In the future, if information communication for automobiles evolves and it becomes easier to obtain vehicle information, it may become easier to receive driving information from the vehicle without providing the acceleration sensor 403 in the work suitability determination system 200.

[0077] A method for grasping the driving situation based on the output of the acceleration sensor 403 and further calculating the driving load, that is, the work load, will be described below.

[0078] As already mentioned, the acceleration sensor 403 can monitor various driving conditions such as acceleration, deceleration, driving speed, cornering speed, and vehicle sway during driving. Based on at least a part of these pieces of information, the load of the driving task can be calculated. The load of the driving task increases as the acceleration or deceleration increases, the vehicle speed increases, and the cornering speed increases. In addition, keeping the vehicle stable within the lane also increases the driving load. Vehicle sway indicates that the vehicle is not stably maintained within the lane. Therefore, the vehicle sway itself is an important indicator of driving suitability. Vehicle sway can be detected from small periodic changes in lateral acceleration. In the calculation of the driving load, the sway acts in the direction of reducing the driving load. This is because a large driving load is required to stably control the vehicle. Here, the acceleration of the vehicle is α, the lateral acceleration is β, and the gentle periodic fluctuation component of the lateral acceleration is γ. Then, the driving load L(t) is expressed by the formula (1). L(t)=k1|α(t)|+k2∫α(t)dt+k3|β(t)|+k4|γ(t)| (1) Here, k1 to k4 are constants, and ∫α(t)dt represents the vehicle speed. Appropriate values ​​are set in advance for the constants k1 to k4 based on, for example, experiments.

[0079] FIG. 6 shows an example of calculation results of acceleration α(t), speed v(t), and driving load L(t) when driving operations of acceleration and deceleration are performed. The driving load L(t) can be easily calculated based on the data of the acceleration sensor 403 and equation (1). The calculation of the driving load L(t) can be performed by a processor built into or connected to the acceleration sensor 403. The signal processing device 208 may calculate the driving load L(t).

[0080] The cerebral blood flow measuring device 401 measures the cerebral blood flow of the driver 402 continuously or intermittently in a non-contact manner using the above-mentioned method.

[0081] Figure 7 shows an example of measured values ​​of changes in a driver's cerebral blood flow during a vehicle driving test on a test course. The driver performed a driving task in which a cycle of acceleration, deceleration, and stopping was repeated every 40 seconds. The shaded sections indicate periods during which the vehicle was accelerating or decelerating, and the white areas indicate periods during which the vehicle was stopped. It can be seen that the driving operations of acceleration and deceleration increased the oxygenated hemoglobin (HbO2) concentration in the driver's cerebral blood flow and decreased the deoxygenated hemoglobin (Hb) concentration. It can also be seen that the blood flow changes in response to the driving operations. This change in cerebral blood flow is thought to occur through the following mechanism.

[0082] First, local neural activity occurs in the frontal lobe due to driving, and oxygen consumption by brain cells increases. To supply oxygen to the activated brain cells, the local blood flow rate of the surrounding capillaries increases. At this time, the increase in blood flow rate for oxygen supply (about 30% to about 50%) is greater than the increase in actual oxygen consumption (about 5%). The blood flow rate and flow velocity in the capillaries and venules increase. Because oxygenated hemoglobin flows rapidly, the oxygenated hemoglobin concentration increases and the deoxygenated hemoglobin concentration decreases. This is the mechanism by which brain activity increases the oxygenated hemoglobin concentration and decreases the deoxygenated hemoglobin concentration. It is important to note here that the increase in blood flow occurs after the brain is activated and consumes oxygen, so the change in cerebral blood flow occurs with a delay from brain activity. As can be seen from Figure 7, the change in cerebral blood flow occurs with a time delay of several seconds from the brain activity caused by driving operations. This time delay is important in estimating the current cerebral blood flow.

[0083] The signal processing device 208 accumulates the acquired individual biometric information and driving load in the storage device 204, and constructs a database. The biometric information estimation unit 203 in the signal processing device 208 estimates the current biometric information from the accumulated data. This estimated biometric information is a value of the current biometric information estimated from the past driving load and biometric information of the driver 402 who is the operator. The operation of this biometric information estimation unit 203 will be described in more detail.

[0084] An estimated value of cerebral blood flow change can be calculated by convoluting a hemodynamic response function (HRF) and a workload obtained from data acquired by the acceleration sensor 403. The hemodynamic response function is a function that represents the time change of cerebral blood flow change caused by brain activity. The calculation formula for the amount of change in cerebral blood flow is expressed by, for example, formula (2). HRF i (t)*L i (t) = ΔHbO i (t) (2)

[0085] Here, HRF i (t) is the hemodynamic response function of driver i, L i (t) is the workload of driver i, ΔHbO i (t) is the change in the estimated oxygenated hemoglobin concentration of the driver i. The symbol "*" represents a convolution integral. The cerebral blood flow measuring device 401 in this embodiment can measure both the oxygenated hemoglobin concentration and the deoxygenated hemoglobin concentration. However, in this example, only the change in the concentration of oxygenated hemoglobin is used as the change in the cerebral blood flow accompanying brain activity. This is because, as shown in FIG. 7, in many cases, the oxygenated hemoglobin concentration and the deoxygenated hemoglobin concentration change inversely, and the change amount of the oxygenated hemoglobin concentration is larger. However, the deoxygenated hemoglobin concentration can also be used, for example, to detect an abnormality in the measurement data. If the measurement is performed correctly, the two will have an inverse relationship as already mentioned. However, when some kind of disturbance occurs, such as a fluctuation in the irradiated light or a body movement, the two may change in the same direction in conjunction with each other. By monitoring the concentration of deoxygenated hemoglobin, it is possible to easily detect measurement abnormalities and remove such abnormal data. The important point here is that the hemodynamic response function HRF i (t) and workload L i (t) changes over time.

[0086] Hemodynamic response function HRF i(t) can be approximated by the function shown in equation (3). HRF i (t)=A o ((t-δ) / τ) 2 exp(-((t-δ) / τ) 2 ) (3)

[0087] The variable A in this equation o , δ, and τ, and workload L i In equation (1) that expresses (t), k1 to k4 are constantly recalculated from the measurement results.

[0088] FIG. 8A shows the hemodynamic response function HRF i FIG. 8B is a diagram showing an example of the workload L i FIG. 8C is a diagram showing an example of the change in oxygenated hemoglobin concentration ΔHbO i FIG. 13 is a diagram showing an example of (t).

[0089] The change in oxygenated hemoglobin concentration, ΔHbO i (t) is calculated using the function shown in Figures 8A and 8B in the manner shown in Equation (2). i and the change in the measured oxygenated hemoglobin concentration, ΔHbO m So that A and B match. o, δ, τ, and k1 to k4 are optimized. For the calculation, not only the data at the current time but also the time-dependent data weighted in the time direction is used. Multivariate analysis is used for this parameter determination. In addition to this example, it is also possible to determine the parameters with higher accuracy using machine learning. The signal processing device 208 learns and optimizes the estimation formula of the biological information in real time from the database of the biological information and driving load of each individual stored in the storage device 204. This makes it possible to determine the driving aptitude of the driver with high accuracy. By creating an individual database, the influence of individual differences can be removed, and by accumulating personal data, it is possible to estimate the change in the biological information of an individual with high accuracy. In addition, by calculating the load associated with driving based on the data from the acceleration sensor 403, it is possible to estimate the change in the biological information with respect to the work or driving situation that changes over time.

[0090] FIG. 9A is a diagram showing the estimated (dotted line) and actual (solid line) changes in cerebral blood flow during normal driving. FIG. 9B is a diagram showing the estimated (dotted line) and actual (solid line) changes in cerebral blood flow when driving aptitude is lacking. The changes in cerebral blood flow here are changes in oxygenated hemoglobin concentration. The vertical axis on the left side of FIG. 9A shows changes in oxygenated hemoglobin concentration (ΔHbO), and the vertical axis on the right side shows acceleration. The vertical axis on the left side of FIG. 9B shows changes in oxygenated hemoglobin concentration (ΔHbO), and the vertical axis on the right side shows speed. Changes in cerebral blood flow are observed with a delay after driving operations of acceleration and deceleration.

[0091] The driving suitability determination unit 207 compares the estimated biological information with the actually measured biological information. If the driver has no problem with the driving suitability, as shown in FIG. 9A, the current cerebral blood flow change estimated from the database and the actually measured cerebral blood flow change are almost the same. On the other hand, if the driver has some abnormality, as shown in FIG. 9B, a difference larger than the variation value calculated from the past data occurs between the estimated value and the actually measured value of the biological information. In the example shown in FIG. 9B, the actually measured cerebral blood flow change amount is significantly smaller than the estimated cerebral blood flow change amount. If such a situation continues, the driving suitability determination unit 207 determines that the driving suitability is insufficient. In that case, the driving suitability determination unit 207 calls attention, warns, or switches the driving mode to the driver 402 via the display device 206 or a device such as a speaker (not shown). For example, a voice advice such as "You seem a little tired. Please take a deep breath" may be given to the driver via the speaker. Alternatively, the system may automatically increase ventilation, introduce outside air, or issue a warning such as "It is dangerous, so please stop the vehicle in a safe place." If the vehicle is equipped with a driving assistance system, the system may take measures such as stopping the vehicle in a safe place or switching to automatic driving depending on the driver's driving aptitude.

[0092] The system in this embodiment is a standalone driver monitoring system that can be added to an existing vehicle later. All of the components shown in FIG. 1 are mounted in an integrated system as shown in FIG. 4A. In recent years, vehicles have become increasingly intelligent. Vehicles are equipped with high-performance arithmetic units or computers, and are able to grasp the vehicle situation by integrating information from various sensors in the vehicle. In addition, the vehicle has a communication function such as the Internet, making it possible to exchange information with computers outside the vehicle. In such vehicles, it is also possible to incorporate some of the functions shown in FIG. 1 as software on the vehicle's computer side. For example, only the biosensing device 106 is hardware, and the other functions can use the vehicle's functions. The driver's individual Vehicle models with human authentication functions already exist. Vehicle operation information (e.g., accelerator, brake, or steering wheel operation) captured by the vehicle's computer can be used as output information of the work sensing device 102. The vehicle's computer also has a storage device. For this reason, it is possible for only the biosensing device 106 to be a dedicated piece of hardware, with the other functions being installed as software in the vehicle's hardware. In this way, it is possible to select which parts of the system in FIG. 1 are to be dedicated hardware and which parts are to be installed as software in the vehicle's computer, depending on the type of vehicle and the situation.

[0093] By using the driver monitoring system of this embodiment, it is possible to reduce false detections due to individual differences and differences in driving conditions, and to stably monitor the driving aptitude of the driver at all times. In this method, non-contact cerebral blood flow monitoring allows the driver to be constantly monitored without feeling any sense of restraint or pressure. This allows for safer and more comfortable driving.

[0094] (Embodiment 2) The task suitability determination system in this embodiment measures the task stress of a worker. This embodiment aims to grasp the workload of a worker involved in clerical work using a computer such as a PC, improve the efficiency of the work, and prevent mental health problems of the worker. This task suitability determination system evaluates the mental stress caused by work in the workplace. In the first embodiment, an imaging system including a light source that emits near-infrared pulsed light modulated in time is used to acquire biological information. In contrast, in this embodiment, an imaging system including a light source that emits near-infrared light modulated in space is used to acquire biological information.

[0095] In recent years, the number of workers who feel strong anxiety, worries, or stress regarding their work or professional life has been increasing. For this reason, it is necessary for workplaces to actively work to maintain and promote mental health. For example, the following method is used in the currently conducted "stress check." First, the worker fills out a questionnaire about stress. By compiling and analyzing these, it is possible to understand the state of one's stress. This method is based on the subjective answers of the individual, and there is a large degree of individual variability. There was a need for an index of workload that could serve as a basis for a more objective assessment. For example, there was a need for a simple measurement method that would allow for a numerical evaluation of stress levels.

[0096] Many biological signals related to stress are known. It is known that biological signals such as heart rate variability, nasal temperature, blinking frequency, breathing frequency, or breathing depth, which have already been mentioned, are related to stress states. The problem with using these biological signals for stress checks is individual differences in biological reactions. As shown in FIG. 2, biological reactions vary greatly from person to person, and cannot be used as an objective stress evaluation index as is. FIG. 2 shows the results of heart rate fluctuations due to stress. Similarly, for other physiological indices that are recognized to be correlated with stress, the stress level cannot be directly determined based on the measured values ​​alone. Here, as in the first embodiment, the problems are individual differences in biological reactions, reproducibility, and stability of work tasks.

[0097] To solve these problems, the task suitability determination system of this embodiment is provided. As already mentioned, the current biometric information predicted from the task is estimated from the database of individual workload and biometric information. The estimated current biometric information is compared with the actually measured biometric information to determine the task suitability at the current time. According to the system of this embodiment, the change in the workload and the change in the biometric information are constantly learned, and the stress associated with the current task can always be estimated in real time.

[0098] The system for assessing the suitability of work using a computer such as a PC is also similar to the structure shown in Figure 1A. The system includes a configuration. The task suitability judgment is performed according to the flowchart shown in FIG. 1B. In this embodiment, the purpose is to judge the task suitability associated with a task using a computer, so a computer such as a PC is used for the task. The computing power of the computer for this task can be used to operate the task suitability judgment system.

[0099] In this embodiment, among the components shown in FIG. 1A, only the biosensing device 106 is unique hardware. All other components are stored as hardware or software in a computer. When a worker starts working on a computer, a work suitability assessment is performed in the background of the intended work. The biosensing device 106, which is separate hardware from a computer, is connected to the computer. Information acquired by the biosensing device 106 is processed by the computer.

[0100] Hereinafter, the procedure leading to stress assessment will be specifically described with reference to FIG. 1A. In this embodiment, the authentication device 101 performs personal authentication based on an ID input by a worker. For example, personal authentication is performed by the worker inputting an ID and a password using a keyboard connected to a computer such as a PC. For business computers, an individual is usually identified by inputting an individual's ID and password or by biometric authentication at the login stage before starting work. Personal authentication data from this computer may be used to perform personal authentication for the system.

[0101] The task suitability determination system of this embodiment can also be called a task stress measurement system.

[0102] FIG. 10A is a diagram showing a schematic diagram of the overall configuration of the task suitability determination system of this embodiment. As with the system of embodiment 1, this system also includes a cerebral blood flow measuring device 401 as the biosensing device 102. The cerebral blood flow measuring device 401 is placed, for example, on the screen of a computer 410 in front of a worker 402. The cerebral blood flow measuring device 401 in this embodiment includes a near-infrared dot array light source and a near-infrared imaging device. The cerebral blood flow measuring device 401 measures the facial blood flow of the worker 402. The system of this embodiment can monitor the blood flow, heart rate, and heart rate variability on the surface of the face in a non-contact manner.

[0103] FIG. 10B is a diagram showing a schematic configuration of the cerebral blood flow measuring device 401 in this embodiment. The cerebral blood flow measuring device 401 in this embodiment includes a light source 601c such as a laser that projects a dot pattern by near-infrared light, an imaging device 603 that acquires a near-infrared image, and a control circuit 604. The light source 601c irradiates a living body (here, the head of the operator 402). The imaging device 603 captures an image of the face of the operator 402 irradiated with near-infrared light. The captured image is analyzed by the control circuit 604. The control circuit 604 may be a processor built into a computer such as a PC. In this case, the cerebral blood flow measuring device 401 is realized by a combination of the processor, the light source 601c outside the computer, and the imaging device 603.

[0104] 11A to 11C, a signal processing flow for acquiring biological information from an image captured by an imaging device 603 that acquires a near-infrared image will be described.

[0105] FIG. 11A shows an example of a captured near-infrared image. Because a dot array light source is used, a strong bright spot corresponding to the irradiation position of the dot array and a relatively weak signal around it are obtained. The relatively weak signal in the periphery is a signal of relatively weak light that enters the body, scatters inside the body, and returns to the surface. Human skin has a small absorption coefficient and a large scattering coefficient for near-infrared light. For this reason, light that penetrates the surface of the skin is diffused by repeated multiple scattering inside the body and is emitted from the body surface over a wide range. Note that the dot array pattern However, a light source that projects near-infrared light in a line-and-space pattern or a checkered pattern may be used. Even when such a light source is used, a similar biological signal can be obtained.

[0106] Fig. 11B is an enlarged view of an image of an information detection region of a living body. The information detection region in this example is the forehead region of the worker, as indicated by the dotted frame in Fig. 11A. Intra-body scattered light 501 that has returned after scattering from inside the skin is detected around the periphery of the dot pattern of the projected infrared light. The surface reflected light 502 contains information about the skin surface, and the intra-body scattered light 501 contains blood information about the capillaries in the body. Therefore, by extracting and calculating only data corresponding to the intra-body scattered light 501 from the image data, it is possible to obtain information about blood flow in the body.

[0107] FIG. 11C shows an example of a heart rate signal obtained in this way. From this heart rate signal, heart rate variability or heart rate fluctuation can be obtained. Furthermore, since the absolute value of the reflection intensity of the scattered light within the body corresponds to the blood volume on the surface, the blood volume can be calculated from the signal.

[0108] In conventional bioinformation sensing systems using cameras, the general method is to detect bioinformation by averaging pixel data of the entire area of ​​the biological part in the image. In contrast, the bioinformation sensing system of this embodiment uses a dot array light source, so it can remove unnecessary surface reflected light components from the skin surface from the two-dimensional image and selectively extract the internal scattered light containing the biological information. By efficiently extracting the internal scattered light, it is possible to obtain bioinformation with high accuracy.

[0109] It is known that psychological stress can be estimated from the temporal fluctuation of the heart rate. When the autonomic nervous system is functioning normally, the heartbeat interval fluctuates, but it is known that the fluctuation of the heartbeat interval decreases due to stress. It is possible to detect the presence or absence or the degree of psychological stress based on the change in the fluctuation of the heartbeat interval.

[0110] Next, a method of acquiring work information by the work sensing device 102 will be described. In this embodiment, the work status of the worker is acquired by analyzing input to the computer. In the case of PC work, the work being performed is input to the PC. Therefore, by monitoring the type of application being used and keyboard input or mouse input, work information can be acquired. For example, a history of work information can be easily acquired, such as "the work content is word processing work, and the number of characters input per unit time is 50 characters / minute" or "the work content is spreadsheet work, and the number of items input per unit time is 30 items / minute." The work sensing device 102 in this embodiment is realized by combining a processor in a computer with input devices such as a keyboard and a mouse.

[0111] Next, the operation of the signal processing device 108 in this embodiment will be described.

[0112] FIG. 12 shows data indicating changes in workload and biometric information for one worker over a one-week period, acquired by the system of this embodiment. The work involved inputting data into a word processor, and the workload represented the number of words entered. As the biometric information, LF / HF, which is considered to be an index of sympathetic nerve activity or a stress index, was used. As described above, LF is the low-frequency component of heart rate fluctuation, and HF is the high-frequency component. In this example, the range of the high-frequency component is 0.20 Hz or more and less than 0.35 Hz, and the range of the low-frequency component is 0.05 Hz or more and less than 0.20 Hz.

[0113] As the workload increases due to an increase in the number of input words, the stress index LF / HF increases, as indicated by the arrow 302. If the work continues, the worker's fatigue increases, work efficiency decreases, and the stress index LF / HF increases further, as indicated by the arrow 301. However, as already mentioned, such reactions vary greatly from person to person, and there is a large degree of variability. Therefore, it is not possible to judge the degree of stress or fatigue simply based on the number of input words and the LF / HF value.

[0114] In this embodiment, an average distribution weighted by the time axis from relatively long-term data as shown in FIG. 12 is created for each worker from a database of individual work history and biological information. This distribution is used as the estimated current biological information. The long-term average distribution represents the relationship between the workload and biological information specific to the worker. This is because it is considered that the actual biological information will show biological information close to this distribution if the worker is in a normal state. In contrast, data on the workload and biological reaction obtained by work for a relatively short period of time (for example, about one hour) up to the present (for example, the plot of the graph shown in FIG. 12) is used as the currently measured biological information. The above processing is executed by the biological information estimation unit 103 in the signal processing device 108. In this way, the system of this embodiment captures the workload and biological reaction as a frequency map and judges the worker's work aptitude.

[0115] The task suitability determination unit 105 in the signal processing device 108 compares the estimated map of the current biometric information, i.e., the long-term average distribution, with the map of the current biometric information, i.e., the short-term average distribution. This makes it possible to determine the task suitability of the worker. For example, if the current map has a high ratio of data belonging to the B region shown in FIG. 12 compared with the map of past data, it means that the worker is performing the task with high task efficiency. On the other hand, if the ratio of data belonging to the A region is high, it is estimated that the worker is not concentrating on the task, that is, that the degree of awakening is low, or that the worker is bored with the task. In such a case, the task suitability determination system displays advice such as "Your concentration is slightly reduced. Please stretch and refresh yourself" on the display device 107 (for example, a computer display). Conversely, if the distribution of the current data is biased toward the C region, it is estimated that fatigue has accumulated and task efficiency has decreased. In this case, the present system can display advice such as "You seem a little tired, please take a coffee break" on the display device 107. In this way, by using the system of this embodiment, it is possible to detect a decrease in task efficiency and provide advice at an appropriate time. This makes it possible to achieve high task efficiency.

[0116] Furthermore, by using this system, it is possible to detect signs of poor mental health such as depression. As already mentioned, the relationship between the workload and the biological response over a relatively long period of time for each worker is constantly updated and recorded in the database. By tracing the long-term fluctuations in this distribution, it is possible to grasp the signs of depression. If the tendency to depression gradually progresses, the distribution gradually shifts in the direction indicated by the arrow 301 in FIG. 12. By detecting such a change in distribution, it is possible to grasp the tendency to depression. If such a tendency is observed, measures such as having the worker undergo a diagnosis by an industrial physician can be taken. In the case of depression, it is known that the low-frequency component LF of heart rate fluctuations, which indicates the activity of the parasympathetic nerves, decreases. Therefore, in addition to the relationship between LF / HF and the workload as shown in FIG. 12, the relationship between LF and the workload may be monitored. By doing so, it is possible to detect the tendency to depression with higher accuracy.

[0117] In fact, this system was installed on the computer terminal of one worker whose job it was to enter data into a word processor, and data was collected over a relatively long period of time (three months). The worker's main task was entering data into a word processor.

[0118] Figure 13A shows the relationship between the number of words entered per unit time during work and the LF / HF during input, obtained from three months of input work. The darker the area, the higher the frequency of occurrence. The average input speed was about 75 characters / min. The LF / HF value was about 1.3 times higher than when not working, indicating that the work caused a slight state of tension. This distribution represents the relationship between the long-term fluctuations in the worker's workload and the fluctuations in the worker's vital signs. If the worker has no particular abnormalities in his or her physical and mental condition, the worker will not deviate significantly from this distribution. This distribution is treated as the estimated vital signs. The worker's suitability for work was judged by comparing this distribution with the current vital signs. As the current vital signs, the distribution of the number of words entered in a short period of time (every hour in this example) and the changes in the vital signs were obtained. Two types of peculiar distributions were frequently measured during the three-month measurement. Examples of these distributions are shown in Figures 13B and 13C.

[0119] In the distribution of FIG. 13B, the number of input characters has decreased compared to the distribution of FIG. 13A. This shows that the work efficiency has decreased. In this case, the LF / HF, which indicates the concentration level or stress, has also decreased. At this time, it can be assumed that the worker was lacking in concentration.

[0120] On the other hand, in the distribution of Fig. 13C, the number of input characters is reduced compared to the distribution of Fig. 13A, and the work efficiency is also reduced in this case. However, LF / HF, which indicates stress, is higher than when the worker is performing a highly efficient task. From this, it can be inferred that the worker's work efficiency was reduced due to fatigue associated with the task.

[0121] As described above, even if only the work efficiency is measured, the decrease in work efficiency can be confirmed, but the cause cannot be identified, and it is not clear what measures are effective for improving work efficiency. By using the system of this embodiment, the work aptitude of the worker can be judged and appropriate advice can be given according to the result. This makes it possible to improve labor productivity.

[0122] As described above, the data showing the relationship between long-term workload and biological information shown in FIG. 13A is constantly updated with the latest data. By analyzing the changes in this distribution, the long-term mental health status of the worker can be checked. For example, if the distribution of the high frequency parts shown in FIG. 13A gradually changes in the direction of the arrow 301 shown in FIG. 12, it can be seen that some mental health problem may be developing. Early detection of mental health problems and appropriate measures are particularly important for occupational safety.

[0123] As described above, by comparing the distribution of the workload and biological information over a relatively long period of time with the current distribution of the workload and biological information over a relatively short period of time, the current work efficiency or work suitability can be determined. Also, by monitoring the time-dependent changes in the distribution of the workload and biological information over a relatively long period of time, measures against mental health problems can be efficiently taken.

[0124] (Embodiment 3) This embodiment is a task suitability assessment system for the purpose of improving the efficiency of learning using a computer such as a PC. The task suitability assessment system in this embodiment can also be called a learning suitability assessment system. This system can be used, for example, in schools, cram schools, or online learning. In this embodiment, the worker is a learner, and the task is learning using a computer.

[0125] In recent years, the market for various learning systems and learning materials using computers has expanded. The advantage is that learners can study at their own convenience at their own time. However, there is a problem that the absence of a teacher leads to a lack of concentration and poor results. In order to solve such problems, this embodiment provides a system that judges the learner's concentration level or learning aptitude during learning and provides feedback to the learner. As in the second embodiment, a computer is used in this embodiment. Therefore, the same hardware configuration as in the second embodiment can be used in this embodiment. As shown in FIG. 10A, a biological Only the sensing device 401 is a unique hardware, and is used by connecting it to a computer 410 such as a PC. All of the components shown in FIG. 1A except for the biosensing device 401 are stored as hardware or software in the computer 410. When a worker starts computer work, the computer 410 judges the work suitability in the background of the intended work. By linking the learning software with this system, the learning content can be changed according to the individual's level of understanding and concentration, making learning more efficient.

[0126] In this embodiment, personal authentication is also performed by inputting an ID into a computer such as a PC.

[0127] The hardware configuration of the cerebral blood flow measuring device 401 is the same as that in the second embodiment. As shown in Fig. 10B, the cerebral blood flow measuring device 401 includes a light source 601c, which is a near-infrared dot array light source, and an imaging device 603. The system of the second embodiment measures the heart rate from changes in blood flow in the face, and uses the fluctuations in the heart rate as biological information. In contrast, the system of the present embodiment determines the concentration level from changes in blood flow in the nose.

[0128] A signal processing flow for acquiring biometric information from an image captured by the imaging device 603 will be described with reference to FIGS. 14A to 14C.

[0129] FIG. 14A shows an example of a captured near-infrared image. FIG. 14B is an enlarged view of an image of the information detection region of the living body. In this example, the information detection region of the living body is the nose region shown in the dotted frame in FIG. 14A. The internal scattered light 501 scattered inside the skin is detected around the projected dot pattern. By extracting and calculating only the data corresponding to the internal scattered light 501 from the image data, the blood flow information in the body can be obtained. FIG. 14C shows an example of blood flow information thus obtained. When the blood flow increases, the reflectance decreases due to the absorption of light by the blood. Therefore, the change in blood flow can be detected from the signal of the reflected light. The difference between the signal shown in FIG. 14C and the signal shown in FIG. 11C is the width of the vertical axis, and the change in reflectance due to heart rate fluctuation is smaller than the change in blood flow. FIG. 14C shows the result of removing the influence of heart rate fluctuation by performing a time moving average. In addition to the measurement result of the nose, FIG. 14C also shows the measurement result of the blood flow change in the forehead as comparative data.

[0130] It is known that stress changes the temperature of the nose. Around the nose, there are many anastomoses between arteries and veins, called arteriovenous anastomoses, which are under the control of the sympathetic nerve. Therefore, changes in blood flow caused by activation or inhibition of the autonomic nerve are directly reflected in changes in the skin temperature of the nose. When a person feels tension or stress, the sympathetic nerve is activated and the blood flow decreases. This causes the skin temperature of the nose to drop. It has been traditionally practiced to evaluate mental stress or concentration by monitoring the skin temperature around the nose with a thermograph. The system of this embodiment directly evaluates the blood flow changes that cause temperature changes using a near-infrared imaging device instead of measuring temperature. As already mentioned, the nose has many arteriovenous anastomoses compared to other parts of the body, and the blood flow of the nose is strongly affected by changes in the autonomic nerve. In contrast, the blood flow of the forehead is less affected by changes in the autonomic nerve and is stable, and is closely related to the body temperature deep inside the body. Therefore, the system of this embodiment measures both the blood flow changes of the forehead and nose, and uses the measurement results of the forehead as a reference. This makes it possible to eliminate the influence of disturbances such as changes in illuminance of illumination light and body movement, and to stably detect the presence or absence, or the degree of, psychological stress based on changes in blood flow in the nose.

[0131] Next, a method for acquiring work information by the work sensing device 102 in this embodiment will be described. A learner proceeds with learning by following instructions in software (hereinafter referred to as "learning software") installed on, for example, a PC or a tablet computer. Therefore, the learning contents are always understood by the computer. The cerebral blood flow measuring device 401 in this embodiment measures the change in blood flow when the learning software presents a question and the learner answers it. The cerebral blood flow measuring device 401 acquires data such as that shown in Fig. 14C. When the learner concentrates on answering, the blood flow in the nose decreases and the reflectance of near-infrared light increases.

[0132] Next, the operation of the biological information estimation unit 103 in the signal processing device 108 will be described. In this embodiment, the biological information estimation unit 103 also analyzes long-term data on the workload and biological reaction of the learner, and builds an individual database in the storage device 104. In this embodiment, the workload is the answering situation to the question, and the biological reaction is the change in nasal blood flow. Data on the time from the presentation of the question to the answer, the difficulty of the question, and whether the answer of the learner is correct or incorrect is available from the computer 410. Based on the data, the biological information estimation unit 103 calculates the amount of change in the average nasal blood flow of the learner under those conditions. In this embodiment, this amount of change is used as an estimate of the biological information.

[0133] The task suitability determination unit 105 in the signal processing device 108 compares the estimated current change in nasal blood flow with the current change in nasal blood flow actually measured. This makes it possible to grasp the learning situation of the learner. If the answer time is shorter than the expected time and the change in nasal blood flow is smaller than the estimated value, it can be determined that the problem is too easy for the learner's current academic level. In that case, it is possible to determine whether to present a more difficult problem or to proceed to the next learning step. If the answer time is longer than the expected time and the change in nasal blood flow is smaller than the estimated value, it is estimated that the concentration level for learning is insufficient. For this reason, it is possible to call attention to the learner through the computer display or to advise him or her to change his or her mood. If the answer is incorrect despite the large change in nasal blood flow and the learner is concentrating, it is estimated that there is a part of the learning range where understanding is insufficient. In this case, it is possible to present a problem that can detect the part of insufficient understanding, or to change the learning content so that the important part is re-learned. If the change in nasal blood flow is large, the learner is concentrating, and the answer is correct, it is estimated that the learner is progressing smoothly. In that case, it becomes possible to determine whether to present a more difficult question or to move to a more difficult learning content. In this way, the task suitability determination unit 105 provides the learner with learning content according to the task suitability based on the determination result of the learner.

[0134] As described above, by using the system of this embodiment, the degree of understanding or concentration of learning can be judged as the task suitability. The computer can be provided with learning content or a warning display according to the judgment result of the learner's task suitability. This enables efficient learning while maintaining the degree of concentration.

[0135] The present disclosure also includes a computer program that defines the operations executed by the signal processing device 108 shown in Fig. 1A. Such a computer program is stored on a recording medium such as a memory in the task suitability determination system, and causes the signal processing device 108 to execute each of the above-mentioned operations. [Industrial Applicability]

[0136] The present disclosure is applicable to a work suitability determination system that determines a worker's work suitability based on estimated biometric information of the worker and actually measured biometric information of the worker. [Explanation of symbols]

[0137] 100, 200 Work suitability assessment system 101, 201 Authentication device 102 Work sensing device 103 Biological sensing device 104, 204 Storage device 105 Work Properness Assessment Department 106, 205 Biometric information acquisition unit 107, 206 Display device

Claims

1. A computer-implemented processing method, comprising: Acquiring biometric information of a user; obtaining information regarding at least one of acceleration and angular velocity; estimating current biometric information of the user based on the past information, the past biometric information, and the current information stored in a storage device; determining a current mental state of the user based on the acquired current biometric information of the user and the estimated current biometric information of the user; method.

2. In estimating the biometric information of the user, estimating the biometric information of the user based on a correlation between the past information stored in the storage device and the past biometric information, and based on the current information; The method of claim 1.

3. Identifying the user using at least one selected from the group consisting of a personal authentication code, an image of the user, and authentication information acquired by a biometric authentication device; and storing the biometric information, the information, and the identified user in the storage device in association with each other. The method according to claim 1 or 2.

4. The mental state indicates at least one state selected from the group consisting of alertness, tension, attention, fatigue, and stress of the user. The method according to claim 1 or 2.

5. at least one processor; The at least one processor Acquire biometric information of the user; obtaining information regarding at least one of acceleration and angular velocity; Estimating current biometric information of the user based on the past information, the past biometric information, and the current information stored in a storage device; determining a current mental state of the user based on the acquired current biometric information of the user and the estimated current biometric information of the user; Processing unit.

6. Acquiring biometric information of a user; obtaining information regarding at least one of acceleration and angular velocity; estimating current biometric information of the user based on the past information, the past biometric information, and the current information stored in a storage device; determining a current mental state of the user based on the acquired current biometric information of the user and the estimated current biometric information of the user; program.

Images