Information acquisition device, learning device, and information acquisition method

Abstract

This information acquisition device (100) is provided with: an imaging information acquisition unit (102) that acquires imaging information of a subject (P1) placed on a placement surface (B1a); and a wakefulness information acquisition unit (104) that acquires information pertaining to the degree of wakefulness of the subject placed on the placement surface. The imaging information acquisition unit acquires imaging information including the face of the subject placed on the placement surface moving between a first position where the angle between the placement surface and the horizontal plane is equal to a first angle and a second position where the angle between the placement surface and the horizontal plane is equal to a second angle.

Description

Technical Field

[0004] ,

[0006] , , ,

[0005] , , ,

[0001] The present disclosure relates to an information acquisition device, a learning device, and an information acquisition method.

Background Art

[0002] Conventionally, a device for determining the degree of wakefulness of a user has been disclosed that uses a learned model trained with biological information and prognostic data of biological information as teacher data (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, it is desirable that the learned model used in the device as described in Patent Document 1 be trained with teacher data of a sufficient scale. For this reason, there is a demand for a device capable of efficiently collecting information regarding the degree of wakefulness of a subject.

[0005] The present disclosure solves the above problems, and an object thereof is to provide an information acquisition device, a learning device, and an information acquisition method capable of collecting information regarding the degree of wakefulness of a subject.

Means for Solving the Problems

[0006] The information acquisition device according to this disclosure comprises an imaging information acquisition unit that acquires imaging information of a subject mounted on a mounting surface, and an alertness information acquisition unit that acquires information regarding the alertness level of a subject mounted on a mounting surface, wherein the imaging information acquisition unit acquires imaging information including the face of a subject mounted on a mounting surface that moves between a first position where the angle between the mounting surface and the horizontal plane is a first angle and a second position where the angle between the mounting surface and the horizontal plane is a second angle. [Effects of the Invention]

[0007] According to this disclosure, information regarding the level of arousal of subjects can be collected. [Brief explanation of the drawing]

[0008] [Figure 1] A side view showing the information collection system according to Embodiment 1. [Figure 2] A side view showing the information collection system according to Embodiment 1. [Figure 3] A block diagram showing the information collection system according to Embodiment 1. [Figure 4] A block diagram showing an example of the hardware configuration of the information collection device according to Embodiment 1. [Figure 5] A block diagram showing an example of the hardware configuration of the information collection device according to Embodiment 1. [Figure 6] A flowchart illustrating the procedure for collecting information using the information collection system according to Embodiment 1. [Figure 7] A block diagram showing the physical condition estimation system according to Embodiment 2. [Figure 8] A schematic diagram showing the neural network used in the learning unit according to Embodiment 2. [Figure 9] A flowchart illustrating the process for generating a trained model performed by the physical condition estimation device according to Embodiment 2. [Figure 10] A flowchart showing the process related to the estimation of alertness performed by the physical condition estimation device according to Embodiment 2. [Figure 11]A block diagram showing the physical condition estimation system according to Embodiment 3. [Figure 12] A flowchart illustrating the process related to setting thresholds performed by the physical condition estimation device according to Embodiment 3. [Figure 13] A flowchart showing the process related to the estimation of alertness performed by the physical condition estimation device according to Embodiment 3. [Modes for carrying out the invention]

[0009] The embodiments relating to this disclosure will be described in detail below with reference to the drawings. Embodiment 1. First, with reference to Figure 1, the information collection system 1A according to Embodiment 1 will be described. Figure 1 is a side view showing the information collection system 1A according to Embodiment 1. The information collection system 1A is a system for collecting information including biological information from a subject. As shown in Figure 1, the information collection system 1A according to Embodiment 1 includes a mounting platform B1 on which a subject P1 is mounted, an imaging unit C1 for imaging the subject P1, a sensor S1 for acquiring biological information of the subject P1, a drive unit M1 for driving the mounting platform B1, an input unit N1, and an information collection device 100.

[0010] The mounting platform B1 has a mounting surface B1a for mounting the subject P1. For example, as shown in Figure 1, the mounting platform B1 is made up of a bed and is configured to allow the subject P1 to be mounted on the mounting surface B1a in a supine position, with the planar mounting surface B1a and the horizontal plane being substantially parallel. The mounting platform B1 is also supported so as to be rotatable around a pivot axis (not shown) arranged along the mounting surface B1a. In Embodiment 1, the position of the mounting surface B1a where the mounting surface B1a and the horizontal plane are substantially parallel is also referred to as the first position. In Embodiment 1, the position of the mounting surface B1a where the mounting surface B1a and the horizontal plane intersect is also referred to as the second position.

[0011] The imaging unit C1 images the subject P1 mounted on the mounting surface B1a. For example, the imaging unit C1 is composed of a camera having an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor). For example, the imaging unit C1 is held on the mounting platform B1 so that its position relative to the mounting surface B1a is constant. This makes it possible for the imaging unit C1 to image a specific range of the mounting surface B1a regardless of the rotational position of the mounting platform B1. In other words, the imaging unit C1 is able to image a specific range such that the relative position between the mounting surface B1a and the center line L0 of the imaging range is constant, regardless of the rotational position of the mounting platform B1. For example, the imaging unit C1 is positioned so that the face of the subject P1 mounted on the mounting platform B1 is included in the imaging range L1, regardless of the rotational position of the mounting platform B1. In other words, the imaging unit C1 acquires imaging information of the subject P1 mounted on the mounting surface B1a such that the imaging information acquired when the mounting surface B1a is in the second position and the imaging information acquired when the mounting surface B1a is in the first position are imaging information for the same range of the mounting surface B1a. The imaging unit C1 may also be configured to acquire imaging information of the subject P1 mounted on the mounting surface B1a such that the imaging information acquired when the mounting surface B1a is in the second position, or when the mounting surface is moving between the first and second positions, and the imaging information acquired when the mounting surface B1a is in the first position, or when the mounting surface is moving between the second and first positions, are imaging information for the same range of the mounting surface B1a.

[0012] Specifically, the imaging unit C1 is positioned so that, regardless of the rotational position of the mounting platform B1, the face of the subject P1 mounted on the mounting platform B1 is included in the imaging range L1, and so that the subject P1 can be imaged from below (to the left as shown in Figure 1) the subject P1 mounted on the mounting platform B1. The imaging unit C1 outputs the captured imaging information. The imaging unit C1 may be a visible light camera that detects visible light, or an infrared camera that detects infrared light.

[0013] The sensor S1 outputs a signal corresponding to the biological activity of the subject P1. For example, the sensor S1 is arranged to contact the subject P1 mounted on the mounting table B1 to detect the biological activity of the subject P1, and outputs a signal corresponding to the detected biological activity as the biological information of the subject P1. Specifically, the sensor S1 detects the heartbeat of the subject P1 and outputs a signal corresponding to the detected heartbeat as biological information. Note that the biological information of the subject P1 output by the sensor S1 is not limited to the heartbeat, and may be, for example, an electrocardiogram waveform, blood pressure, electroencephalogram, blood oxygen saturation (SPO2), body temperature, skin luminance, skin moisture content, etc., or any one of these pieces of information, or information composed of a plurality of combinations of these. Also, the sensor S1 may be an electrode for detecting the current flowing through the human body, a sensor for detecting infrared rays, a sensor for detecting radio waves, or a sensor using other physical phenomena.

[0014] The drive unit M1 drives the mounting table B1 so that the mounting table B1 rotates about the rotation axis. For example, the drive unit M1 is composed of a motor and a speed reducer, and rotates the mounting table B1 by the supply of current and a control signal from the information collection device 100. In other words, the drive unit M1 drives the mounting table B1 so that the mounting surface B1a moves between a first position where the mounting surface B1a is substantially parallel to the horizontal plane and a second position where the mounting surface B1a intersects the horizontal plane. Also, in other words, the drive unit M1 drives the mounting table B1 so that the mounting surface B1a moves between a first position where the angle between the mounting surface B1a and the horizontal plane is a first angle and a second position where the angle between the mounting surface B1a and the horizontal plane is a second angle.

[0015] FIG. 2 is a side view showing the information collection system 1A in a state where the mounting table B1 is rotated and the mounting surface B1a is located at the second position. In this state, the subject P1 is mounted on the mounting surface B1a in a standing position where the head is located above the feet. Thus, the information collection system 1A is configured to be able to perform a head-up tilt test that induces reflex syncope of the subject P1 by changing the subject P1 mounted on the mounting table B1 from a lying position to a standing position and observe the presence or absence of reflex syncope.

[0016] The input unit N1 receives an input operation by an operator of the information collection system 1A. For example, the input unit N1 receives an input operation for inputting information regarding the arousal level of the subject P1 mounted on the mounting surface B1a. Specifically, the input unit N1 receives an input operation indicating the presence or absence of syncope of the subject P1 mounted on the mounting surface B1a. For example, the input unit N1 is constituted by an input device such as a switch, a touch panel, a keyboard, or a mouse that receives an input operation. The input unit N1 outputs a signal corresponding to the input operation. For example, when the operator determines that the subject P1 has lost consciousness, the operator performs an input operation indicating the syncope of the subject P1 on the input unit N1.

[0017] Figure 3 is a block diagram showing the information acquisition system 1A according to Embodiment 1. As shown in Figure 3, the information acquisition device 100, as an information acquisition device, comprises a control unit 101, an imaging information acquisition unit 102, a biological information acquisition unit 103, and an awakening information acquisition unit 104, and is electrically connected to a drive unit M1, an imaging unit C1, a sensor S1, and an input unit N1. The control unit 101 controls the drive of the drive unit M1 by outputting a control signal to the drive unit M1. The imaging information acquisition unit 102 acquires imaging information from the imaging unit C1. Specifically, the imaging unit C1 acquires imaging information of the subject P1 mounted on the mounting surface B1a. The biological information acquisition unit 103 acquires biological information of the subject P1 from the sensor S1. In other words, the biological information acquisition unit 103 acquires biological information of the subject P1 regardless of the imaging information acquired by the imaging information acquisition unit 102. The awakening information acquisition unit 104 acquires signals from the input unit N1. For example, the arousal information acquisition unit 104 acquires a signal corresponding to an input operation by the operator that provides information regarding the arousal level of subject P1. In other words, the arousal information acquisition unit 104 acquires information regarding the arousal level of subject P1.

[0018] Next, the hardware configuration of the information acquisition device 100 according to Embodiment 1 will be described with reference to Figures 4 and 5. Figure 4 is a block diagram showing an example of the hardware configuration of the information acquisition device 100 according to Embodiment 1, and Figure 5 is a block diagram showing an example of a hardware configuration of the information acquisition device 100 according to Embodiment 1 that is different from Figure 4. For example, as shown in Figure 4, the information acquisition device 100 has a processor 100a, a memory 100b, and an I / O port 100c, and is configured so that the processor 100a reads and executes a program stored in the memory 100b. The memory 100b may be a non-volatile or volatile semiconductor memory such as RAM, ROM, flash memory, EPROM, or EEPROM. The memory 100b may also be a magnetic disk, flexible disk, optical disk, compact disk, minidisc, DVD, etc. Furthermore, the memory 100b may be an HDD or SSD.

[0019] Furthermore, as shown in Figure 5, for example, the information acquisition device 100 has dedicated hardware, namely a processing circuit 100d and an I / O port 100c. The processing circuit 100d is composed of, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a system LSI (Large-Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the information acquisition device 100 is realized by these processors 100a or the dedicated hardware, namely the processing circuit 100d, executing a program which is software, firmware, or a combination of software and firmware.

[0020] Next, with reference to Figure 6, the procedure for collecting information using the information collection system 1A according to Embodiment 1 will be described. Figure 6 is a flowchart showing the procedure for collecting information using the information collection system 1A according to Embodiment 1. As shown in Figure 6, first, when collecting information, the operator confirms that the mounting surface B1a of the mounting platform B1 is parallel to the horizontal plane, and places the subject P1 on the mounting surface B1a so that it lies down (step ST1). In other words, the operator places the subject P1 on the mounting surface B1a which is in the first position. In this step, the operator places the subject P1 on the mounting surface B1a in a position in which the subject P1 can collect information by the information collection device 100. For example, in this step, the operator secures the subject P1 to the mounting surface B1a with a belt (not shown).

[0021] Step ST1 is performed, and the operator attaches the sensor S1 to the subject P1 and starts acquiring the subject P1's biological information using the information acquisition device 100 (Step ST2). In this step, the information acquisition device 100 starts acquiring biological information from the subject P1, which is mounted on the mounting surface B1a located in the first position, using the sensor S1. For example, the information acquisition device 100 acquires biological information from the subject P1 that changes over time using the sensor S1.

[0022] When the information acquisition device 100 completes the process in step ST2, the operator starts acquiring imaging information of subject P1 using the information acquisition device 100 (step ST3). In this step, the information acquisition device 100 starts acquiring imaging information of subject P1 captured by the imaging unit C1. For example, in this step, the operator starts the imaging unit C1 and the information acquisition device 100, and starts acquiring imaging information of subject P1 by the imaging unit C1. Alternatively, for example, the information acquisition device 100 acquires imaging information of subject P1 as video information that changes over time using the imaging unit C1. Furthermore, when the information acquisition device 100 completes the process in step ST3, the control unit 101 starts timing.

[0023] The information gathering device 100 determines whether a predetermined time 1 has elapsed since the processing in step ST3 (step ST4). In this process, the control unit 101 compares the time being measured with a preset predetermined time 1 to determine whether the time since the processing in step ST3 has exceeded the predetermined time 1. For example, the predetermined time 1 is a preset time of 5 minutes or more and 10 minutes or less. If the predetermined time 1 has not elapsed since the processing in step ST3 (NO in step ST4), the information gathering device 100 waits for subsequent processing.

[0024] If a predetermined time 1 has elapsed since the information acquisition device 100 performed the processing in step ST3 (YES in step ST4), the information acquisition device 100 changes the inclination of the mounting platform B1 to a second angle (step ST5). In this process, the control unit 101 drives the drive unit M1 to rotate the mounting platform B1 so that the angle between the mounting surface B1a and the horizontal plane is the second angle and the subject P1 is in an upright position. For example, the second angle is a preset angle of 60 degrees or more and 80 degrees or less.

[0025] When the information gathering device 100 performs step ST5, the operator determines whether or not subject P1 is unconscious by observing subject P1 (step ST6). For example, in this step, the operator determines whether or not subject P1 is unconscious based on the external appearance of subject P1, subject P1's response to external stimuli, and biological information acquired by sensor S1. For example, the operator determines whether or not subject P1 is unconscious based on the following: weakness of subject P1, loss of response of subject P1 to external stimuli, decrease in subject P1's vital activity, etc.

[0026] In step ST6, if it is determined that subject P1 is not fainted (NO in step ST6), in other words, if the operator has not performed an input operation to the input unit N1 indicating that subject P1 is fainted in step ST6, the information gathering device 100 determines whether a predetermined time 2 has elapsed since the processing in step ST3 (step ST7). In this process, the control unit 101 compares the time being measured with a preset predetermined time 2 to determine whether the time since the processing in step ST3 has exceeded the predetermined time 2. For example, the predetermined time 2 is a preset time of 50 minutes or more and 60 minutes or less. If the predetermined time 2 has not elapsed since the processing in step ST3 (NO in step ST7), the information gathering device 100 returns to the process in step ST6.

[0027] If it is determined in step ST6 that subject P1 has fainted (YES in step ST6), and if the timing time has elapsed for a predetermined time 2 in step ST7 (YES in step ST7), the information acquisition device 100 changes the inclination of the mounting platform B1 to the first angle (step ST8). In this step, the operator inputs to the input unit N1 that subject P1 has fainted. As a result, the awakening information acquisition unit 104 acquires information regarding whether or not subject P1, mounted on the mounting surface B1a, has fainted. Also in this step, the control unit 101 rotates the mounting platform B1 by driving the drive unit M1 so that the angle between the mounting surface B1a and the horizontal plane becomes the first angle, i.e., subject P1 is in a supine position. Furthermore, once the information gathering device 100 has completed the processing in step ST8, the control unit 101 starts timing.

[0028] When the information gathering device 100 performs step ST8, it determines whether a predetermined time 3 has elapsed since the processing in step ST8 (step ST8). In this process, the control unit 101 compares the time being measured with a preset predetermined time 3 to determine whether the time since the processing in step ST8 has exceeded the predetermined time 3. For example, the predetermined time 3 is a preset time of 5 minutes or more and 10 minutes or less. If the predetermined time 3 has not elapsed since the processing in step ST8 (NO in step ST9), the information gathering device 100 waits for subsequent processing.

[0029] If a predetermined time 3 has elapsed since the information collection device 100 performed the processing in step ST8 (YES in step ST9), the information collection device 100 terminates the acquisition of imaging information by the imaging unit C1 (step ST10) and terminates the acquisition of biological information by the sensor S1 (step ST11). When the information collection device 100 completes steps ST10 and ST11, the information collection of subject P1 using the information collection system 1A is completed. The operator performs steps ST1 to ST11 for multiple subjects using the information collection system 1A to collect information from the imaging information of subjects whose level of alertness is to be determined, which can be used as training data to generate a trained model for determining the level of alertness of the subject.

[0030] For example, if subject P1 is determined to be in syncope during step ST6, and the imaging information of subject P1 before syncope contains characteristic information, it becomes possible to generate a trained model for predicting syncope from the subject's imaging information by generating a trained model using training data that includes data labeled as syncope containing the characteristic information and data labeled as non-syncope that does not contain the characteristic information.

[0031] Furthermore, for example, if subject P1 is determined to be in syncope during the processing of step ST6, and the imaging information of subject P1 after syncope contains characteristic information, it becomes possible to generate a trained model for determining whether or not a subject is in syncope based on the imaging information of the subject by generating training data that includes data labeled as syncope containing the characteristic information and data labeled as non-syncope that does not contain the characteristic information.

[0032] Furthermore, for example, if subject P1 is determined to be in syncope during the processing of step ST6, and the changes in subject P1's imaging information before and after syncope contain characteristic information, a trained model can be generated using training data that includes data labeled as syncope containing such characteristic information and data labeled as non-syncope that does not contain such characteristic information. This makes it possible to generate a trained model for predicting syncope in a subject or determining whether or not the subject is in syncope based on the subject's imaging information. The operator may also use the information collection system 1A to perform steps ST1 to ST11 multiple times for a single subject to collect information from the imaging information of the subject whose level of alertness is to be determined, which can be used as training data for generating a trained model for determining the alertness level of the subject.

[0033] As described above, the information collection system 1A according to Embodiment 1 includes an imaging information acquisition unit 102 that acquires imaging information of a subject P1 mounted on the mounting surface B1a, and an arousal information acquisition unit 104 that acquires information regarding the arousal level of the subject mounted on the mounting surface B1a. The imaging information acquisition unit 102 acquires imaging information, including the face of the subject P1 mounted on the mounting surface B1a, which moves between a first position where the angle between the mounting surface B1a and the horizontal plane is a first angle, and a second position where the angle between the mounting surface B1a and the horizontal plane is a second angle. In this way, the information collection system 1A can induce a change in the arousal level of subject P1 by changing the angle between the mounting surface B1a on which subject P1 is mounted and the horizontal plane, and can acquire imaging information of subject P1 when the arousal level changes, making it possible to collect information that can be used as training data for generating a trained model that determines the arousal level of a subject based on the imaging information.

[0034] For example, in a vehicle equipped with an imaging unit that images the occupants, when estimating the occupants' physical condition using the imaging information captured by the imaging unit, it is conceivable to perform the estimation using a trained model generated with the occupants' imaging information when there is a change in their physical condition as training data. However, it is rare for occupants to experience changes in their physical condition, such as fainting, while actually riding in a vehicle, making it difficult to collect a sufficient amount of imaging information. The information collection system 1A according to Embodiment 1 makes it possible to induce changes in the subject's level of alertness in an environment that simulates being on a vehicle, without actually collecting imaging information from the occupants while they are on the vehicle, and to acquire imaging information of the subject when there is a change in their level of alertness.

[0035] For example, when generating a trained model to estimate changes in a driver's alertness based on imaging information from an in-vehicle camera that images the driver, it becomes possible to collect information with the position and imaging range of the imaging unit C1 of the information collection system 1A pre-adjusted to the position and imaging range of the in-vehicle camera of the vehicle using the trained model. By generating a trained model using the information collected in this way, it becomes possible to improve the reliability of the trained model. Specifically, if the in-vehicle camera of the vehicle using the trained model is positioned to image the driver from the left front and diagonally below, it becomes possible to improve the reliability of the generated trained model by holding the imaging unit C1 of the information collection system 1A on the mounting platform B1 so that it images subject P1 from the left front and diagonally below, regardless of the angle between the mounting surface B1a and the horizontal plane. Furthermore, for example, if the in-vehicle camera of a vehicle using a trained model is positioned to image the driver from the right front and diagonally above, the reliability of the generated trained model can be improved by holding the imaging unit C1 of the information collection system 1A on the mounting platform B1 so that it images subject P1 from the left front and diagonally below, regardless of the angle between the mounting surface B1a and the horizontal plane.

[0036] Furthermore, the information collection system 1A according to Embodiment 1 includes a biometric information acquisition unit 103 that acquires biometric information of subject P1 using a sensor S1. This makes it possible to improve the accuracy of feature extraction, the reliability of feature extraction from image information, and the efficiency of creating training data when, for example, features related to subject P1's biometric information are extracted from imaging information, and the extracted features are used as training data to generate a trained model that determines the subject's level of alertness from imaging information. Specifically, when extracting features related to subject P1's heart rate from imaging information, and generating a trained model that determines the subject's level of alertness from imaging information using the extracted features are used as training data, acquiring biometric information related to subject P1's heart rate using sensor S1 along with the imaging information makes it possible to improve the accuracy of extraction when extracting features related to subject P1's heart rate from imaging information, the efficiency of creating training data, and the reliability of the generated trained model.

[0037] It is desirable, but not limited to, that the biometric information of subject P1 from which features are extracted from imaging data and the biometric information acquired by sensor S1 contain the same or corresponding information. The biometric information of subject P1 from which features are extracted from imaging data and the biometric information acquired by sensor S1 may both contain information that affects subject P1's level of arousal. For example, the biometric information of subject P1 from which features are extracted from imaging data may be changes in subject P1's facial expression, and the biometric information acquired by sensor S1 may be information related to subject P1's heart rate. Furthermore, corresponding information means information that is closely related to each other, such as heart rate and RR interval, mean blood pressure and diastolic or systolic blood pressure, skin color and body temperature, etc.

[0038] The imaging information acquisition unit 102 acquires imaging information including the face of subject P1 mounted on the mounting surface B1a located at least at the second position. In the head-up tilt test, changes in the level of alertness of subject P1, including syncope, often occur when subject P1 is standing and are often accompanied by changes in the appearance around the face. Therefore, by having the imaging information acquisition unit 102 acquire imaging information including the face of subject P1 in a standing position, it becomes easier to acquire imaging information containing characteristic information related to changes in the level of alertness, thereby improving the efficiency of creating training data and the reliability of the generated trained model.

[0039] In Embodiment 1, the information gathering device 100 is configured such that the awakening information acquisition unit 104 acquires information indicating that subject P1 has fainted in response to an input operation to the input unit N1, but it is not limited to this. The awakening information acquisition unit only needs to be configured to acquire information regarding the level of awakening of subject P1 mounted on the mounting surface B1a. For example, the awakening information acquisition unit may be configured to acquire information indicating that the subject has not yet completely lost consciousness but has a reduced level of awakening in response to an input operation to the input unit, or it may be configured to acquire information indicating a reduced level of awakening other than fainting or a state in which there are signs of a decrease in the level of awakening.

[0040] Furthermore, in Embodiment 1, the information collection device 100 is configured such that the imaging unit C1 acquires imaging information as video information of the subject P1 that changes over time, and the imaging information acquisition unit 102 acquires this imaging information. However, the device is not limited to this configuration. The imaging information acquisition unit only needs to be configured to acquire imaging information of the subject P1 mounted on the mounting surface B1a. For example, the imaging information acquisition unit may be configured to acquire imaging information, which is still image information at a specific single moment (timing), from the imaging unit C1, or it may be configured to acquire imaging information, which is still image information at multiple moments when collecting information from a single subject.

[0041] Furthermore, in Embodiment 1, the information acquisition device 100 is configured such that the imaging unit C1 acquires imaging information of the subject P1 from a supine position to a standing position and then back to a supine position, and the imaging information acquisition unit 102 acquires this information. However, the device is not limited to this configuration. The imaging information acquisition unit only needs to be configured to acquire imaging information including the face of the subject P1 mounted on the mounting surface B1a at any point after the subject P1 has changed from a supine position to a standing position. In other words, the imaging information acquisition unit only needs to be configured to acquire imaging information including the face of the subject P1 mounted on the mounting surface B1a at any point after the position of the mounting surface B1a has changed from at least a first position to a second position. For example, the imaging information acquisition unit may start acquiring imaging information from the imaging unit C1 when the position of the mounting surface B1a changes from a first position to a second position, or it may be configured to stop acquiring imaging information when the position of the mounting surface B1a changes from a second position to a first position.

[0042] Furthermore, in Embodiment 1, the information gathering device 100 is configured such that the control unit 101 controls the angle between the mounting surface B1a and the horizontal plane so that the mounting surface B1a moves between a first position where the angle between the mounting surface B1a and the horizontal plane is approximately parallel, a second position where the angle between the mounting surface B1a and the horizontal plane is approximately parallel so that the subject P1 is in a supine position, and a third position where the mounting surface B1a and the horizontal plane intersect so that the subject P1 is in an upright position. However, the device is not limited to this configuration. The control unit only needs to control the rotational position of the mounting platform B1 so as to induce a change in the arousal level of the subject P1 by changing the posture of the subject P1 mounted on the mounting surface B1a. For example, the mounting surface on which the subject P1 is mounted does not have to be a flat surface, but may be composed of multiple intersecting surfaces, a curved surface, or a soft material that is easily deformed by external forces. Specifically, the control unit may be configured to control the position of the mounting surface between a first position for mounting a subject P1 in a supine position and a second position for mounting a subject P1 in a seated position. The mounting surface may also consist of a first surface that contacts and supports the subject P1's back when the subject P1 is in a supine position, and a second surface that contacts and supports the subject P1's soles when the subject P1 is in a standing position.

[0043] Furthermore, in Embodiment 1, the information gathering device 100 is configured such that the biological information acquisition unit 103 acquires biological information of subject P1 based on input signals from sensor S1, but is not limited to this. For example, the biological information acquisition unit may be configured to acquire biological information of subject P1 based on input signals from input unit N1, or it may be configured to acquire biological information of subject P1 based on input signals from sensor S1 and input unit N1. Specifically, the biological information acquisition unit may be configured to acquire biological information of subject P1 acquired by observation of subject P1 by a sensor (not shown) or by an operator, by an operator performing an input operation to the input unit. Examples of biological information of subject P1 acquired by observation of subject P1 by an operator include, for example, information on facial expressions, degree of eye opening, skin color, pupil diameter, degree of mouth opening, presence or absence of muscle weakness, and other characteristics of subject P1's appearance. Furthermore, the information input from the input unit to the information gathering device 100 is not limited to the above, and may include other information about the subject, such as the subject's age, medical history, height, weight, and gender.

[0044] Furthermore, in Embodiment 1, the information acquisition system 1A is configured such that the imaging unit C1 is held on the mounting base B1 so that the relative position between the imaging unit C1 and the mounting surface B1a remains constant, but it is not limited to this. The information acquisition system 1A only needs to be configured to acquire imaging information for a specific range of the mounting surface B1a regardless of the position of the mounting surface B1a. For example, the information acquisition system may include an imaging position control unit (not shown) that controls the position of the imaging unit so that the imaging unit moves in conjunction with the rotation of the mounting base, thereby enabling the acquisition of imaging information for a specific range of the mounting surface B1a regardless of the position of the mounting surface B1a.

[0045] Embodiment 2. Next, with reference to Figures 7 to 10, the physical condition estimation system 2A according to Embodiment 2 will be described. Figure 7 is a block diagram of the physical condition estimation system 2A according to Embodiment 2. As shown in Figure 7, the physical condition estimation system 2A according to Embodiment 2 comprises the information collection device 100 according to Embodiment 1 and the physical condition estimation device 200. The explanation of the information collection device 100 will be omitted as it overlaps with that of Embodiment 1.

[0046] The physical condition estimation device 200, which functions as both an alertness estimation device and a learning device, generates a trained model using information collected by the information collection device 100, and estimates the alertness level of a subject from the subject's imaging information using the trained model. As shown in Figure 7, the physical condition estimation device 200 comprises a feature extraction unit 201, a learning unit 202, a model storage unit 203, and a state estimation unit 204.

[0047] The feature extraction unit 201 extracts features from the subject's imaging information and biological information collected by the information acquisition device 100. For example, the feature extraction unit 201 may be configured to extract one of the following features from the subject's imaging information and biological information collected by the information acquisition device 100: RR interval, heart rate, diastolic blood pressure, systolic blood pressure, mean blood pressure, degree of eye opening, degree of mouth opening, facial muscle movement, pupil change rate, brightness change amount, etc., or it may be configured to extract multiple of these features.

[0048] The learning unit 202, acting as a trained model generation unit, generates a trained model using the features extracted by the feature extraction unit 201, along with information on the subject's level of arousal associated with the imaging and biological information from which the features were extracted, as training data. The learning algorithm used by the learning unit 202 to generate the trained model can be any known supervised learning algorithm. Below, we will describe the case where the learning unit 202 applies a neural network as an example of a learning algorithm.

[0049] Figure 8 is a schematic diagram showing the neural network used in the learning unit 202 according to Embodiment 2. The learning unit 202 learns the relationship between input imaging information and the arousal level of the subject (subject) related to said imaging information, for example, by so-called supervised learning according to a neural network model. Here, supervised learning is a method in which training data, which is a set of input and result (label) data, is provided to the learning unit 202 as a learning device, thereby learning features in that training data and inferring the result from the input.

[0050] A neural network consists of an input layer made up of multiple neurons, a hidden layer (intermediate layer) made up of multiple neurons, and an output layer made up of multiple neurons. The hidden layer can be one or more layers. For example, in a three-layer neural network as shown in Figure 8, when multiple inputs are input to the input layer (X1-X3), the values ​​are multiplied by weights W1 (w11-w16) and input to the hidden layer (Y1-Y2), and the result is further multiplied by weights W2 (w21-w26) and output from the output layer (Z1-Z3).

[0051] The output result varies depending on the values ​​of weights W1 and W2. In Embodiment 2, the neural network learns whether the subject is in a state of decreased arousal or a state in which there are signs of decreased arousal, through so-called supervised learning, according to training data created based on the features extracted by the feature extraction unit 201. For example, the neural network learns whether the subject is in a state of fainting or a state in which there are signs of fainting, according to training data created based on the features extracted by the feature extraction unit 201. In this way, the neural network learns by adjusting weights W1 and W2 so that the result output from the output layer after inputting the extracted features into the input layer approaches the features that indicate the subject is in a state of decreased arousal or a state in which there are signs of decreased arousal. The learning unit 202 generates a trained model by performing the above learning and stores the generated trained model in the model storage unit 203, which acts as a memory unit.

[0052] The state estimation unit 204, acting as an inference unit, performs inference regarding the arousal level of a subject based on the trained model stored in the model storage unit 203 and the imaging information of the subject whose arousal level is to be estimated. For example, the state estimation unit 204 estimates whether the subject is in a state of decreased arousal or a state in which there are signs of decreased arousal, based on the trained model stored in the model storage unit 203 and the imaging information of the subject whose arousal level is to be estimated. The features used by the state estimation unit 204 for estimation may be imaging information or biometric information acquired from the information collection device 100, or imaging information or biometric information acquired from an imaging device or biometric information acquisition device (not shown). The state estimation unit 204 outputs the estimation result to the outside.

[0053] The physical condition estimation device 200 may have a processor, memory, and I / O ports, and may be configured so that the processor reads and executes a program stored in memory, or it may have a dedicated hardware processing circuit and I / O ports, and the processing circuit may be configured to execute a program. The hardware configuration of the physical condition estimation device 200 is the same as that of the information collection device 100 according to Embodiment 1, so a description is omitted.

[0054] Next, with reference to Figures 9 and 10, the processing performed by the physical condition estimation device 200 according to Embodiment 2 will be described. Figure 9 is a flowchart showing the processing performed by the physical condition estimation device 200 according to Embodiment 2 for the generation of a trained model. First, when processing is started, the physical condition estimation device 200 acquires biological information collected by the information collection device 100 (step ST21) and acquires imaging information collected by the information collection device 100 (step ST22). The physical condition estimation device 200 may be configured to acquire information from the information collection device 100 while electrically connected to the information collection device 100 so that information can be communicated with the information collection device 100, or it may be configured to acquire information from the information collection device 100 via a recording medium.

[0055] After performing steps ST21 and ST22, the physical condition estimation device 200 extracts features from the acquired information (step ST23). In step ST23, the physical condition estimation device 200 may be configured to extract features related to the subject's biological information from the imaging information. For example, the physical condition estimation device 200 may be configured to extract features of the imaging information corresponding to the biological information acquired by sensor S1, based on the imaging information and the biological information acquired by sensor S1 when the imaging information was acquired by imaging unit C1, or it may be configured to extract features related to the subject's biological information based on characteristic changes in the appearance of the human body corresponding to known biological activity. When extracting features from the imaging image, the physical condition estimation device 200 can use known algorithms.

[0056] After performing the process in step ST23, the physical condition estimation device 200 generates a trained model based on the extracted features and the information on the subject's level of alertness associated with the imaging information and biological information from which the features were extracted (step ST24). In this process, the learning unit 202 can generate a trained model using known machine learning algorithms such as the neural network described above. After performing the process in step ST24, the physical condition estimation device 200 stores the generated trained model in the model storage unit 203 and terminates the process (step ST25).

[0057] Next, the process related to the estimation of alertness performed by the physical condition estimation device 200 according to Embodiment 2 will be explained using as an example the process by which the physical condition estimation device 200 estimates the alertness of a subject based on imaging information and biological information acquired from an imaging device and a biological information acquisition device (not shown). Figure 10 is a flowchart showing the process related to the estimation of alertness performed by the physical condition estimation device 200 according to Embodiment 2. In Embodiment 2, the state estimation unit 204 constitutes an imaging information acquisition unit that acquires imaging information including imaging of the subject's face.

[0058] First, upon starting the process, the health condition estimation device 200 acquires the subject's biological information from a biological information acquisition device (not shown) (step ST31) and acquires imaging information, including an image of the subject's face, from an imaging device (not shown) (step ST32). For example, the health condition estimation device 200 acquires the biological information and imaging information while being electrically connected to the imaging device (not shown) and the biological information acquisition device in a state where information can be communicated. For example, the imaging device (not shown) is an imaging device that images the occupants of a vehicle, and the biological information acquisition device (not shown) is a sensor installed in the vehicle for acquiring the occupants' biological information.

[0059] After performing steps ST31 and ST32, the physical condition estimation device 200 extracts features from the acquired information (step ST33). In this process, the feature extraction unit 201 extracts features from the acquired information according to the algorithm used to generate the trained model. For example, the feature extraction unit 201 extracts features related to the subject's biological information from the acquired imaging information.

[0060] After performing the process in step ST33, the physical condition estimation device 200 refers to the information stored in the model storage unit 203 and loads the trained model (step ST34). If the learning unit 202 is configured to generate multiple trained models, the physical condition estimation device 200 may be configured to select one of the trained models based on the extracted features.

[0061] After completing step ST34, the physical condition estimation device 200 estimates the subject's level of alertness using the state estimation unit 204, based on the extracted features and the trained model stored in the model memory unit 203 (step ST35). After completing step ST35, the physical condition estimation device 200 outputs information regarding the estimated level of alertness of the subject and terminates the process (step ST36).

[0062] As described above, the physical condition estimation system 2A according to Embodiment 2 includes a model storage unit 203 that stores a trained model generated using training data that includes feature quantities extracted from imaging information including an image of the subject's face and information regarding the subject's level of alertness, and a state estimation unit 204 that acquires imaging information including an image of the subject's face. The state estimation unit 204 performs inference regarding the subject's level of alertness based on the trained model stored in the model storage unit 203 and the acquired imaging information. This makes it possible to estimate the subject's level of alertness based on the subject's imaging information, making it easier to manage the subject's physical condition than before.

[0063] Furthermore, the physical condition estimation device 200 according to Embodiment 2 includes a feature extraction unit 201 that extracts features from imaging information acquired by the information collection device 100, and a learning unit 202 that generates a trained model based on information about the subject's level of alertness acquired by the information collection device 100 and the features extracted by the feature extraction unit 201. As a result, the physical condition estimation device 200 can generate a trained model based on a sufficient amount of information acquired by the information collection device 100, thereby improving the reliability of the trained model.

[0064] In Embodiment 2, the physical condition estimation system 2A is configured to generate a trained model based on the subject's biological information acquired by sensors, etc., extracted features, features extracted from the subject's imaging information, and information regarding the subject's level of alertness, but is not limited to this. The physical condition estimation system only needs to be configured to generate a trained model based on at least features extracted from imaging information and information regarding the subject's level of alertness. For example, the physical condition estimation system 2A may be configured not to acquire biological information other than imaging information, or it may be configured to estimate the subject's level of alertness without acquiring biological information other than imaging information.

[0065] Embodiment 3. Next, with reference to Figures 11 to 13, the physical condition estimation system 3A according to Embodiment 3 will be described. Figure 11 is a block diagram of the physical condition estimation system 3A according to Embodiment 3. As shown in Figure 11, the physical condition estimation system 3A according to Embodiment 3 comprises an information collection device 100 according to Embodiment 1 and a physical condition estimation device 300. The physical condition estimation device 300 according to Embodiment 3 differs from the physical condition estimation device 200 according to Embodiment 2 in that it can output physical condition estimation results without using a trained model. However, some of the configurations are the same as those of the physical condition estimation device 200 according to Embodiment 2, and the same reference numerals are used for the same configurations as in Embodiment 2, and their description is omitted.

[0066] The physical condition estimation device 300, which serves as both an alertness level estimation device and a threshold setting device, is a device that estimates the alertness level of a subject based on the subject's imaging information using information collected by the information collection device 100. As shown in Figure 11, the physical condition estimation device 300 comprises a feature extraction unit 201, a threshold adjustment unit 302, and a determination unit 303.

[0067] The threshold adjustment unit 302, acting as a threshold setting unit, sets a threshold for estimating the subject's level of alertness using the features extracted by the feature extraction unit 201. For example, if the feature extracted by the feature extraction unit 201 is a feature indicating heart rate, and the threshold adjustment unit 302 determines that the subject's level of alertness has decreased when the change in heart rate over a predetermined period (e.g., 10 seconds) is 20 bpm or more based on that feature, it sets 20 bpm as the threshold. The threshold adjustment unit 302 also adjusts the pre-set threshold to an appropriate value based on the features extracted by the feature extraction unit 201. For example, if the threshold for estimating the level of alertness is set to a value that does not provide sufficient accuracy in estimating the level of alertness, the threshold adjustment unit 302 adjusts the threshold to improve the accuracy of estimating the level of alertness, based on the features extracted by the feature extraction unit 201 and information about the subject's level of alertness related to the extracted features.

[0068] The determination unit 303, acting as an estimation unit, determines the level of alertness of the subject using a threshold set by the threshold adjustment unit 302. For example, the determination unit 303 estimates that the subject is in a state of decreased alertness or shows signs of decreased alertness if the feature quantities extracted by the feature quantity extraction unit 201 from the subject's imaging information or biological information exceed the threshold set by the threshold adjustment unit 302. The feature quantities used by the determination unit 303 for estimation may be imaging information or biological information acquired from the information collection device 100, or imaging information or biological information acquired from an imaging device or biological information acquisition device (not shown). The determination unit 303 outputs the estimation result to the outside.

[0069] The physical condition estimation device 300 may have a processor, memory, and I / O ports, and may be configured so that the processor reads and executes a program stored in memory, or it may have a dedicated hardware processing circuit and I / O ports, and the processing circuit may be configured to execute a program. The hardware configuration of the physical condition estimation device 300 is the same as that of the information collection device 100 according to Embodiment 1, so a description is omitted.

[0070] Next, with reference to Figures 12 and 13, the process related to setting thresholds performed by the physical condition estimation device 300 according to Embodiment 3 will be described. Figure 12 is a flowchart showing the process related to setting thresholds performed by the physical condition estimation device 300 according to Embodiment 3. First, when processing is started, the physical condition estimation device 300 acquires biological information collected by the information collection device 100 (step ST21) and acquires imaging information collected by the information collection device 100 (step ST22). The physical condition estimation device 300 may be configured to acquire information from the information collection device 100 while electrically connected to the information collection device 100 in a state where information communication is possible, or it may be configured to acquire information from the information collection device 100 via a recording medium.

[0071] After completing steps ST21 and ST22, the physical condition estimation device 300 extracts features from the acquired information (step ST23). After completing step ST23, the physical condition estimation device 300 sets a threshold for estimating the subject's level of alertness (step ST44). In this process, the threshold adjustment unit 302 sets a new threshold if no threshold has been set in advance, and if a threshold has been set in advance, it sets a new threshold by adjusting the threshold to a more appropriate value. After completing step ST44, the physical condition estimation device 300 stores the new threshold in a memory unit (not shown) (step ST45).

[0072] Next, the process for estimating the level of alertness performed by the physical condition estimation device 300 according to Embodiment 3 will be explained using as an example the process by which the physical condition estimation device 300 estimates the level of alertness of the subject based on imaging information and biological information acquired from an imaging device and a biological information acquisition device (not shown). Figure 13 is a flowchart showing the process for estimating the level of alertness performed by the physical condition estimation device 300 according to Embodiment 3. In Embodiment 3, the determination unit 303 constitutes an imaging information acquisition unit that acquires imaging information including imaging of the subject's face.

[0073] First, upon starting the process, the health condition estimation device 300 acquires the subject's biological information from a biological information acquisition device (not shown) (step ST31) and acquires imaging information, including an image of the subject's face, from an imaging device (not shown) (step ST32). For example, the health condition estimation device 300 acquires the biological information and imaging information while electrically connected to the imaging device (not shown) and the biological information acquisition device in a state where information communication is possible. For example, the imaging device (not shown) is an imaging device that images the occupants of a vehicle, and the biological information acquisition device (not shown) is a sensor installed in the vehicle for acquiring the occupants' biological information.

[0074] After performing steps ST31 and ST32, the physical condition estimation device 300 extracts features from the acquired information (step ST33). In this process, the feature extraction unit 201 extracts features from the acquired information according to a set threshold. For example, the feature extraction unit 201 extracts features related to the subject's biological information from the acquired imaging information.

[0075] After performing the process in step ST33, the physical condition estimation device 300 refers to the information stored in a memory unit (not shown) and reads the set threshold (step ST54). If the threshold adjustment unit 302 is configured to set multiple thresholds, the physical condition estimation device 300 may be configured to select one of the thresholds based on the extracted feature quantities.

[0076] After completing step ST54, the physical condition estimation device 300 uses the determination unit 303 to estimate the subject's level of alertness based on the extracted features and the threshold values ​​stored in the memory unit (step ST55). After completing step ST55, the physical condition estimation device 300 outputs information regarding the estimated level of alertness of the subject and terminates the process (step ST56).

[0077] As described above, the physical condition estimation system 3A according to Embodiment 3 includes a threshold adjustment unit 302 that sets a threshold based on feature quantities extracted from imaging information including an image of the subject's face and information regarding the subject's level of alertness, and a determination unit 303 that acquires imaging information including an image of the subject's face. The determination unit 303 estimates the subject's level of alertness based on the threshold set by the threshold adjustment unit 302 and the acquired imaging information. This makes it possible to estimate the subject's level of alertness based on the subject's imaging information, making it easier to manage the subject's physical condition than before.

[0078] Furthermore, the physical condition estimation device 300 according to Embodiment 3 includes a feature extraction unit 201 that extracts features from imaging information acquired by the information collection device 100, and a threshold adjustment unit 302 that sets a threshold based on the information about the subject's level of alertness acquired by the information collection device 100 and the features extracted by the feature extraction unit 201. As a result, the physical condition estimation device 300 can estimate the subject's level of alertness based on a threshold set based on a sufficient amount of information acquired by the information collection device 100, thereby improving the reliability of the alertness estimation result.

[0079] Furthermore, this disclosure allows for free combination of each embodiment, modification of any component of each embodiment, or omission of any component in each embodiment. [Industrial applicability]

[0080] The information collection system described in this disclosure can be used, for example, in the production of a health condition estimation device for estimating the health condition of occupants of vehicles such as cars. Furthermore, the health condition estimation device described in this disclosure can be used, for example, in a vehicle capable of estimating the health condition of its occupants. [Explanation of symbols]

[0081] 1A Information acquisition system, 2A Physical condition estimation system, 3A Physical condition estimation system, 100 Information acquisition device, 101 Control unit, 102 Imaging information acquisition unit, 103 Biological information acquisition unit, 104 Awakening information acquisition unit, 200 Physical condition estimation device, 201 Feature extraction unit, 202 Learning unit, 203 Model memory unit, 204 State estimation unit, 300 Physical condition estimation device, 302 Threshold adjustment unit, 303 Judgment unit, B1 Mounting platform, B1a Mounting surface, C1 Imaging unit, L0 Centerline, L1 Imaging range, M1 Drive unit, N1 Input unit, P1 Subject, S1 Sensor.

Claims

1. An imaging unit that captures images of a subject while being held in a constant position relative to the mounting surface, and an imaging information acquisition unit that acquires imaging information of the subject mounted on the mounting surface, The mounting surface includes an awakening information acquisition unit that acquires information regarding the level of arousal of a subject mounted on it, The imaging information acquisition unit acquires imaging information, including the face of a subject mounted on the mounting surface, which moves between a first position where the angle between the mounting surface and the horizontal plane is a first angle, and a second position where the angle between the mounting surface and the horizontal plane is a second angle. An information acquisition device characterized by the following features.

2. The mounting surface is equipped with a biometric information acquisition unit that acquires biometric information of a subject mounted on it. The information acquisition device according to claim 1, characterized by the feature.

3. The second position is a position where the angle with respect to the horizontal plane is greater than that of the first position. The imaging information acquisition unit acquires imaging information, including the face of the subject mounted on the mounting surface, at any time after the position of the mounting surface has changed from at least the first position to the second position. The information acquisition device according to claim 1, characterized by the feature.

4. The imaging information acquisition unit acquires imaging information of a subject mounted on the mounting surface when the mounting surface is located at the second position, when the mounting surface is moving between the first and second positions, when the mounting surface is located at the first position, or when the mounting surface is moving between the second and first positions, so that the imaging information acquired when the mounting surface is located at the first position, or when the mounting surface is moving between the second and first positions, covers the same range of imaging information for the mounting surface. The information acquisition device according to any one of claims 1 to 3.

5. The system includes a control unit that controls the angle between the mounting surface and the horizontal plane so that the mounting surface moves between the first position and the second position. The information acquisition device according to any one of claims 1 to 3.

6. An imaging unit that captures images of a subject while being held in a constant position relative to the mounting surface, and an imaging information acquisition unit that acquires imaging information of the subject mounted on the mounting surface, The aforementioned mounting surface includes an awakening information acquisition unit that acquires information regarding the level of arousal of the subject, The aforementioned imaging information acquisition unit extracts feature quantities from the imaging information acquired by the imaging information acquisition unit, The system includes a trained model generation unit that generates a trained model that learns using training data including information on the subject's level of alertness acquired by the alertness information acquisition unit and features extracted by the feature extraction unit, and outputs a determination result of the subject's level of alertness based on the input of imaging information. The imaging information acquisition unit acquires imaging information, including the face of a subject mounted on the mounting surface, which moves between a first position where the angle between the mounting surface and the horizontal plane is a first angle, and a second position where the angle between the mounting surface and the horizontal plane is a second angle. A learning device characterized by the following features.

7. An information acquisition method performed by an apparatus comprising an imaging information acquisition unit and an awakening information acquisition unit, The imaging information acquisition unit acquires imaging information of the subject mounted on the mounting surface from an imaging unit that is held in a constant position relative to the mounting surface and images the subject. The arousal information acquisition unit includes the step of acquiring information regarding the arousal level of a subject mounted on the mounting surface, The imaging information acquisition unit acquires imaging information, including the face of a subject mounted on the mounting surface, which moves between a first position where the angle between the mounting surface and the horizontal plane is a first angle, and a second position where the angle between the mounting surface and the horizontal plane is a second angle. A method for obtaining information characterized by the following features.

Images