Biometric data-related index measurement system

Abstract

To provide a biological data-related index measuring system that allows biological data to be used efficiently.SOLUTION: A biological data-related index measuring system 1 for measuring a predetermined biological data-related index D3 based on biological data of a measurement object includes: a biological feature amount calculation unit 121 for calculating a biological feature amount and a degree of reliability of the biological feature amount on the basis of the biological data D1; a motion feature amount calculation unit 122 for calculating a motion feature amount on the basis of motion data D2 of the measurement object; and a biological data-related index calculation unit 123 for calculating a predetermined biological data-related index D3 on the measurement object on the basis of the biological feature amount, the degree of reliability of the biological feature amount, and the motion feature amount.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a biological data-related index measurement system, an information processing system, and a biological data-related index measurement method. [Background technology]

[0002] Although some factories or maintenance sites (hereinafter referred to as factories, etc.) are highly mechanized, many factories, etc. rely on workers. Therefore, reducing human error by workers is the key to improving quality and productivity. There are known technologies that measure biomarkers without using data from periods when intense movement is detected in order to eliminate measurement errors associated with human movement (Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-283041 A [Patent Document 2] JP 2014-200389 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology does not use data measured when a person is moving vigorously, but in reality, even if the movement is vigorous, it is not always difficult to measure the biomarkers. There are cases where the movement is vigorous but the biomarkers can be calculated stably, and cases where the movement is not vigorous but the biomarkers are difficult to calculate.

[0005] If we ignore this reality and simply filter based on the level of activity of physical movement, we may end up excluding data from sections where movement is intense but the calculation of biomarkers is stable, or conversely, using data from sections where movement is not intense but the calculation of biomarkers is not stable.

[0006] An object of the present disclosure is to provide a biological-data-related index measurement system, an information processing system, and a biological-data-related index measurement method that enable efficient use of biological data. [Means for solving the problem]

[0007] In order to solve the above problems, the biometric data-related index measurement system according to the present disclosure is a biometric data-related index measurement system that measures predetermined biometric data-related indexes based on the biometric data of a measurement subject, and includes a biometric feature calculation unit that calculates a biometric feature and a reliability of the biometric feature based on the biometric data, a motion feature calculation unit that calculates a motion feature based on motion data of the measurement subject, and a biometric data-related index calculation unit that calculates the predetermined biometric data-related index for the measurement subject based on the biometric feature, the reliability of the biometric feature, and the motion feature. Effect of the Invention

[0008] According to the present invention, it is possible to calculate a predetermined biometric data-related index for a measurement target based on a biometric feature amount, the reliability of the biometric feature amount, and a motion feature amount. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing an overall outline of an embodiment of the present invention; [Diagram 2] FIG. 1 is a diagram showing the configuration of a biological indicator measuring device. [Diagram 3] FIG. 4 is an explanatory diagram showing feature amounts of electrocardiogram data as an example of biological data. [Figure 4] FIG. 2 is a diagram used to explain a method for calculating biometric features. [Diagram 5] 13 is a flowchart of a process for calculating a biometric feature with a reliability level. [Figure 6] 13 is an example of a table for managing the reliability that determines whether or not a biometric feature can be used. [Figure 7] 13 is an example of another table for managing the reliability that determines whether or not a biometric feature can be used. [Figure 8]FIG. 11 is a block diagram of a biological index measuring system according to a second embodiment. [Figure 9] FIG. 11 is a block diagram of a biological index measuring system according to a third embodiment. [Figure 10] FIG. 13 is a block diagram of a biological index measuring system according to a fourth embodiment. [Figure 11] FIG. 13 is a block diagram including an information processing system that utilizes a biological index measuring system according to a fifth embodiment. [Figure 12] FIG. 13 is a block diagram including an information processing system that utilizes a biological index measuring system according to a sixth embodiment. [Figure 13] FIG. 13 is a block diagram including an information processing system that utilizes a biological index measuring system according to a seventh embodiment. [Figure 14] FIG. 13 is a block diagram including an information processing system that utilizes a biological index measuring system according to an eighth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, the accuracy of the biometric indicator as a "biometric data related indicator" is improved by using the biometric data of a worker as an example of a measurement target as effectively as possible. In this embodiment, the biometric feature used to calculate the biometric indicator is calculated from the biometric feature based on the biometric data using the motion feature based on the motion data of the worker. In this embodiment, the occurrence of human error is suppressed by giving a warning or advice to the worker based on the calculation result of the worker's biomarker. Furthermore, this embodiment also contributes to improving the environment in which the worker is placed by calculating statistical information on the worker's biomarker.

[0011] In the following, a human being such as a worker will be described as an example of a "measurement target", but this embodiment is not limited to humans, and can also measure livestock, pets, wild animals, and animals kept in zoos. When a human being is the measurement target, this embodiment is not limited to a worker. The measurement target in this embodiment is not limited to a worker, and can include, for example, a driver, a pilot, a doctor, a nurse, a cook, a delivery person, a fitness club instructor, a hairdresser, a barber, and the like. As an example, the measurement target in this embodiment is preferably a human being engaged in an activity including vigorous physical exercise. In this embodiment, the biological data measured during vigorous physical exercise is adopted based on its reliability. However, in this specification, "preferable" is not intended to be limited to a configuration or method that is considered to be preferable. EXAMPLES

[0012] A first embodiment will be described with reference to Figures 1 to 7. Figure 1 is an explanatory diagram showing a block diagram of a biological index measuring system 1 and biological features and the like.

[0013] The lower part of Fig. 1 shows a worker 3 as a "measurement target" performing work on a work target 4. The worker 3 is associated with one or more sensors 2(1) to 2(3).

[0014] The sensor 2(1) is a type of sensor worn by a worker, that is, a so-called wearable sensor. Examples of wearable sensors include glasses-type sensors, wristwatch-type sensors, shirt-type sensors, and wearable cameras. Glasses-type sensors can measure and output, for example, electrooculography, gaze, or video. Shirt-type sensors can acquire cardiac potential, acceleration, temperature, and clothing stretch and distortion. Wristwatch-type sensors measure and output, for example, pulse, body temperature, sweat rate, electrical activity of the skin, acceleration, posture (gyro sensor), and geomagnetism. FIG. 1 shows glasses-type sensor 2(1) and wristwatch-type sensor 2(2) as examples. Sensors 2(1) and 2(2) are sensors worn by worker 3, while sensor 2(3) is a sensor installed outside the worker. The sensor 2(3) is a sensor that measures the worker 3 and the environment surrounding the worker 3, such as a camera, a thermometer, a hygrometer, a barometer, a sound level meter, a vibration meter, an illuminance meter, or a solar radiation meter.

[0015] Hereinafter, when there is no need to distinguish between the sensors 2(1) to 2(3), they will be referred to as the sensor 2. The sensor 2 is not limited to the configuration described above. The surrounding environment (temperature, humidity, noise, etc.) of the worker 3 may be measured by a wearable terminal such as glasses or a wristwatch. Alternatively, the heart rate, etc. may be measured from the displacement of blood vessels visible on the skin of the worker 3 photographed by a camera.

[0016] The work object 4 is an object that is the target of work by the worker 3. The work object 4 may be, for example, various products such as automobiles, machine tools, office equipment, medical equipment, and home appliances, as well as various objects such as food, take-out food, clothing, patients, and animals. The biometric measurement system 1 of this embodiment collects as much reliable biometric data as possible and calculates biometric indicators when a significant change occurs in the biometric data of the worker 3 due to the worker 3 performing work on the work object 4. For this reason, the work object 4 may be any work that involves work that may cause a change in the biometric data of the worker 3. Therefore, the worker 3 is not limited to a factory worker, and may be, for example, a doctor, nurse, hairdresser, cook, driver, pilot, flight attendant, conductor, etc.

[0017] The middle part of Fig. 1 shows the relationship between biometric data and physical movement data. An explanation will be given taking electrocardiogram data as an example of biometric data. Physical movement data is data that indicates the movement and posture of worker 3 (hereinafter referred to as movement, etc.). The larger the amplitude of the waveform and the shorter the wavelength of the waveform, the more quickly the movement, etc. of worker 3 changes. In the example of Fig. 1, in sections DS1 and DS3, the worker is relatively quiet, but in section DS2, the worker 3 is moving, etc. vigorously.

[0018] Focusing on the biometric data, in section DS1 where the worker 3 is resting, the biometric data is stable. In section DS2 where the worker 3 is moving vigorously above a predetermined value, the biometric data is usually unstable. However, in the illustrated example, the biometric data of the worker 3 is stable even when the worker 3 is moving vigorously. For example, the physical ability of the worker 3 is originally high, and the cardiac potential is not affected by moving the body a little. On the other hand, in section DS3, the biometric data is unstable even though the worker 3 is resting. Although the worker 3 himself is not moving much, if the surrounding environment (temperature, humidity, noise, illuminance) changes significantly, the biometric data may be disturbed.

[0019] In this embodiment, as described later, the reliability of the biometric feature obtained from the biometric data of the worker 3 is evaluated, and a utilization standard (e.g., a threshold value) for the reliability is determined based on the motion feature obtained from the body motion data, and the reliable biometric feature is adopted based on the reliability to calculate the biometric index. In the example in the middle part of FIG. 1, the reliability of the biometric feature based on the biometric data in the sections DS1 and DS2 is evaluated as high and adopted, but the reliability of the biometric feature based on the biometric data in the section DS3 is evaluated as low and not adopted. For example, if the biometric data in a certain section A is similar to that in another section B, the reliability will also be similar (e.g., 0.91 and 0.92). However, since the motion features in each section A and B are different, the threshold value of the reliability is different for each section A and B (e.g., 0.7 and 0.95). As a result, there are cases in which the biometric data in the section A is adopted because its reliability is above the threshold (0.91>0.7), but the biometric data in the section B is not adopted because its reliability is high but below the threshold (0.92<0.95).

[0020] 1 shows functional blocks of the biomarker measurement system 1. The biomarker measurement system 1 includes, for example, a reliability-added biomarker calculation unit 121, a motion feature calculation unit 122, and a biomarker calculation unit 123.

[0021] The biometric feature amount calculation unit 121 with reliability (hereinafter sometimes abbreviated as biometric feature amount calculation unit 121) calculates a biometric feature amount based on the biometric data D1 and also calculates the reliability of the calculated biometric feature amount. The biometric feature amount calculation unit 121 sends the calculated biometric feature amount and its reliability to the biometric index calculation unit 123.

[0022] The biometric data D1 includes, for example, electrocardiogram, electrooculogram, electromyogram, electroencephalogram, gaze, amount of sweat, body temperature, etc. The biometric feature amount calculation unit 121 may directly acquire the biometric data from the sensor 2, or may later read and process the biometric data stored in the storage device.

[0023] The biometric feature calculation unit 121 calculates the posterior probability that a certain biometric feature will be obtained from the biometric data observed up to a certain point in time and the features up to that point, and outputs the calculated posterior probability to the biometric index calculation unit 123 as the reliability of the biometric feature.

[0024] Biometric features are obtained by applying a specified process to raw biometric data in order to calculate biometric indicators, such as the peak position and interval of the waveform (RRI: RR Interval), the height of the peak, the length of time that the value exceeds a certain value, and the signal strength of a specific frequency component.

[0025] The motion feature amount calculation unit 122 calculates a motion feature amount based on the body motion data D2, and sends the calculated motion feature amount to the biometric index calculation unit 123.

[0026] Physical movement data D2 is data used to estimate the physical movements of worker 3, such as data obtained from an acceleration sensor, a gyro sensor, a geomagnetic sensor, a fixed camera, a camera built into a wearable terminal, a distance camera, a laser sensor, a distortion sensor, etc.

[0027] The biometric calculation unit 123 calculates a biometric index as a "biometric data related index" by considering both the biometric feature and its reliability and the motion feature. The biometric calculation unit 123 determines which of the biometric features received from the biometric feature calculation unit 121 to use in calculating the biometric index according to the motion feature, as will be described in detail later. For example, the biometric calculation unit 123 can use a biometric feature having a reliability equal to or higher than a threshold determined by the motion feature for calculating the biometric index. The method of determining whether or not to use a biometric feature according to the motion feature is not limited to selection based on a threshold. Any method that can select a biometric feature to be used in calculating a biometric index based on the motion feature can be applied to this embodiment.

[0028] The biomarker calculation unit 123 outputs the calculation result D3 of the biomarker. A system for providing information to the worker 3 and understanding the working environment of the worker 3 using the calculated biomarker D3 will be described later.

[0029] Examples of biomarkers include heart rate, RRI, RRV (RR interval variability), LF (low frequency component), HF (high frequency component), LF / HF, NN50 (total number of consecutive adjacent RRIs with a difference of more than 50 ms), pNN50 (ratio of adjacent RRIs with a difference of more than 50 msec), blink frequency, blink length, signal strength and ratio (e.g., β / α, etc.) of various frequency bands of electroencephalograms, gaze movement, various statistics such as time average, variance, and median, time derivative (difference) and integral value, and one or more of these with a predetermined transformation. It may also be a value obtained by applying some function f to the value x. The function f(x) may be a simple structure or a complex function such as that expressed by a neural network.

[0030] 2 is a configuration diagram of a biological index measuring device 10 in a biological index measuring system 1. The biological index measuring device 10 is connected to a plurality of sensors 21 and 22. The sensor 21 is a biological data sensor that detects biological data. The sensor 22 is a physical motion data sensor that detects physical motion data. The sensors 21 and 22 are examples of the sensor 2 described in FIG. 1. The biological data sensor 21 and the physical motion data sensor 22 may be provided in the same housing.

[0031] The biometric measuring device 10 is configured as a computer including, for example, a signal processing unit 11, a control unit 12, and a communication unit 13. The control unit 12 realizes each function such as a reliability-added biometric feature calculation unit 121, a motion feature calculation unit 122, and a biometric feature calculation unit 123, using hardware resources including a processor 101 and a communication memory 102, and a predetermined computer program stored in the memory 102.

[0032] The biomarker measuring device 10 is connected to an external device 5 from the communication unit 13 via a communication network CN. Here, the external device 5 is a collective term for, for example, a computer 5(1), a web server 5(2), and a storage system 5(3). These external devices 5 can receive and use a biomarker D3 from the biomarker measuring device 10. As described later, the biomarker measuring system 1 and an information processing system 140 that uses the biomarker may be directly connected without going through the communication network CN, or may be provided in the same computer.

[0033] A storage medium MM, such as a flash memory or a hard disk drive, can be removably connected to the biological index measuring device 10. The storage medium MM can transmit and receive part or all of a computer program or data to and from the memory 102.

[0034] <Explanation of the reliability-based biometric feature calculation unit> A specific example of the reliability-added biometric feature calculation unit 121 will be described with reference to Fig. 3. Here, an example will be described in which a cardiac potential is calculated as biometric data and an average RRI is calculated as a biometric index. The vertical axis of Fig. 3 indicates the cardiac potential, and the horizontal axis indicates time. In the figure, x indicates the entire biometric data (raw data of the cardiac potential) before processing. In this embodiment, estimation of biometric features is expressed by a probabilistic model.

[0035] When observed data x (and the value of feature y up to that point) is obtained, the posterior probability that a certain biometric feature y is obtained is calculated, and this is regarded as the reliability of the biometric feature. When electrocardiogram data is used as biometric data and RRI is used as biometric feature, for example, the posterior probability p(y t+1 │x,y t ,…,y1) can be calculated. Qualitatively, when the observed data x and the features (RRI) up to the tth feature are obtained, the t+1th feature is y t+1 The posterior probability p can be expressed by the following equation (1).

[0036]

number

[0037]

number

[0038] Moreover, Y can also be expressed by the following formula 3.

[0039]

number

[0040] Here, p(x│y t+1 ,y t , …, y1) and p(y t+1 |y t ,…,y1) can be modeled arbitrarily to match the characteristics of the features.

[0041] For example, an example of RRI calculation can be expressed as Equation 4.

[0042]

number

[0043] In addition, if it is difficult to model such data properties, p(x│y t+1 ,y t ,…,y1) is assumed to be uniformly distributed, so that p(y t+1 |y t ,…,y1) from p(y t+1 │x,y t , ..., y1), or conversely, if we assume that there is no dependency on past feature calculation results, we can calculate p(y t+1 |y t ,…,y1) may be assumed to be uniformly distributed. t+1 |y t , ..., y1) can be expressed, for example, as the following numbers 5, 6, and 7.

[0044]

number

number

number

[0045]

number

[0046] In equation 7, x_t is expressed by equation 9 below.

[0047]

number

[0048] 1 / y of the number 7 t+1 corresponds to the period in which the peak of the electrocardiogram data is observed. Since it is expected that the signal strength of the frequency band corresponding to this period will be strong in the electrocardiogram data, Equation 7 is qualitatively modeled so that the RRI value becomes higher as the period is frequently observed in the data observed so far. p(y t+1 │y t ,…,y1), these may be used in combination.

[0049] In this way, p(x│y t+1 ,y t , …, y1) and p(y t+1 |y t ,…,y1) and use them to calculate the posterior probability p(y t+1 │x,y t ,…,y1) can be calculated.

[0050] The t+1th RRI can be calculated, for example, from equation (10).

[0051]

number

[0052] As described above, in this embodiment, the reliability is expressed by a probability model, and as a result, the reliability is a value of [0,1], but it is not necessary to design it in this way. For example, in the previous example, the peak height of the cardiac potential can be used as the reliability as it is. However, in this case, it becomes difficult to intuitively grasp the meaning of the threshold value XX, which means that "if the cardiac potential is XX or more, the biometric feature is adopted for biometric index calculation," and when a different biometric feature is used, the threshold value must be reset to match the biometric feature.

[0053] In contrast, if the posterior probability of obtaining a biometric feature is calculated according to a probability model and the calculated posterior probability is used as the reliability of the biometric feature, as in this embodiment, not only can the meaning of the threshold be intuitively understood, but it can also be applied to other biometric features, and it is easy to use for designers and users of the biometric measurement system. However, if ease of use is not a major concern, the other methods described above may be adopted.

[0054] FIG. 5 is a flowchart showing a method for calculating a biometric feature with a reliability level.

[0055] The reliability-added biometric feature calculation unit 121 executes the processes of steps S11 to S19 shown in FIG. 5 to calculate pairs of biometric features and their reliability, as described above.

[0056] In step S11, the first feature amount y1 and its reliability p(y1|x) are calculated and recorded. In step S12, t is set to 1, and in step S13, the following formula 11 is calculated.

[0057]

number

[0058]

number

number

[0059]

number

[0060] <Explanation of motion feature calculation unit> A method for calculating the motion feature amount will be described. As described above, the motion feature amount can be, for example, the motion intensity, the motion type, the posture, etc. The following is an example of the calculation of the motion intensity.

[0061] For example, as the body motion data, the three-axis acceleration data (a x ,t,ay ,t,a z When the operating intensity It at time t is obtained, t =√(a x 2 +a y 2 +a z 2 ) or by using only a specific axis, x,t Alternatively, acceleration data may be acquired for a certain period of time, and a specific direction may be determined by applying, for example, principal component analysis to the data, and calculations may be performed using the components in that direction. Calculations may also be performed based on the time differential or integral values ​​of the acceleration data. Similar processing can be performed for gyro sensors, geomagnetic sensors, and the like.

[0062] When images can be acquired from a camera built into the wearable device as physical movement data, for example, the optical flow can be calculated using any existing method, and the magnitude of the optical flow at each point in the image can be added up to determine the movement intensity.

[0063] That is, let h be the horizontal and vertical optical flow at the point (x, y) in the nth frame image. x,y ,v x,y Then, the motion strength at the nth frame I n can be calculated as shown in the following equation (15).

[0064]

number

[0065] Furthermore, when images can be obtained from a fixed camera as body motion data, the motion strength can be calculated, for example, as follows: First, a skeleton extraction process is performed using any existing method on the person (worker 3) whose biometrics are to be calculated, and a set of three-dimensional coordinates of the joint positions of the person in the n-th frame of the image (Equation 16) is obtained.

[0066]

number

[0067]

number

[0068]

number

[0069] When using a motion type or posture as a motion feature, any existing behavior identification method can be used to identify the motion type from acceleration data, image data, etc., and output the ID of the motion type.

[0070] The motion types include, for example, standing still, standing, sitting, sleeping, walking, running, climbing stairs, traveling by vehicle, sports (which may be further classified into soccer, baseball, basketball, etc.), work (which may be further classified into screwdriving, wiring, welding, hammering nails, visual inspection, etc.), etc. In addition to such classifications, the motion type may also be determined by whether the motion is in a normal state or an abnormal state.

[0071] As the motion feature amount, a combination of the motion type and the motion intensity may be output.

[0072] The means for acquiring body movement data does not need to be limited to a single sensor, but may use multiple fixed cameras, or a combination of a fixed camera and a wearable camera, a camera and an acceleration sensor, or an acceleration sensor and a gyro sensor.

[0073] <Explanation of biomarker calculation section> The biometric calculation unit calculates the biometric index by considering both the biometric feature and its reliability and the motion feature. The biometric index includes, for example, heart rate, RRI, RRV, LF, HF, LF / HF, NN50, pNN50, blink frequency, blink length, signal strength and ratio (e.g., β / α) of various frequency bands of electroencephalogram, gaze movement, various statistics such as time average, variance, and median of these, time derivative (difference) and integral value, and one or more of these subjected to a predetermined conversion.

[0074] Below, we will explain an example in which RRI (unit: msec), which is the interval between peaks (R waves) of the cardiac potential, is used as a biofeature, and the average heart rate (unit: beats / minute) over a certain period (e.g., 5 minutes) is used as a bioindicator.

[0075] Here, when N RRI measurement points (Equation 19) are obtained during the period, the average heart rate HR bar can be calculated, for example, by the following Equation 20.

[0076]

number

[0077]

number

[0078]

number

number

[0079]

number

[0080] The reliability thresholds corresponding to the motion features do not necessarily need to be stored in a format such as Table T1. For example, they may be expressed as an equation such as Equation 24, and their parameters (coefficients a and b in the example of Equation 24) may be stored.

[0081]

number

[0082]

number

[0083] When the type of motion or posture is output as the motion feature, a reliability threshold corresponding to each motion type (=motion feature) may be determined separately as shown in Table T1a in Fig. 7. After that, the biomarker can be calculated using, for example, Equation 23 in the same manner as described above.

[0084] Since the motion strength may vary even within a single motion type, the motion type and strength may be output as motion feature quantities, and the methods described in Equation 23 and Equation 25 may be combined to calculate the biometric index.

[0085] According to this embodiment configured as described above, while excluding data (biometric features) from sections where the calculation reliability is low, the data can be used as efficiently as possible, thereby making it possible to measure biomarkers efficiently and with high accuracy.

[0086] According to the biological index measuring system of this embodiment, biological indexes can be stably measured even in a situation where the body of a worker is moving, such as when the worker is working. EXAMPLES

[0087] The second embodiment will be described with reference to Fig. 8. In the following embodiments including this embodiment, the differences from the first embodiment will be mainly described. In this embodiment, the reference value of the biometric feature is selected according to the type of movement of the worker 3 to be measured.

[0088] 8 is a block diagram of a biomarker measurement system 1A of this embodiment. The biomarker measurement system 1A is obtained by adding a motion type discrimination unit 125 for discriminating motion types using body motion data and a motion-specific biomarker reference value database 124 for managing reference values ​​of biomarkers corresponding to motions to the biomarker measurement system 1 described in the first embodiment. Hereinafter, the motion-specific biomarker reference value database 124 may be abbreviated to reference value database 124.

[0089] When calculating a biometric feature and its reliability, the reliability-added biometric feature calculation unit 121 reads out the reference value of the biometric feature for the relevant action type from the reference value database 124 according to the discrimination result of the action type discrimination unit 125, and calculates the biometric feature and its reliability by referring to the reference value. The approximate range of values ​​that the biometric indicators and features can take may be determined depending on the action type. For example, the heart rate is low when at rest, but tends to be high when exercising, such as jogging. By modeling such prior knowledge, it is expected that more accurate calculation of biometric features and biometric indicators will be possible. The motion state is discriminated using body motion data such as acceleration data and video data, and the prediction model for the biometric feature (the parameters in the model) is changed according to the discrimination result. Mathematically, p(y t+1 |y t ,…,y1) will change depending on the operation type.

[0090] In equations 5 to 7, we modeled that the RRI observed up to that point will not suddenly differ significantly from the next RRI, but instead of or in addition to these, a model such as equation 26 can be used.

[0091]

number

[0092] The reference values ​​of biofeature quantities may be set manually based on knowledge of exercise science, etc., or may be determined using machine learning from past performance data. Here, an example of determining the type of movement from body movement data has been explained, but if external information that specifies the movement state is available, it may be used to determine the type of movement. For example, if information from a Manufacturing Execution System (MES) is available, the work content at each time, i.e., the type of movement, can be known.

[0093] The present embodiment thus configured also has the same operational effects as those of the first embodiment. Furthermore, in this embodiment, the reference value of the biometric feature is selected according to the type of movement, so that the biometric index can be measured with even higher accuracy. EXAMPLES

[0094] A third embodiment will be described with reference to Fig. 9. In this embodiment, a reference value of a biometric feature is selected for each worker (individual). Fig. 9 is a block diagram of a biometric indicator measuring system 1B of this embodiment.

[0095] The biometric measurement system 1B of this embodiment has a biometric feature reference value database 124B that manages the reference values ​​of the biometric feature for each individual, in comparison with the biometric measurement system 1 described in the first embodiment. When calculating the biometric feature and its reliability, the reliability-added biometric feature calculation unit 121 reads out the reference value of the biometric feature for the individual (worker 3) from the reference value database 124B and calculates the biometric feature and its reliability by referring to it.

[0096] The approximate range of values ​​that biometric indicators and biometric features can take may be determined for each individual. For example, it is known that athletes have low average heart rates. By modeling such knowledge, it is possible to calculate biometric features and biometric indicators with higher accuracy. In the reliability-based biometric feature calculation unit 121, the prediction model (parameters in the model) of the biometric feature is changed for each individual.

[0097] Mathematically, p(y t+1 |y t , ..., y1) will vary depending on the individual. In equations 5 to 7, we modeled that the RRI observed up to that point will not suddenly differ significantly from the next RRI, but instead of or in addition to these, we can also use the model shown in equation 27.

[0098]

number

[0099] The reference value of the biometric feature may be set based on a certain conversion formula from personal profile data such as height, weight, age, sex, exercise experience, and habits, or may be automatically estimated from the subject's past log data.

[0100] The present embodiment thus configured also has the same operational effects as those of the first embodiment. Furthermore, in this embodiment, the history of biometric features for each worker to be measured is managed, and individual reference values ​​for biometric features are used, so that the biometric indicators can be measured with higher accuracy. EXAMPLES

[0101] A fourth embodiment will be described with reference to Fig. 10. In this embodiment, biometric features are calculated with reference to information about the surrounding environment in which the worker 3 works.

[0102] 10 is a block diagram of a biomarker measurement system 1C of this embodiment. The biomarker measurement system 1C of this embodiment adds an environmental condition determination unit 126 that determines an environmental condition using environmental information data D4, and a biomarker reference database 124C that manages reference values ​​of biomarkers for each environmental condition, to the biomarker measurement system 1 described in the first embodiment. When calculating a biomarker and its reliability, a reliability-added biomarker calculation unit 121 reads out the reference value of the biomarker under the environmental condition from the database 124C, and calculates the biomarker and its reliability by referring to the reference value.

[0103] The environmental information data D4 includes, for example, temperature, humidity, illuminance, sound (noise), carbon dioxide concentration, etc. Sensors for detecting the environmental information data D4 include, for example, a thermometer, a hygrometer, a light meter, a microphone, a sound level meter, a carbon dioxide concentration meter, etc.

[0104] The approximate range of values ​​that biometric indicators and biometric features can take may be determined for each environmental condition. For example, high room temperature and humidity may increase sweating, and loud noise may increase heart rate. By modeling such prior knowledge, biometric features and biometric indicators can be calculated with higher accuracy. The reliability-based biometric feature calculation unit 121 changes the parameters in the prediction model of biometric features for each environmental condition. Mathematically, p(y t+1 |y t , ..., y1) will change depending on the environmental conditions. In equations 5 to 7, we modeled that the RRI observed up to that point will not suddenly differ significantly from the next RRI, but instead of or in addition to these, we can also use the model shown in equation 28.

[0105]

number

[0106] The environmental conditions may be continuous, e.g., temperature, or may be classified into several categories using one or more environmental sensors. When the environmental conditions are continuous, ν(e t+1 ) is, for example, an arbitrary function for converting the continuous value into another continuous value. The parameters of the function may be determined manually in advance, or may be determined by machine learning from data. These parameters are stored in the environmental condition-specific biometric feature reference value database 124C. In addition, when the environmental condition is a categorical value, the reference value for each category is stored in the database 124C.

[0107] The present embodiment thus configured also has the same operational effects as those of Example 1. Furthermore, in this embodiment, the reference values ​​of the biometric features are selected according to the working environment of the worker, so that the biometric indicators can be measured with higher accuracy. EXAMPLES

[0108] A fifth embodiment will be described with reference to Fig. 11. In each of the following embodiments including this embodiment, an information processing system including a biomarker utilization system 140 that utilizes biomarkers output from a biomarker measurement system 1 will be described. In this embodiment, the internal state of a worker is estimated from the biomarkers of the worker.

[0109] The information processing system 6 of this embodiment includes a biological indicator measurement system 1 and a biological indicator utilization system 140. The biological indicator utilization system 140 includes an internal state estimation database 142 that manages calculation parameters for estimating an internal state according to one or more biological indicators, and an internal state estimation unit 141 that estimates the internal state of a worker by using the calculated biological indicator D3 and the calculation parameters for internal state estimation.

[0110] The internal state is a state that represents the inside of a person, such as concentration level, fatigue level, stress level, rest level, excitement level, agitation level, confidence level, anger level, manic depression level, impatience level, vitality, motivation, error potential, etc. The internal state may be expressed as a continuous value, or may be expressed by quantizing or categorizing it as low, medium, high, rest, or the like.

[0111] In this embodiment, the concentration level of a worker is estimated using biomarkers such as the average heart rate and blinking frequency. Environmental information may be additionally used as necessary. For example, when a microphone is used as an environmental information sensor, if there is a lot of noise in the surroundings, or if it is recognized by voice recognition that someone is speaking to the worker, it is considered that there is a high possibility that the concentration level is decreasing.

[0112] An estimation formula may be determined manually in advance based on knowledge of bioscience or psychology, its parameters may be stored in the internal state estimation database 142, and the internal state of a person may be estimated using the parameters. Alternatively, data representing biomarkers and the internal state at that time may be preliminarily collected, parameters for estimation may be learned by a machine learning technique, the parameters may be stored in the internal state estimation database 142, and the internal state of a person may be estimated using the parameters. In this case, the data representing the internal state (ground truth data) preliminarily collected for learning the machine learning model may be collected using any other means, such as a subject questionnaire.

[0113] When estimating an internal state, a numerical value indicating its likelihood (confidence, certainty, likelihood, etc.) may be output together. For example, when using a neural network as a machine learning model for classification, the numerical value indicating the likelihood may be the output value of the final layer or a value obtained by applying a softmax function to the output value. In other cases, the numerical value may be determined by referring to the output result of the machine learning model.

[0114] The reliability of the biometric feature on which the internal state is estimated may also be output together, for example, as the average reliability of the biometric feature used in the calculation. In this way, by outputting a numerical value indicating the likelihood of the internal state estimation result together with the estimation result, more flexible analysis and judgment can be performed using the numerical value indicating the likelihood, as described later.

[0115] The present embodiment thus configured also has the same operational effects as those of the first embodiment. Furthermore, since the information processing system 6 of this embodiment includes the biomarker utilization system 140, it is possible to estimate the internal state of the worker 3 from the biomarkers of the worker, and the usability of the information processing system 6 for the user is improved. EXAMPLES

[0116] A sixth embodiment will be described with reference to Fig. 12. An information processing system 6A of this embodiment outputs an alarm D6 as "predetermined information" according to the estimation result of the internal state of the worker.

[0117] In the biomarker utilization system 140A of the information processing system 6A of this embodiment, an alarm output unit 143 that outputs an alarm D6 based on the internal state estimation result D5 is added to the biomarker utilization system 140 of the information processing system 6 described in the fifth embodiment.

[0118] The biomarker utilization system 140A outputs an alarm D6, for example, when the concentration level of the worker falls below a predetermined threshold or when the fatigue level of the worker exceeds a predetermined threshold. The alarm output destination may be the worker himself who is the measurement target, or may be another person such as a manager who supervises the worker.

[0119] The alarm D6 can be output using, for example, sound, vibration, color change, lighting / flashing, screen display, notification icon, email, and the like.

[0120] As a method for outputting the alarm, if the internal state estimation result is a continuous value, the above-mentioned threshold processing may be used, or if it is a category, the alarm may be output when it falls into a certain category.

[0121] In addition to a simple binary decision such as outputting an alarm or not outputting an alarm, the content of the alarm D6 may be changed according to a value or category.

[0122] In order to prevent false alarms and excessive warnings, a warning may be output when the internal state estimation result D5 exceeds a threshold a certain number of times within a unit time. A warning may be output when the internal state estimation result D5 exceeds the threshold continuously for a certain period of time. Whether or not to output a warning may be determined based on a moving average of the internal state estimation result D5.

[0123] As shown in the upper part of Fig. 12, the alarm D6 may include a value that indicates the likelihood of the internal state estimation, such as "concentration level: 38, reliability level: 0.94". This allows a worker or manager to decide whether to follow the alarm D6 when it is output, for example, to inform the worker of a decrease in concentration. For example, if there is sufficient work time, the worker can take a break, but if there is not enough work time and the reliability of the alarm is low, the worker can ignore the alarm.

[0124] Furthermore, if personal information of the data collection target is available, the decision on whether to issue an alert can be changed depending on that personal information. For example, if a worker is a novice, he or she is unfamiliar with the work and has a lower level of concentration and a higher level of fatigue than an experienced worker, so the conditions for issuing an alert can be relaxed.

[0125] Data indicating the worker's physical condition, data indicating the amount of sleep, and the like may be input to the biomarker utilization system 140A, and the conditions for issuing the alarm D6 or the contents of the alarm D6 may be adjusted using this personal data.

[0126] It is also possible to use the reliability of the estimation result of the internal state to determine whether or not to output an alarm D6. For example, for an expert, an alarm is output only if the reliability is 0.90 or more, but for a beginner, an alarm is output if it is 0.60 or more, and for a person in poor health, an alarm is output just in case even if the reliability is low. This makes it possible to support flexible decisions regarding the output of an alarm.

[0127] The present embodiment thus configured also has the same effects as those of the fifth embodiment. Furthermore, in this embodiment, an alarm D6 is output based on the internal state estimation result D5, which further improves usability for the worker who is the measurement target and the user of the system 6. EXAMPLES

[0128] The seventh embodiment will be described with reference to Fig. 13. An information processing system 6B of this embodiment accumulates estimation results D5 of the internal state of a worker, and statistically processes the accumulated estimation results D5 for use.

[0129] In the biomarker utilization system 140A of the information processing system 6B of this embodiment, an internal state record database 144 that manages records of a plurality of internal state estimation results D5 estimated in a certain activity environment, and a statistical information calculation unit 145 that calculates statistical information of the internal state in the activity environment based on the data stored in the database 144 are added to the biomarker utilization system 140 of the information processing system 6 described in the fifth embodiment.

[0130] For example, when estimating concentration as an internal state, by saving the concentration history, it is possible to compare the concentration of workers at one work site with that of workers at other work sites. If this reveals that workers at a certain work site have a tendency to have low concentration, it is possible to create improvement measures for that site, such as changing the layout of the work environment, reviewing the work order, controlling the temperature, lighting, and sound environment, and replacing personnel.

[0131] As shown in the upper part of FIG. 13, the internal state record database 144 may include, for example, a time stamp (TS in the figure), an individual ID, an environment ID, an activity ID, and a set of one or more internal state estimation results and their reliability.

[0132] The timestamp is the date and time when the internal state estimation result D5 was registered in the database 144. The individual ID is information that identifies the worker who is the subject of measurement. The environment ID is an ID that identifies, for example, a factory or a production line. The activity ID is an ID that identifies the type of work. Other information such as the tools used, the IDs of workers who worked together, and event information may be added to the table 144 as necessary.

[0133] The internal state record may be for one person or for multiple people. The internal state does not have to be of one type, but may be of multiple types.

[0134] The statistical information calculated from the accumulated internal state estimation results D5 is, for example, an average value, a variance, a time transition, etc. The statistical information may be displayed as a graph. The biomarker utilization system 140B may have a function of proposing improvement measures based on the statistical information. The improvement measures include, for example, improvement measures for the work site, improvement measures for the work process, improvement measures for the design drawings, improvement measures for the work procedures, etc.

[0135] As described above, a value indicating the likelihood of the internal state estimation D5 may be recorded as a "reliability" together with the internal state estimation result D5 in database 144. In this way, for example, when a sufficient amount of data has been accumulated, it is possible to perform processing such as tallying up only the estimation results D5 with a reliability above a certain value and calculating statistical information, which is expected to result in more accurate statistical information. EXAMPLES

[0136] An eighth embodiment will be described with reference to Fig. 14. An information processing system 6C of this embodiment stores the estimation results D5 of the internal state of the worker in association with the environmental information data D4, and statistically processes the stored data D5 and D4 for use.

[0137] In the biomarker utilization system 140C of the information processing system 6B of this embodiment, an internal state recording database 144C that records a plurality of internal state estimation results D5 estimated in a certain activity environment D4, and a statistical information calculation unit 145C that calculates statistical information of the internal state in the activity environment based on the data D5 stored in the database 144 are added to the biomarker utilization system 140 of the information processing system 6 described in the fifth embodiment.

[0138] If the estimation result D5 of the worker's internal state is associated with the environmental information data D4 such as temperature and illuminance and recorded in the database 144C, the relationship between the work environment and the internal state can be inferred, improving the usability for the user of the information processing system 6C. Examples of the relationship between the work environment and the internal state include that when the temperature is high, concentration tends to decrease, when the illuminance is increased, the concentration level tends to increase, when the noise level is reduced, the fatigue level tends to decrease, etc.

[0139] The internal state record database 144C may include, for example, a time stamp, a personal ID, an environment ID, an activity ID, environment information, and a set of one or more internal state estimation results and their reliability. A column for environment information is provided for each item of environment information such as temperature, humidity, illuminance, noise, etc.

[0140] The above-mentioned embodiments are described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. In addition, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0141] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be partly or entirely realized in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

[0142] Furthermore, the program code for realizing the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), and the like.

[0143] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read out and execute the program code stored in the storage means or storage medium.

[0144] In the above-mentioned embodiment, the control lines and information lines are shown as those considered necessary for the explanation, and not all the control lines and information lines are shown in the product. All the components may be connected to each other. [Explanation of symbols]

[0145] 1, 1A, 1B, 1C: biometric indicator measurement system, 2: sensor, 3: worker, 4: work object, 5: external device, 6, 6A, 6B, 6C: information processing system, 121: biometric feature calculation unit with reliability, 122: motion feature calculation unit, 123: biometric indicator calculation unit, 124: biometric feature reference database by motion, 124B: individual biometric feature reference database, 124C: biometric feature reference database by environmental condition, 125: motion type discrimination unit, 126: environmental condition calculation unit, 141: internal state estimation unit, 142: internal state estimation database, 143: alarm output unit, 144, 144C: ​​internal state record database, 145, 145C: statistical information calculation unit

Claims

1. A biometric data-related index measurement system for measuring a predetermined biometric data-related index based on biometric data of a measurement subject, comprising: a biometric feature calculation unit that calculates a biometric feature and a reliability of the biometric feature based on the biometric data; a motion feature amount calculation unit that calculates a motion feature amount based on motion data of a measurement target; a biometric data-related index calculation unit that calculates a predetermined biometric data-related index for the measurement target based on the biometric feature amount, the reliability of the biometric feature amount, and the motion feature amount; Equipped with The biometric feature calculation unit calculates a posterior probability of obtaining a biometric feature based on a predetermined probability model, and outputs the calculated posterior probability value as a reliability of the biometric feature. Biometric data-related indicator measurement system.

2. The predetermined probability model can be set according to the characteristics of the biometric feature. The biological data related index measuring system according to claim 1 .

3. The present invention further includes a motion type discrimination unit that discriminates a motion type based on the motion data of the measurement target, and a motion-specific reference value management unit that manages a reference value of a biometric feature for each motion type or posture (hereinafter, motion type, etc.) of the measurement target, The biometric feature calculation unit acquires a reference value of a biometric feature corresponding to the action type determined by the action type determination unit from the action-specific reference value management unit, and calculates a biometric feature and a reliability of the biometric feature from biometric data using the acquired reference value of the biometric feature. The biological data related index measuring system according to claim 2 .

4. A measurement object-specific reference value management unit that manages a reference value of the biometric feature amount for each measurement object, The biometric feature calculation unit acquires a reference value of the biometric feature corresponding to the measurement object from the measurement object reference value management unit, and calculates the biometric feature and a reliability of the biometric feature from biometric data using the acquired reference value of the biometric feature. The biological data related index measuring system according to claim 1 .

5. The apparatus further includes an environmental condition determination unit that determines an environmental condition based on environmental information about the surroundings of the measurement target, and an environmental condition-specific reference value management unit that manages a reference value of the biometric feature for each environmental condition, The biometric feature calculation unit acquires a reference value of a biometric feature corresponding to the environmental condition determined by the environmental condition determination unit from the environmental condition-specific reference value management unit, and calculates a biometric feature and a reliability of the biometric feature from biometric data using the acquired reference value of the biometric feature. The biological data related index measuring system according to claim 1 .

Images