Model generation device, model generation method, biological data estimation device, and biological data estimation method

Abstract

This model generation device comprises: an imaging data acquisition unit (11) that acquires imaging data including a skin area of a subject; a luminance data acquisition unit (12) that acquires, from the acquired imaging data, luminance data representing a luminance change in the skin area of the subject; a biological reference data acquisition unit (13) that acquires biological reference data of the subject, wherein the biological reference data is acquired using a contact sensor from at least one skin area that is identical, adjacent, or similar to the skin area in which the luminance data is acquired; a data processing unit (14) that generates training waveform data from the acquired luminance data and the acquired biological reference data; and a model generation unit (15) that generates a machine learning model for estimating biological data, using the generated training waveform data and by performing machine learning in which the acquired biological reference data is used as teacher data.

Description

Model generation device, model generation method, biological data estimation device, and biological data estimation method

[0001] The present disclosure relates to a model generation device, a model generation method, a biological data estimation device, and a biological data estimation method.

[0002] Patent Document 1 discloses a technique (claim 1 of Patent Document 1) for receiving a plurality of image frames, generating a plurality of estimated depth maps for the plurality of image frames using a convolutional neural network (CNN) implemented by a computer processor, generating an estimated remote photoplethysmography (rPPG) signal for the plurality of image frames using a recurrent neural network (RNN) implemented by a computer processor, determining a classification score for the plurality of image frames based on the estimated rPPG signal and a selected estimated depth map from the plurality of estimated depth maps, and classifying the plurality of image frames as corresponding to a presentation attack based on the classification score.

[0003] International Publication No. 2019 / 152983

[0004] When a learning model is constructed according to the teachings of conventional technologies such as Patent Document 1, in which brightness data measured on the face and true value data measured at the fingertip using a pulse wave sensor are used as input to estimate a pulse wave, there is a problem in that the accuracy of the learning model is reduced because there is a phase difference between the data used during learning.

[0005] The present disclosure has been made to solve such problems, and aims to provide a model generation technology that can generate a learning model in which phase difference is suppressed, or to provide a biometric data estimation technology that estimates biometric data using a learning model thus generated.

[0006] One aspect of a model generation device according to an embodiment of the present disclosure includes an imaging data acquisition unit that acquires imaging data including a skin region of a subject; a brightness data acquisition unit that acquires brightness data indicating brightness changes in the skin region of the subject from the acquired imaging data; a biometric reference data acquisition unit that acquires biometric reference data of the subject, wherein the biometric reference data is acquired by a contact sensor from at least one skin region that is the same as, an adjacent skin region, or a similar skin region as the skin region from which the brightness data is acquired; a data processing unit that generates training waveform data from the acquired brightness data and the acquired biometric reference data; and a model generation unit that uses the generated training waveform data to perform machine learning using the acquired biometric reference data as training data to generate a machine learning model for estimating biometric data.

[0007] According to the model generation device of the embodiment of the present disclosure, it is possible to acquire biological data with reduced phase difference compared to conventional techniques and generate a machine learning model.

[0008] FIG. 1 is a block diagram of a model generating device according to a first embodiment; FIG. 2 is a block diagram of a biometric data estimation device according to a first embodiment; FIG. 3 is a diagram illustrating problems with the prior art; FIG. 4 is an explanatory diagram illustrating a method for acquiring biometric reference data; FIG. 5 is a diagram illustrating the bilateral symmetry of blood vessel arrangement (Online Dermal Filler Complications Course|reed.co.uk); FIG. 6 is an explanatory diagram illustrating a method for acquiring luminance data and biometric reference data from bilaterally symmetrical regions of a subject's body; FIG. 7 is an explanatory diagram illustrating a method for acquiring biometric reference data from multiple skin regions; FIG. 8 is a diagram illustrating an example of data processing or data complementation; FIG. 9 is a diagram illustrating an example of data processing or data complementation; FIG. 10 is a diagram illustrating an overview of Turnip; FIG. 11 is an enlarged view of a recursive network structure; FIG. 12 is an explanatory diagram for explaining processing by an estimation unit; FIG. 13 is a flow chart illustrating the operation of a model generating device; FIG. 14 is a block diagram illustrating an example of a biometric data estimation system according to a second embodiment; FIG. 15 is a diagram illustrating the presence of a phase difference in pulse wave data acquired from multiple regions of the face; FIG. 16 is a table summarizing the relationships between the states or trends of blood pressure values, blood vessels, and phase differences; FIG. 17 is a diagram illustrating an example of a case where luminance data cannot be acquired; FIG. 18 is a diagram illustrating an outline of a learning method for learning a model without using luminance data of some skin regions. A model generating device and a biological data estimating device are configured to estimate biological data without using brightness data of a part of a skin region.

[0009] Various embodiments of the present disclosure will be described in detail below with reference to the drawings. In the drawings, identical or similar parts are designated by identical or similar reference numerals, and redundant explanations of such parts will be omitted. In addition, in this disclosure, the term "or" is used to mean an inclusive logical OR unless otherwise specified.

[0010] First Embodiment <Configuration; Learning Phase> A model generation device 10 according to a first embodiment of the present disclosure will be described with reference to Fig. 1. The model generation device 10 is a device that acquires data in which the phase difference is suppressed and generates a machine learning model.

[0011] To better explain the purpose of the model generation device 10 according to the present disclosure, the problems of the prior art will be described with reference to FIG. 3 . As shown in FIG. 3 , a phase difference occurs between the luminance change acquired from the subject's facial region and the continuous blood pressure waveform or pulse waveform acquired from the subject's fingertip. This is because the distance from the subject's heart to the fingertip (the length of the blood vessel) is longer than the distance from the subject's heart to the face (the length of the blood vessel). The model generation device 10 according to the present disclosure aims to suppress this phase difference and generate a machine learning model.

[0012] 1, the model generating device 10 includes an imaging data acquisition unit 11, a brightness data acquisition unit 12, a biometric reference data acquisition unit 13, a data processing unit 14, and a model generating unit 15. These functional units will be described in more detail below.

[0013] (Imaging Data Acquisition Unit) The imaging data acquisition unit 11 acquires imaging data including the subject's skin, for example, the face, captured by a camera (not shown). The imaging data acquisition unit 11 may acquire the imaging data directly from the camera or from a memory (not shown) storing the imaging data. The frame rate of the imaging data is, for example, 30 fps (frames per second). The imaging data may be RGB image (color image) data or IR image (monochrome image) data. In the case of an IR image, an IR light source (not shown) may be provided to irradiate IR light in order to acquire an IR image from the subject. The imaging data acquisition unit 11 supplies the acquired imaging data to the luminance data acquisition unit 12.

[0014] (Brightness Data Acquisition Unit) The brightness data acquisition unit 12 acquires brightness data indicating skin brightness from the imaging data acquired by the imaging data acquisition unit 11. More specifically, the brightness data acquisition unit 12 detects the subject's skin area (hereinafter, sometimes simply referred to as the "skin area") from the imaging data supplied from the imaging data acquisition unit 11 using a known image processing technique, and acquires brightness data from the skin area. The brightness data acquisition unit 12 supplies the acquired brightness data to the data processing unit 14.

[0015] As an example, the skin region detection unit 112 detects the subject's face according to known technology and detects multiple skin regions within the facial region based on facial feature quantities. Examples of skin regions included within the facial region include the forehead region, cheek region, and chin region. For the detected skin regions, the brightness data acquisition unit 12 calculates brightness data indicating changes in brightness. The brightness data is acquired as waveform data indicating changes in brightness. Since the brightness of the skin region in the captured image changes due to changes in blood flow, changes in the brightness data over time are observed as waveforms.

[0016] (Biometric Reference Data Acquisition Unit) The biometric reference data acquisition unit 13 acquires biometric reference data, such as pulse wave data, of the subject detected by a known contact sensor. A photoplethysmography pulse wave sensor can be used as the contact sensor. Photoplethysmography is a method that utilizes the light absorption characteristics of blood from the visible to near-infrared range to detect changes in blood vessel volume from changes in reflected or transmitted light associated with increases or decreases in blood volume. The biometric reference data is data serving as true values ​​or training data. When a pulse wave sensor is used, the biometric reference data is pulse wave data. The contact sensor is placed on the skin of the subject, such as the face. For example, in the example of FIG. 4 , the contact sensor is placed at the position indicated by the right cheek area RC. By attaching a contact sensor to the face to acquire true value data, the time difference or shape difference between the pulse wave in the facial image and the true value pulse wave waveform can be minimized, thereby improving the accuracy of model learning. The contact sensor acquires the biometric reference data from a skin area that is the same as, adjacent to, or similar to the skin area from which brightness data is acquired. Contact sensors do not capture biometric data from the fingertip.

[0017] Here, "similar" refers to skin regions that are symmetrically located on the left and right sides of the subject with respect to the subject's body center axis in the image. For example, if brightness data is acquired from the cheek region LC on the left side (the left side when viewing FIG. 4, i.e., the left side when viewing the subject from the front), the cheek region RC on the right side (the right side when viewing FIG. 4) is a similar skin region. Furthermore, "identical" does not mean physical, but is attribute-identical. For example, the left cheek region LC and the right cheek region RC are not physically identical because they are different parts, but because they both belong to the cheek, the left cheek region LC and the right cheek region RC are attribute-identical.

[0018] The biometric reference data acquiring unit 13 may acquire the biometric reference data thus acquired by the contact sensor directly from the contact sensor or from a memory (not shown) that stores the biometric reference data. The biometric reference data acquiring unit 13 supplies the acquired biometric reference data to the data processing unit 14.

[0019] As shown in Fig. 5A, the arrangement of blood vessels on the face is generally symmetrical. Therefore, as shown in Fig. 5B, by acquiring the luminance data and the biometric reference data from symmetrical skin regions, such as the left cheek region LC and the right cheek region RC, the phase difference (time difference) or waveform shape difference between the luminance data and the biometric reference data can be minimized.

[0020] As shown in Figure 6, biometric reference data may be acquired from multiple skin regions, i.e., in addition to acquiring biometric reference data from the right cheek region RC, biometric reference data may also be acquired from the right forehead region RF, which is a skin region adjacent to or similar to the left forehead region LF.

[0021] (Data Processing Unit) The data processing unit 14 generates training waveform data from the luminance data and the biometric reference data. The training waveform data is composed of a pair of processed luminance data that serves as the input for the model and processed biometric reference data that serves as the teacher for the model output.

[0022] When the skin area from which brightness data is acquired and the skin area from which biometric reference data is acquired are set symmetrically, brightness data cannot be acquired from the skin area on the biometric reference data acquisition side (right cheek area RC in the example of Figure 4), because the skin area on the biometric reference data acquisition side is blocked by the contact sensor. Note that when biometric data is estimated using a model, it is assumed that the brightness data of both cheeks is used for estimation.

[0023] Therefore, to address such occlusion issues, the data processing unit 14 duplicates the luminance data acquired in the left skin region based on the bilateral symmetry of blood vessels and uses it as luminance data acquired in the right skin region. In this way, the data processing unit 14 processes or complements the data. Note that the term "complement" is used interchangeably with the term "processing." Figure 7 , which shows an example of data processing, shows that luminance data acquired in the left cheek region LC is duplicated and used as luminance data acquired in the right cheek region RC. By processing data in this manner, it is possible to address the occlusion issue caused by the contact sensor.

[0024] Similarly, the contact sensor cannot acquire biometric reference data for the skin area on the side where brightness data is acquired. Therefore, the data processing unit 14 may duplicate the biometric reference data acquired for the right skin area using the contact sensor based on the bilateral symmetry of blood vessels, and use the duplicated data as biometric reference data for the left skin area. Figure 8 , which shows an example of data processing, adds to the case of Figure 7 by duplicating the biometric reference data acquired for the right cheek area RC and using it as biometric reference data for the left cheek area LC.

[0025] In this way, the data processing unit 14 may generate learning waveform data by complementing the luminance data of the skin region from which the biometric reference data is acquired with luminance data acquired in at least one skin region that is adjacent to or similar to the skin region from which the biometric reference data is acquired. Also, the data processing unit 14 may generate learning waveform data by complementing the biometric reference data of the skin region from which the luminance data is acquired with biometric reference data acquired in at least one skin region that is adjacent to or similar to the skin region from which the luminance data is acquired.

[0026] (Model Generation Unit) The model generation unit 15 uses the learning waveform data generated by the data processing unit 14 to generate a machine learning model that estimates biometric data from the processed brightness data. According to this machine learning model, for example, if the biometric reference data is pulse wave data, the estimated biometric data is pulse wave data. There are no particular restrictions on the learning method. As an example, the learning method described in the following Non-Patent Document 1 may be used. (Non-Patent Document 1) Turnip: Time-Series U-Net With Recurrence For Nir Imaging Ppg (signalprocessingsociety.org)

[0027] FIG. 9A shows a schematic diagram of Turnip, and FIG. 9B shows an enlarged view of the recursive network structure enclosed by the dashed line in FIG. 9A. As shown in FIG. 9A, for each frame, brightness data for each region of a person's face is acquired from a camera image, and a waveform of the brightness data is obtained. FIG. 9A illustrates the case where brightness data waveforms are obtained from 48 skin regions (channels; dimensions). This 48-dimensional time series data is supplied to a U-Net with a recursive network structure, which maps the received time series data to the desired biometric reference data. The recursive network structure is the structure enclosed by the dashed line in FIG. 9A. The recursive network structure is a structure provided in a skip connection connecting an encoder and a decoder.

[0028] The recurrent network structure, as shown in Fig. 9B, includes a 1x1 convolution layer and a recurrent skip connection in parallel to the convolution layer. The output of the convolution layer and the output of the recurrent skip connection are concatenated, and the concatenated result is transmitted to the decoder side.

[0029] For example, the input to Turnip is a time series (e.g., 10-second time window) of luminance values ​​acquired for each of multiple regions (e.g., 48 regions) on a face. A normalized blood pressure waveform (with the same time width as the input) is obtained as output from Turnip. If the training data is a waveform from a pulse oximeter, a pulse waveform is obtained as output from Turnip.

[0030] <Configuration; Inference Phase> The biological data estimation device 20 according to the first embodiment of the present disclosure will be described with reference to Fig. 2. The biological data estimation device 20 is a device that uses a machine learning model generated by the model generation device 10 to output an estimation result based on imaging data using the machine learning model. To achieve this purpose, the biological data estimation device 20 includes an imaging data acquisition unit 21, a luminance data acquisition unit 22, and an estimation unit 23.

[0031] (Imaging Data Acquisition Unit) The imaging data acquisition unit 21 is functionally identical to the imaging data acquisition unit 11 of the model generation device 10. The imaging data acquisition unit 21 acquires estimation imaging data including a human skin region.

[0032] (Brightness Data Acquisition Unit) The brightness data acquisition unit 22 is functionally identical to the brightness data acquisition unit 12 of the model generation device 10. The brightness data acquisition unit 22 acquires estimation brightness data indicating brightness changes in a human skin region from the estimation imaging data acquired by the imaging data acquisition unit 21.

[0033] (Estimation Unit) The estimation unit 23 inputs the time series of luminance data output from the luminance data acquisition unit 22 into the machine learning model generated by the model generation device 10, and acquires biometric data from the machine learning model. The estimation unit 23 outputs the acquired biometric data as an estimation result.

[0034] For example, as shown in Fig. 10 , the estimation unit 23 acquires a pulse wave waveform from a time series of brightness data using a machine learning model that enables pulse wave output for each region. Fig. 10 shows an example in which an estimated value of the pulse wave in the left cheek region LC is output from brightness data acquired from the left cheek region LC, and an estimated value of the pulse wave in the left forehead region LF is output from brightness data acquired from the left forehead region LF. Blood pressure can be estimated from the phase difference between the waveform of the estimated value of the pulse wave in the left cheek region LC and the waveform of the estimated value of the pulse wave in the left forehead region LF.

[0035] Next, examples of the hardware configuration of the model generating device 10 and the biological data estimating device 20 will be described with reference to Figures 18A and 18B. Each function of the model generating device 10 or the biological data estimating device 20 is realized by a processing circuitry. The processing circuitry may be a dedicated processing circuit 100a as shown in Figure 18A, or a processor 100b that executes a program stored in a memory 100c as shown in Figure 18B.

[0036] When the processing circuitry is a dedicated processing circuit 100a, the dedicated processing circuit 100a may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. The functions of the model generating device 10 or the biological data estimating device 20 may be realized by separate processing circuits, or the functions of the model generating device 10 and the biological data estimating device 20 may be realized together by a single processing circuit.

[0037] When the processing circuitry is a processor 100b, the functions of the model generating device 10 or the biological data estimating device 20 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 100c. The processor 100b realizes the functions of the model generating device 10 or the biological data estimating device 20 by reading and executing the programs stored in the memory 100c. Here, examples of the memory 100c include non-volatile or volatile semiconductor memories such as random access memory (RAM), read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM), as well as magnetic disks, flexible disks, optical disks, compact disks, minidisks, and DVDs.

[0038] Note that some of the functions of the model generating device 10 or the biological data estimating device 20 may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware. In this way, the processing circuit can realize the functions of the model generating device 10 and the biological data estimating device 20 by hardware, software, firmware, or a combination of these.

[0039] <Operation> Next, the operation of the model generating device 10 will be described with reference to Fig. 11. Fig. 11 is a flow diagram showing the operation of the model generating device 10.

[0040] (Step ST11) In step ST11, the imaging data acquisition unit 11 acquires imaging data including the skin of the subject, for example, the face.

[0041] (Step ST12) In step ST12, the luminance data acquisition unit 12 acquires luminance data indicating the brightness of the skin from the imaging data acquired by the imaging data acquisition unit 11.

[0042] (Step ST13) In step ST13, which is performed in parallel with steps ST11 and ST12, the biometric reference data acquiring unit 13 acquires the biometric reference data of the subject.

[0043] (Step ST14) In step ST14, the data processing unit 14 generates learning waveform data from the luminance data acquired by the luminance data acquisition unit 12 and the biometric reference data acquired by the biometric reference data acquisition unit 13. For example, luminance data acquired from a certain skin region on the face (e.g., left cheek region LC) is duplicated and set as luminance data for a skin region symmetrical to the original skin region (e.g., right cheek region RC), and this duplicated luminance data is paired with the biometric reference data acquired from the skin region symmetrical to the original skin region (e.g., right cheek region RC). This data pair is the learning waveform data.

[0044] (Step ST15) In step ST15, the model generation unit 15 uses the learning waveform data generated by the data processing unit 14 to generate a machine learning model that estimates biological data from the processed luminance data.

[0045] Next, the operation of the biological data estimating device 20 will be described with reference to Fig. 12. Fig. 12 is a flow chart showing the operation of the biological data estimating device 20.

[0046] (Step ST21) In step ST21, the imaging data acquisition unit 21 acquires imaging data including human skin, for example, a face.

[0047] (Step ST22) In step ST22, the luminance data acquisition unit 22 acquires luminance data indicating the brightness of the skin from the imaging data acquired by the imaging data acquisition unit 21.

[0048] (Step ST23) In step ST23, the estimation unit 23 inputs the time series of the luminance data output from the luminance data acquisition unit 22 into the machine learning model generated by the model generation device 10, and acquires biometric data from the machine learning model. The estimation unit 23 outputs the acquired biometric data as an estimated value.

[0049] Second Embodiment <Configuration> A biological data estimation system 30 according to a second embodiment of the present disclosure will be described with reference to Fig. 13 . The biological data estimation system 30 is a system that estimates blood pressure values ​​or changes in blood pressure based on pulse wave data for multiple skin regions output by the biological data estimation device 20. To achieve this objective, as shown in Fig. 13 , the biological data estimation system 30 has a configuration in which a blood pressure estimation unit 40 is added to the configuration of the biological data estimation device 20. Since the biological data estimation device 20 has already been described, the blood pressure estimation unit 40 will be described here.

[0050] (Blood Pressure Estimation Unit) The blood pressure estimation unit 40 is a functional unit that estimates blood pressure values ​​or changes in blood pressure based on pulse wave data in multiple skin regions output by the biological data estimation device 20. The principle of estimation performed by the blood pressure estimation unit 40 will be described with reference to Figs. 14A and 14B.

[0051] The blood pressure estimation unit 40 estimates the blood pressure value or change in blood pressure using the time difference (phase difference) of the pulse wave propagation through the face region. For example, pulse wave data at the chin or cheek and the forehead are acquired as estimation results, and the blood pressure value or change in blood pressure is estimated from the phase difference between the respective waveforms (see FIG. 14A ).

[0052] As can be seen from the table in FIG. 14B, when the blood pressure value is high, the blood propagates faster, resulting in a smaller phase difference between the chin or cheek and the forehead. On the other hand, when the blood pressure value is low, the phase difference is larger. Therefore, by linking these phase differences to the absolute values ​​of the blood pressure values ​​in advance, the blood pressure value can be estimated. Furthermore, it is also possible to detect changes in blood pressure, for example, a change from high to low blood pressure, from changes in the phase difference.

[0053] Therefore, the biological data estimation system 30 may include a storage device (not shown) that stores a table linking the phase difference between corresponding skin regions with the absolute value of the blood pressure value.

[0054] In this way, when calculating biometric data from the phase difference between data for different skin regions, a model trained using biometric reference data for the different skin regions is required. Therefore, in the second embodiment, the biometric reference data acquisition unit 13 included in the model generating device 10 acquires biometric reference data for multiple different skin regions, such as the cheeks, forehead, and chin.

[0055] However, as shown in FIG. 15, if the forehead is hidden by bangs, for example, the luminance data of the forehead cannot be acquired, and therefore the phase difference cannot be calculated.

[0056] Therefore, as shown in FIG. 16 , the model generation unit 15 may train the model without using the luminance data of the skin region that is expected to be occluded during model generation. The example of FIG. 16 illustrates a case in which, assuming that the forehead region is occluded by hair, the model is trained from the luminance data of the right cheek region RC, the left forehead region LF, and the left cheek region LC, without using the luminance data of the forehead region (right forehead region RF in the example of FIG. 16 ). That is, the biometric reference data includes biometric reference data acquired from a skin region (left cheek region LC) located symmetrically on the subject's body to the skin region (right cheek region RC) from which luminance data is acquired, and biometric reference data acquired from a skin region (left forehead region LF) different from this skin region (left cheek region LC). The model generation unit 15 generates a training model using the luminance data (luminance data of the right cheek region RC) and the biometric reference data, using the pulse wave data of the left cheek region LC and the pulse wave data of the left forehead region LF as training data. Using the learning model generated in this way, the pulse wave of the cheek region and the pulse wave of the forehead region can be estimated from the brightness data of the cheek region. Therefore, even if the forehead is obscured by hair and brightness data cannot be obtained from the forehead, blood pressure can be estimated from the phase difference between the pulse wave of the cheek region and the pulse wave of the forehead region.

[0057] The hardware configuration for realizing the blood pressure estimation unit 40 may be a dedicated processing circuit 100a as shown in FIG. 18A, or a processor 100b that executes a program stored in a memory 100c as shown in FIG. 18B.

[0058] It is possible to combine the embodiments, and to modify or omit each embodiment as appropriate.

[0059] The model generation device of the present disclosure can be used, for example, as a device for generating machine learning models to be used in a driver monitoring system that monitors vehicle drivers.

[0060] 10 Model generation device, 11 Imaging data acquisition unit, 12 Brightness data acquisition unit, 13 Biometric reference data acquisition unit, 14 Data processing unit, 15 Model generation unit, 20 Biometric data estimation device, 21 Imaging data acquisition unit, 22 Brightness data acquisition unit, 23 Estimation unit, 30 Biometric data estimation system, 40 Blood pressure estimation unit, 100a Processing circuit, 100b Processor, 100c Memory.

Claims

1. A model generation device comprising: an imaging data acquisition unit that acquires imaging data including a skin region of a subject; a brightness data acquisition unit that acquires brightness data indicating brightness changes in the skin region of the subject from the acquired imaging data; a biometric reference data acquisition unit that acquires biometric reference data of the subject, wherein the biometric reference data is acquired by a contact sensor from at least one skin region that is the same as, an adjacent skin region, or a similar skin region as the skin region from which the brightness data is acquired; a data processing unit that generates training waveform data from the acquired brightness data and the acquired biometric reference data; and a model generation unit that uses the generated training waveform data to perform machine learning using the acquired biometric reference data as training data to generate a machine learning model for estimating biometric data.

2. The model generating device according to claim 1, wherein the contact sensor is a pulse wave sensor using a photoplethysmography method, and the biological reference data is pulse wave data.

3. The model generating device according to claim 2, wherein the skin region from which the biometric reference data is acquired and the skin region from which the brightness data is acquired are located symmetrically on the left and right sides of the subject's body.

4. The model generating device of claim 3, wherein the data processing unit generates the learning waveform data by complementing the brightness data in the skin area from which the biometric reference data is obtained with brightness data obtained in at least one skin area that is adjacent to or similar to the skin area from which the biometric reference data is obtained.

5. The model generating device of claim 4, wherein the data processing unit generates the training waveform data by complementing the biometric reference data in the skin area from which the luminance data is obtained with biometric reference data obtained in at least one skin area that is adjacent to or similar to the skin area from which the luminance data is obtained.

6. The model generation device according to claim 5, wherein the machine learning model is a model that estimates biological data for a plurality of skin regions.

7. The model generating device according to claim 6, wherein the biometric reference data acquisition unit acquires the biometric reference data for a plurality of skin regions of the subject.

8. The model generation device described in claim 1, wherein the biometric reference data includes first biometric reference data acquired from a first skin area located symmetrically on the left and right of the skin area from which the brightness data is acquired, and second biometric reference data acquired from a second skin area different from the first skin area, and the model generation unit generates the machine learning model using the acquired brightness data, the acquired first biometric reference data, and the acquired second biometric reference data, using the acquired first biometric reference data and the acquired second biometric reference data as training data.

9. The model generating device according to claim 8, wherein the second skin region is either the forehead or the cheek.

10. A biometric data estimation device that estimates a person's biometric data using a machine learning model generated by a model generation device described in any one of claims 1 to 9, comprising: a second imaging data acquisition unit that acquires estimation imaging data including the person's skin region; a second luminance data acquisition unit that acquires estimation luminance data indicating luminance changes in the person's skin region from the acquired estimation imaging data; and an estimation unit that inputs the acquired luminance data for estimation into the machine learning model to acquire biometric data, and estimates the acquired biometric data as the person's biometric data.

11. The biological data estimation device according to claim 10, wherein the biological data is data relating to the person's blood pressure.

12. A model generation method performed by a model generation device comprising an imaging data acquisition unit, a luminance data acquisition unit, a biometric reference data acquisition unit, a data processing unit, and a model generation unit, the model generation method comprising: a step in which the imaging data acquisition unit acquires imaging data including a skin region of a subject; a step in which the luminance data acquisition unit acquires luminance data indicating luminance changes in the skin region of the subject from the acquired imaging data; a step in which the biometric reference data acquisition unit acquires biometric reference data of the subject, wherein the biometric reference data is acquired by a contact sensor from at least one skin region that is the same as, an adjacent skin region, or a similar skin region to the skin region from which the luminance data is acquired; a step in which the data processing unit generates training waveform data from the acquired luminance data and the acquired biometric reference data; and a step in which the model generation unit uses the generated training waveform data to perform machine learning using the acquired biometric reference data as training data, to generate a machine learning model for estimating biometric data.

13. A biometric data estimation method in which a biometric data estimation device comprising a second imaging data acquisition unit, a second luminance data acquisition unit, and an estimation unit estimates a person's biometric data using a machine learning model generated by a model generation device described in any one of claims 1 to 9, the method comprising the steps of: the second imaging data acquisition unit acquiring estimation imaging data including the person's skin region; the second luminance data acquisition unit acquiring estimation luminance data indicating luminance changes in the person's skin region from the acquired estimation imaging data; and the estimation unit inputting the acquired luminance data for estimation into the machine learning model to acquire biometric data, and estimating the acquired biometric data as the person's biometric data.

Images