Biometric information acquisition device and biological information acquisition program

Abstract

When tracking skin regions or regions of interest frame by frame, the same coordinates may not be detected due to factors such as the image scale and the relative position to the light source. [Solution] The biological information acquisition device comprises an imaging unit that images a living body and generates a plurality of frame image data, a face recognition unit that identifies a face image included in the frame image data and identifies a plurality of feature points in the face image, and a detection unit that detects the pulse wave signal of the living body. The detection unit selects a tracking feature point from the plurality of feature points, tracks the tracking feature point included in each of the plurality of frame image data, and acquires the pulse wave signal based on the tracking feature point detection light amount of each of the plurality of frame image data.

Description

[Technical Field]

[0001] This disclosure relates to a biological information acquisition device and a biological information acquisition program. [Background technology]

[0002] A biometric information acquisition device is known that measures biological information such as pulse waves and blood pressure of a subject. The biometric information measurement device described in Patent Document 1 is an example of a biometric information acquisition device. The biometric information measurement device extracts a region of interest set within the subject's skin area based on the brightness value of the video signal. The biometric information measurement device tracks the coordinates of the region of interest for each frame that makes up the video and obtains the coordinates of the tracking result. The biometric information measurement device extracts the green signal of the region of interest and calculates the pulse wave based on the green signal. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2016-190022 [Overview of the project] [Problems that the invention aims to solve]

[0004] When tracking skin regions or regions of interest frame by frame, the same coordinates may not be detected due to factors such as the image scale and the relative position to the light source. [Means for solving the problem]

[0005] The biological information acquisition device of this disclosure comprises an imaging unit that images a living body and generates a plurality of frame image data, a face recognition unit that identifies a face image included in the frame image data and identifies a plurality of feature points in the face image, and a detection unit that detects the pulse wave signal of the living body. The detection unit selects a tracking feature point from the plurality of feature points, tracks the tracking feature point included in each of the plurality of frame image data, and acquires the pulse wave signal based on the tracking feature point detection light amount of each of the tracking feature points in the plurality of frame image data.

[0006] The biological information acquisition program of this disclosure causes a computer connected to an imaging unit that images a living body and generates multiple frame image data to identify a face image included in the frame image data, identify multiple feature points in the face image, select a tracking feature point included in the multiple feature points, track the tracking feature point included in each of the multiple frame image data, and acquire a pulse wave signal based on the tracking feature point detection light intensity of each of the tracking feature points in the multiple frame image data. [Brief explanation of the drawing]

[0007] [Figure 1] A diagram showing the schematic configuration of the measuring device. [Figure 2] A diagram showing the block configuration of the measuring device. [Figure 3] A diagram showing an image captured with facial mesh information. [Figure 4] A diagram showing a portion of the facial mesh information. [Figure 5] A diagram showing an example of grayscale detection values. [Figure 6] A diagram showing an example of mesh point-related information. [Figure 7] A diagram showing an example of the control flow performed by the measuring device. [Figure 8] This figure shows the time-series data of the nth facial feature point. [Figure 9] A diagram illustrating an example of an analysis procedure for detecting pulse wave signals. [Figure 10] A figure showing the analysis results of the noise-reduced signal. [Figure 11] A diagram showing an example of the control flow performed by the measuring device. [Figure 12] A diagram showing an example of the control flow performed by the measuring device. [Figure 13] A diagram showing an example of the control flow performed by the measuring device. [Figure 14] A diagram showing an example of the control flow performed by the measuring device. [Modes for carrying out the invention]

[0008] Figure 1 shows a schematic configuration of the measuring device 10. The measuring device 10 corresponds to an example of a biological information acquisition device. The measuring device 10 detects the pulse wave signal of the observer M using video data. The pulse wave signal is a signal that shows the pulse wave of the observer M. Observer M corresponds to an example of a living organism. Based on the pulse wave signal, the measuring device 10 calculates the biological information of the observer M. The biological information includes pulse rate, pulse rate variability, oxygen saturation concentration, blood pressure, etc. Based on the biological information, the measuring device 10 may also perform evaluations of sleep apnea syndrome, etc. The measuring device 10 displays the biological information calculated based on the pulse wave signal.

[0009] The measuring device 10 is composed of an information processing device such as a personal computer. The measuring device 10 shown in Figure 1 is a laptop computer, but is not limited to this. The measuring device 10 can be any device that has the function of capturing video, or a device that can be connected to a device that captures video. The measuring device 10 is composed of a desktop computer, a tablet terminal, a smartphone, etc. The measuring device 10 includes an imaging unit 11, a display unit 13, and an input unit 15. The measuring device 10 may also include a communication unit, etc., which is not shown.

[0010] The imaging unit 11 captures an image of the measurer M by receiving reflected light, external light, etc. reflected by the measurer M, etc. as detection light. The imaging unit 11 generates moving image data including the face of the measurer M. The moving image data is data for displaying a video and is composed of a plurality of image data. The imaging unit 11 generates a plurality of image data. The image data corresponds to an example of frame image data. The image data is composed of a plurality of output values output in pixel units. The image data is data for causing the display unit 13 to display the captured image 100. The captured image 100 is a still image. The imaging unit 11 captures images at a predetermined frame rate and generates moving image data. The imaging unit 11 corresponds to an example of an imaging unit.

[0011] As an example, the imaging unit 11 is a camera including an optical element, an imaging element, etc. The optical element condenses light onto the imaging element. The imaging element converts the detection light into output values of an electrical signal. The imaging element generates output values for each of a plurality of pixels. The output value of each pixel indicates the intensity of light of each pixel. The imaging element generates the output value of each pixel. The imaging element is a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc. The output value includes gradation values of a plurality of color lights. The imaging unit 11 generates gradation values of a plurality of color lights for each pixel. The plurality of color lights are, as an example, red light, green light, and blue light. Red light, green light, and blue light are lights with different wavelength bands respectively. The red wavelength band, which is the wavelength band of red light, is 600 nm to 800 nm. The green wavelength band, which is the wavelength band of green light, is 520 nm to 550 nm. The blue wavelength band, which is the wavelength band of blue light, is 430 nm to 490 nm. The imaging unit 11 may include IR (Infrared) color light, etc. The imaging unit 11 generates, for each pixel, a red gradation value that is a gradation value of red light, a green gradation value that is a gradation value of green light, and a blue gradation value that is a gradation value of blue light. The image data is composed of output values for each pixel including the red gradation value, the green gradation value, and the blue gradation value. The image data includes the luminance, etc. of each pixel.

[0012] The imaging unit 11 shown in FIG. 1 is a camera built into the measuring device 10, but is not limited thereto. The imaging unit 11 may also be an external camera connected to the measuring device 10. The external camera may be a near-infrared camera, a web camera, a smartphone camera, or the like.

[0013] The display unit 13 displays various information such as the captured image 100. The display unit 13 displays various biological information based on the pulse wave signal. The display unit 13 may display comments or the like based on the biological information. The display unit 13 is composed of a liquid crystal panel, an organic EL (electro-luminescence) panel, or the like. The display unit 13 may have a touch input function. When having the touch input function, the display unit 13 functions as the input unit 15. The display unit 13 shown in FIG. 1 is included in the measuring device 10, but is not limited thereto. The display unit 13 may also be a display externally attached to the measuring device 10.

[0014] The input unit 15 receives various input operations by the measurer M. The input unit 15 generates various input signals according to the input operations. The input unit 15 shown in FIG. 1 is a keyboard included in the measuring device 10, but is not limited thereto. The input unit 15 may also be a mouse, a keyboard, a touch panel, a tablet, or the like connected to the measuring device 10.

[0015] The measurer M operates the measuring device 10 at a position facing the imaging unit 11 of the measuring device 10. The measurer M operates the measuring device 10 when detecting biological information with the measuring device 10. The measurer M may operate the measuring device 10 when performing tasks such as document creation. The measuring device 10 detects the biological information of the measurer M in the background when the measurer M is performing tasks such as document creation. By detecting the biological information in the background, the measuring device 10 can detect the biological information of the measurer M in the normal activity state.

[0016] Figure 2 shows the block configuration of the measuring device 10. The measuring device 10 comprises an imaging unit 11, a display unit 13, an input unit 15, a control unit 31, and a storage unit 41.

[0017] The imaging unit 11 transmits video data to the control unit 31. The imaging unit 11 transmits video data to the control unit 31 at predetermined timings. The imaging unit 11 may also transmit image data included in the video data to the control unit 31 at predetermined time intervals. The imaging unit 11 transmits image data consisting of output values ​​including red, green, and blue gradation values ​​for each pixel to the control unit 31. The imaging unit 11 may also transmit video data to the storage unit 41 and have the storage unit 41 store the video data.

[0018] The display unit 13 displays various images based on the control of the control unit 31. The display unit 13 receives display data from the control unit 31 and displays various images based on the display data. The display unit 13 may also display video captured by the imaging unit 11 based on video data. The display unit 13 may also display captured images 100 based on image data included in the video data.

[0019] The input unit 15 transmits an input signal to the control unit 31. By transmitting the input signal to the control unit 31, the input unit 15 causes the control unit 31 to perform various controls. As an example, the input unit 15 transmits a display instruction signal to the control unit 31 to display biological information. The display instruction signal is an example of an input signal. Based on the display instruction signal, the control unit 31 generates biological information display data to display biological information based on the pulse wave signal on the display unit 13. The control unit 31 transmits the biological information display data to the display unit 13. Based on the biological information display data, the display unit 13 displays a screen containing the biological information.

[0020] The control unit 31 is a control controller that controls the operation of various units. The control unit 31 is, for example, a processor having a CPU (Central Processing Unit). The control unit 31 is composed of one or more processors. The control unit 31 is connected to the imaging unit 11, the display unit 13, etc., in a communicative manner. By executing the biological analysis program PG, the control unit 31 functions as an image recognition processing unit 33, a data processing unit 35, and a display control unit 37. The control unit 31 may also function as a functional unit other than the image recognition processing unit 33, the data processing unit 35, and the display control unit 37 by executing the biological analysis program PG. The control unit 31 corresponds to an example of a computer.

[0021] The image recognition processing unit 33 acquires video data transmitted from the imaging unit 11. The image recognition processing unit 33 acquires multiple image data contained in the video data. The image recognition processing unit 33 identifies face images contained in the image data. The image recognition processing unit 33 identifies face images by performing face recognition processing on each image data. Face recognition processing is a process that detects face image feature points contained in the image data and extracts the face image region 121 by matching the face image feature points with a pre-registered face image database. The face image database is a database that stores information related to face image feature points used for face recognition. Examples of face image feature points contained in the image data are the positions and contours of the eyes, nose, and mouth. The face image database is pre-stored in the storage unit 41. If the measuring device 10 is connected to a server via a network, the face image database may be pre-stored on the server. The face image region 121 is the region where the face of the measurer M is displayed. The image recognition processing unit 33 identifies the face image region 121 by performing face recognition processing. The image recognition processing unit 33 corresponds to an example of a face recognition unit.

[0022] For face recognition processing, one example is the use of FaceMesh (hereinafter referred to as FaceMesh), which is included in the Media Pipe provided by Google. FaceMesh is a machine learning model that detects keypoints of a face from an image. The image recognition processing unit 33 performs face recognition processing using FaceMesh on each of the multiple image data, and obtains the face image region 121 and face mesh information 131. The face mesh information 131 is represented by multiple keypoints contained in the face image region 121.

[0023] Figure 3 shows an image 100 containing facial mesh information 131. Figure 3 shows an example of facial mesh information 131. Facial mesh information 131 varies depending on the measurer M. Figure 3 shows mesh points 131a and mesh lines 131b representing the facial mesh information 131.

[0024] A mesh point 131a is a point corresponding to a keypoint. For example, there are 468 keypoints. The image recognition processing unit 33 identifies multiple mesh points 131a contained within the face image region 121. The image recognition processing unit 33 identifies multiple mesh points 131a for each of the multiple image data. The number of mesh points 131a identified for each of the multiple image data is the same. Each of the multiple mesh points 131a is assigned a predetermined code. Mesh points 131a with the same code, from among the multiple mesh points 131a contained in each of the multiple image data, correspond to the same position within the face image region 121. A mesh point 131a corresponds to an example of a feature point. A mesh line 131b is a line connecting two adjacent mesh points 131a.

[0025] Figure 4 shows a portion of the facial mesh information 131. Figure 4 shows a magnified view of the facial mesh information 131 near the left eye of the measurer M. Figure 4 shows multiple mesh points 131a and multiple mesh lines 131b. Figure 4 shows the code names of some of the mesh points 131a among the multiple mesh points 131a.

[0026] Figure 4 shows the code names assigned to some of the mesh points 131a. Examples of code names assigned to mesh points 131a include the first code name TBL-01, the second code name TBL-02, the third code name TBL-03, and the fourth code name TBL-04. The code names shown in Figure 4 represent the tear trough of the left eye of the measurer M. The code name configuration can be set as appropriate. The image recognition processing unit 33 assigns a unique code name to each of the multiple mesh points 131a.

[0027] The data processing unit 35 shown in Figure 2 acquires video data transmitted from the imaging unit 11. The data processing unit 35 acquires multiple image data contained in the video data. The data processing unit 35 acquires the red tone value, green tone value, blue tone value, pixel brightness, etc., for each pixel contained in the image data. The data processing unit 35 appropriately performs data processing such as correction processing on the red tone value, green tone value, blue tone value, etc.

[0028] The data processing unit 35 detects the pulse wave signal of the measurer M. The data processing unit 35 analyzes biological information based on the pulse wave signal. The data processing unit 35 corresponds to an example of a detection unit. The data processing unit 35 acquires image data from the imaging unit 11, consisting of output values ​​for each pixel, including red gradation values, green gradation values, and blue gradation values. The data processing unit 35 acquires face mesh information 131 from the image recognition processing unit 33. The data processing unit 35 associates the mesh points 131a included in the face mesh information 131 with the output values ​​of the pixels. For each of the multiple image data, the data processing unit 35 associates the mesh points 131a with the output values ​​of the pixels.

[0029] The data processing unit 35 tracks predetermined mesh points 131a contained in the face mesh information 131 of each of the multiple image data. The multiple image data is generated in a time series. The data processing unit 35 tracks the positions of predetermined mesh points 131a contained in each of the multiple image data generated in a time series by identifying the positions of predetermined mesh points 131a contained in each of the multiple image data.

[0030] The data processing unit 35 may track one of the multiple mesh points 131a, or it may track all of the multiple mesh points 131a. The data processing unit 35 may also track one or more mesh points 131a selected from the multiple mesh points 131a based on predetermined conditions. The selection of mesh points 131a is set as appropriate.

[0031] The data processing unit 35 associates a predetermined mesh point 131a with the output value of a pixel for each image data. The data processing unit 35 acquires the output value associated with the predetermined mesh point 131a. The data processing unit 35 acquires the output value associated with the predetermined mesh point 131a for each image data. By acquiring the output value associated with the predetermined mesh point 131a for each image data generated in time series, the data processing unit 35 acquires a group of time-series output values ​​associated with the predetermined mesh point 131a.

[0032] The data processing unit 35 detects the pulse wave signal of the measurer M using a group of output values ​​associated with a predetermined mesh point 131a. The data processing unit 35 generates a detected value using multiple output values ​​included in the group of output values. The detected value includes a grayscale detected value calculated using each grayscale value. The detected value may be a group of output values ​​associated with one mesh point 131a, or a calculated value calculated based on multiple groups of output values ​​associated with each of multiple mesh points 131a. The calculated value is the average value of the output values ​​associated with each of the multiple mesh points 131a for each image data. The detected value is calculated for each image data within the video data. The data processing unit 35 acquires the pulse wave signal based on the detected value of a predetermined mesh point 131a included in each of the multiple image data.

[0033] The data processing unit 35 detects the pulse wave signal using each grayscale detection value included in the detected value. As an example, the data processing unit 35 detects the pulse wave signal using at least one of the red grayscale detection value Dr, the green grayscale detection value Dg, and the blue grayscale detection value Db. The red grayscale detection value Dr is calculated using the red grayscale value. The green grayscale detection value Dg is calculated using the green grayscale value. The blue grayscale detection value Db is calculated using the blue grayscale value. The pulse wave signal is detected based on the green grayscale detection value Dg. The pulse wave signal is detected based on the difference between the green grayscale detection value Dg and at least one of the red grayscale detection value Dr and the blue grayscale detection value Db.

[0034] Figure 5 shows an example of grayscale detection values. Figure 5 shows the red grayscale detection value Dr, the green grayscale detection value Dg, and the blue grayscale detection value Db. Figure 5 shows the time-dependent changes of the red grayscale detection value Dr, the green grayscale detection value Dg, and the blue grayscale detection value Db as waveform signals. Figure 5 shows the red grayscale detection value Dr, the green grayscale detection value Dg, and the blue grayscale detection value Db in the body movement interval S1 and the red grayscale detection value Dr, the green grayscale detection value Dg, and the blue grayscale detection value Db in the resting interval S2. The body movement interval S1 is the interval in which facial movement and changes in expression occur. The resting interval S2 is the interval in which facial movement and changes in expression are smaller than a predetermined amount of change.

[0035] Figure 5 shows the green tone detection value Dg. The green tone detection value Dg corresponds to the amount of green light contained in the output value at one or more predetermined mesh points 131a. The green tone detection value Dg is included in the detected value.

[0036] As shown in Figure 5, in the movement interval S1, the green grayscale detection value Dg fluctuates due to the effects of body movement. The pulse wave signal included in the green grayscale detection value Dg is difficult to detect due to the fluctuating noise. In the resting interval S2, the influence of the fluctuating noise caused by body movement on the green grayscale detection value Dg decreases, and the pulse wave signal becomes detectable.

[0037] Figure 5 shows the red tone detection value Dr. The red tone detection value Dr corresponds to the light intensity value of red light included in the output value at one or more predetermined mesh points 131a. The red tone detection value Dr is included in the detected value.

[0038] As shown in Figure 5, in the motion interval S1, the red gradation detection value Dr fluctuates due to the effects of body movement. The pulse wave signal included in the red gradation detection value Dr is difficult to detect due to fluctuating noise. In the resting interval S2, the effect of fluctuating noise due to body movement on the red gradation detection value Dr decreases, but the signal-to-noise ratio is small, making it difficult to detect the pulse wave signal.

[0039] Figure 5 shows the blue tone detection value Db. The blue tone detection value Db corresponds to the amount of blue light contained in the output value at one or more predetermined mesh points 131a. The blue tone detection value Db is included in the detected value.

[0040] As shown in Figure 5, in the motion interval S1, the blue gradation detection value Db fluctuates due to the effects of body movement. The pulse wave signal included in the blue gradation detection value Db is difficult to detect due to fluctuating noise. In the resting interval S2, the effect of fluctuating noise due to body movement on the blue gradation detection value Db decreases, but the signal-to-noise ratio is small, making it difficult to detect the pulse wave signal.

[0041] The data processing unit 35 detects the pulse wave signal using the red gradation detection value Dr, the green gradation detection value Dg, and the blue gradation detection value Db shown in Figure 5. The data processing unit 35 detects the pulse wave signal using the analysis procedure described later.

[0042] The data processing unit 35 shown in Figure 2 calculates biological information such as pulse rate by calculating the period, amplitude, etc., of the pulse wave signal. The data processing unit 35 transmits the biological information, including the pulse wave signal, to the display control unit 37. The data processing unit 35 may also store the biological information, etc., in the storage unit 41.

[0043] The display control unit 37 controls the display by the display unit 13. The display control unit 37 acquires pulse wave signals and biological information from the data processing unit 35. The display control unit 37 generates biological information display data including biological information. The display control unit 37 transmits the biological information display data to the display unit 13. The display control unit 37 displays the biological information display data on the display unit 13. By displaying the biological information display data on the display unit 13, the display control unit 37 can notify the person taking the measurement M of the detection results of the biological information.

[0044] The display control unit 37 may generate message data indicating the operating status of the biological analysis program PG. The message data may include a start message, an execution message, and an end message. The start message indicates that the detection of biological information has begun. The execution message indicates that biological information is being detected. The end message indicates that the detection of biological information has ended. The display control unit 37 transmits the message data to the display unit 13. The display control unit 37 causes the display unit 13 to display the message data.

[0045] The memory unit 41 stores various programs, various data, etc. The memory unit 41 stores the bioanalysis program PG and mesh point-related information MT. The memory unit 41 stores document creation programs, spreadsheet programs, etc. The memory unit 41 may also store various detection data such as video data and biometric information. The memory unit 41 may also store a facial image database. The memory unit 41 is composed of semiconductor memory such as RAM (Random Access Memory) and ROM (Read Only Memory). The memory unit 41 may have an HDD (Hard Disk Drive). The memory unit 41 may also function as a work area for the control unit 31.

[0046] The bioanalysis program PG is a program that causes the measuring device 10 to detect pulse wave signals. The bioanalysis program PG is executed by the control unit 31. By executing the bioanalysis program PG in the control unit 31, the control unit 31 functions as various functional units. The bioanalysis program PG detects various biological information based on the pulse wave signals. The bioanalysis program PG may be executed in the background when the control unit 31 is running a document creation program or the like. The bioanalysis program PG corresponds to an example of a biological information acquisition program.

[0047] The mesh point association information MT is information relating to the adjacent mesh points 131a of each of the multiple mesh points 131a. The mesh point association information MT is generated when the image recognition processing unit 33 performs face recognition processing and is stored in the storage unit 41. The mesh point association information MT is used when the data processing unit 35 selects one or more mesh points 131a from the multiple mesh points 131a. The mesh point association information MT corresponds to an example of linked information.

[0048] Figure 6 shows an example of mesh point-related information MT. Figure 6 shows mesh point-related information MT in table format. Figure 6 shows information related to mesh point 131a for the first code name TBL-01, second code name TBL-02, third code name TBL-03, and fourth code name TBL-04 shown in Figure 4.

[0049] The mesh point association information MT shows each mesh point 131a and the adjacent mesh points 131a. Two adjacent mesh points 131a are connected by a mesh line 131b. The mesh point association information MT shows two or more mesh points 131a that are connected to one mesh point 131a by a mesh line 131b. As an example, the mesh point association information MT shows that mesh point 131a of the first code name TBL-01 is adjacent to mesh point 131a of the second code name TBL-02.

[0050] Figure 7 shows an example of the control flow performed by the measuring device 10. Figure 7 shows the control flow for acquiring a pulse wave signal using the output value corresponding to the mesh point 131a. The control flow is performed by running the bioanalysis program PG. Figure 7 shows the control flow as a flowchart.

[0051] In step S101, the measuring device 10 acquires video data. The measuring device 10 causes the imaging unit 11 to generate video data. The imaging unit 11 images the person being measured M and generates video data. The video data includes multiple image data. The multiple image data is generated in a time series. The imaging unit 11 generates multiple image data. For example, when the frame rate when the imaging unit 11 generates the video data is 30 fps (frames per second) and the measurement time is 8 seconds, the number of image data will be 240. The imaging unit 11 transmits the video data to the control unit 31.

[0052] After acquiring video data, the measuring device 10 starts acquiring time-series data in step S103. The data processing unit 35 of the control unit 31 acquires the video data transmitted from the imaging unit 11. The control unit 31 acquires multiple image data contained in the video data. The data processing unit 35 acquires time-series data for each mesh point 131a by performing the processing from step S103 to step S111. Details of the time-series data will be described later.

[0053] In step S105, the measuring device 10 acquires image data. The image recognition processing unit 33 of the control unit 31 sequentially acquires the image data generated in time series. When the video data contains k image data, the image recognition processing unit 33 sequentially acquires the first to the kth image data. k is an arbitrary integer. k is set by the frame rate of the video data and the measurement time.

[0054] After acquiring image data, the measuring device 10 performs face recognition processing in step S107. The image recognition processing unit 33 performs face recognition processing for each of the multiple image data. The image recognition processing unit 33 generates face mesh information 131 contained in each image data. The face mesh information 131 includes multiple mesh points 131a and multiple mesh lines 131b. The number of mesh points 131a contained in each image data is the same.

[0055] After performing face recognition processing, the measuring device 10 identifies the mesh point coordinates of each mesh point 131a in step S109. The data processing unit 35 acquires face mesh information 131 for each image data. The data processing unit 35 identifies the mesh point coordinates of multiple mesh points 131a included in each image data. Mesh point coordinates are x and y coordinates with a predetermined position in the image data as the origin. Face mesh information 131 includes N mesh points 131a, including the nth mesh point 131a. The nth mesh point coordinate Cn, which is the mesh point coordinate of the nth mesh point 131a in one image data, is expressed by the following equation (1). Cn=(xn,yn) (1) Here, n is any integer from 1 to N.

[0056] The nth mesh point 131a contained in each of the multiple image data points indicates the same position within the face image region 121. The coordinates Cn of the nth mesh point in each of the multiple image data points vary depending on the position and orientation of the measurer M's face in the image data. The control unit 31 can track the nth mesh point 131a over time by acquiring the coordinates Cn of the nth mesh point contained in each image data point. The control unit 31 can track the positions of all mesh points 131a over time by acquiring the coordinates Cn of the nth mesh point in each of the multiple image data points.

[0057] After identifying each mesh point coordinate, the measuring device 10 acquires the output value of each mesh point coordinate in step S111. The data processing unit 35 acquires the output value of the pixel at the position corresponding to the mesh point coordinate as the output value of the mesh point coordinate. The data processing unit 35 may also acquire the output value of the mesh point coordinate based on the output values ​​of the pixel at the position corresponding to the mesh point coordinate and the output values ​​of surrounding pixels which are pixels within a predetermined area for the pixel at the position corresponding to the mesh point coordinate. The predetermined area is set in advance. When acquiring the output values ​​of surrounding pixels, the data processing unit 35 may, as an example, acquire the average value, aggregated value, etc., of the output value of the output value of the pixel at the position corresponding to the mesh point coordinate and the output values ​​of surrounding pixels as the output value of the mesh point coordinate. The nth mesh point output value Bn, which is the output value of the nth mesh point coordinate Cn within a single image data, is expressed by the following equation (2). Bn = (rn, gn, bn) (2) Here, rn is the red gradation value included in the output value of the nth mesh point coordinate Cn, gn is the green gradation value included in the output value of the nth mesh point coordinate Cn, and bn is the blue gradation value included in the output value of the nth mesh point coordinate Cn.

[0058] The data processing unit 35 obtains the nth face feature point data Dn in the form of the following equation (3) using the coordinates Cn of the nth mesh point and the output value Bn of the nth mesh point within a single image data. Dn=(xn,yn,rn,gn,bn) (3)

[0059] The data processing unit 35 acquires the nth facial feature point data Dn for each image data. By acquiring the nth facial feature point data Dn for each image data, the data processing unit 35 acquires the nth facial feature point time series data Dn(t). The nth facial feature point time series data Dn(t) is an example of time series data of the nth mesh point coordinate Cn. The nth facial feature point time series data Dn(t) includes the nth facial feature point data Dn(T) acquired at time T with a frame rate of dt, the nth facial feature point data Dn(T-dt) acquired at time T-dt, and the nth facial feature point data Dn(T+dt) acquired at time T+dt. Here, time T is any time between the start of measurement and the end of measurement. The nth facial feature point time series data Dn(t) is shown in Figure 8.

[0060] The data processing unit 35 obtains the nth face feature point time series data Dn(t) shown in Figure 8 by acquiring the nth face feature point data Dn at the nth mesh point coordinate Cn for each image data. The nth face feature point time series data Dn(t) is the time series data of the nth mesh point 131a tracked for each image data.

[0061] After acquiring the output values ​​of each mesh point coordinate, the measuring device 10 determines in step S113 whether or not it has acquired the output values ​​of all mesh point coordinates. The data processing unit 35 determines whether or not it has acquired all nth face feature point time series data Dn(t) from n = 1 to N. If the data processing unit 35 determines that it has acquired all nth face feature point time series data Dn(t), the measuring device 10 proceeds to step S115 (step S113: YES). If the data processing unit 35 determines that it has not acquired all nth face feature point time series data Dn(t), the measuring device 10 returns to step S109 (step S113: NO). The measuring device 10 continues to acquire the nth face feature point time series data Dn(t).

[0062] The measuring device 10 finishes acquiring time-series data in step S115. The data processing unit 35 acquires time-series data Dn(t) of the nth facial feature point, where n is from 1 to N, and finishes acquiring time-series data.

[0063] After acquiring time-series data, the measuring device 10 selects a measurement point in step S117. The data processing unit 35 selects one mesh point 131a from among a plurality of mesh points 131a as a tracking measurement point. A tracking measurement point is an example of a measurement point and corresponds to an example of a tracking feature point. The data processing unit 35 may also select multiple mesh points 131a, including the one mesh point 131a selected as a tracking measurement point, as analysis measurement points. The data processing unit 35 may also select all mesh points 131a, including the one mesh point 131a selected as a tracking measurement point, as analysis measurement points. An analysis measurement point is an example of a measurement point and corresponds to an example of a measurement feature point. The method for selecting measurement points will be described later.

[0064] After selecting a measurement point, the measuring device 10 acquires a pulse wave signal in step S119. The data processing unit 35 acquires the pulse wave signal using the nth facial feature point time series data Dn(t) of the measurement point. The nth facial feature point time series data Dn(t) includes a group of output values ​​associated with a predetermined mesh point 131a. The group of output values ​​is the nth mesh point output value Bn included in the nth facial feature point time series data Dn(t). The data processing unit 35 generates detection values ​​using the nth mesh point output value Bn. The data processing unit 35 detects the red gradation detection value Dr, the green gradation detection value Dg, and the blue gradation detection value Db shown in Figure 5 using the nth facial feature point time series data Dn(t).

[0065] Figure 9 shows an example of the analysis procedure for detecting a pulse wave signal. Figure 9 shows an example of the analysis procedure as a flowchart. The analysis procedure shown in Figure 9 is executed by the data processing unit 35. The analysis procedure shown in Figure 9 detects the pulse wave signal using the red gradation detection value Dr, the green gradation detection value Dg, and the blue gradation detection value Db.

[0066] In step S201, the data processing unit 35 performs sampling of each grayscale detection value at predetermined time intervals. The time interval and sampling frequency are set as appropriate. Preferably, the time interval is a time that includes one or more pulse waves. For example, the time interval is 3 to 10 seconds. For example, the sampling frequency is 10 Hz to 50 Hz. The data processing unit 35 acquires sampled data by performing sampling. The sampled data includes the sampled red grayscale detection value Dr, the green grayscale detection value Dg, and the blue grayscale detection value Db.

[0067] After performing sampling, the data processing unit 35 normalizes the sampled data in step S203. The data processing unit 35 normalizes the green tone detection value Dg included in the sampled data.

[0068] The data processing unit 35 calculates the green mean Gmean, which is the average of multiple green tone detection values ​​Dg, and the green standard deviation Gstd, which is the standard deviation of multiple green tone detection values ​​Dg. The data processing unit 35 normalizes each green tone detection value Dg using the following equation (4). Gnorm m =(Gm-Gmean) / Gstd (4) Here, m is any integer greater than or equal to 1. Gm is the m-th green tone detection value Dg. Gnorm m This is the normalized value of the m-th green tone detection value Dg.

[0069] Similar to the green gradation detection value Dg, the data processing unit 35 normalizes a plurality of red gradation detection values Dr and a plurality of blue gradation detection values Db included in the sampling data. The data processing unit 35 calculates a red average value Rmean, which is the average value of the plurality of red gradation detection values Dr, and a red standard deviation value Rstd, which is the standard deviation value of the plurality of red gradation detection values Dr. The data processing unit 35 calculates a blue average value Bmean, which is the average value of the plurality of blue gradation detection values Db, and a blue standard deviation value Bstd, which is the standard deviation value of the plurality of blue gradation detection values Db. The data processing unit 35 normalizes each red gradation detection value Dr and each blue gradation detection value Db using the following formula (5) and the following formula (6). Rnorm m =(Rm - Rmean) / Rstd (5) Bnorm m =(Bm - Bmean) / Bstd (6) Here, m is an arbitrary integer of 1 or more. Rm is the m-th red gradation detection value Dr. Rnorm m is the value obtained by normalizing the m-th red gradation detection value Dr. Bm is the m-th blue gradation detection value Db. Bnorm m is the value obtained by normalizing the m-th blue gradation detection value Db.

[0070] After normalizing the sampling data, the data processing unit 35 performs noise removal processing in step S205. The data processing unit 35 performs noise removal processing using the normalized green gradation detection value Dg, the normalized red gradation detection value Dr, and the normalized blue gradation detection value Db. The data processing unit 35 performs noise removal processing using the following formula (7) to generate a noise removal signal S. Sm = Gnorm m + αBnorm m + βRnorm m (7) Here, m is an arbitrary integer of 1 or more. Sm is the m-th noise removal signal S. α is the first coefficient, and β is the second coefficient.

[0071] For example, α and β are -0.5 and -0.5, respectively. When α and β are negative values, the data processing unit 35 detects the noise reduction signal S by subtracting the normalized red tone detection value Dr and the normalized blue tone detection value Db from the normalized green tone detection value Dg. At least one of α and β may be 0. When α=0 and β=-0.5, the data processing unit 35 detects the noise reduction signal S by calculating the difference between the green tone detection value Dg and the red tone detection value Dr. When α=-0.5 and β=0, the data processing unit 35 detects the noise reduction signal S by calculating the difference between the green tone detection value Dg and the blue tone detection value Db. α and β are set appropriately depending on the noise reduction status.

[0072] Figure 10 shows the analysis results of the noise-reduced signal S. Figure 10 is analyzed based on the red gradation detection value Dr, green gradation detection value Dg, and blue gradation detection value Db shown in Figure 5. Figure 10 shows the noise-reduced signal S when α = -0.5 and β = -0.5 are substituted into equation (7). Figure 10 shows the noise-reduced signal S for the motion section S1 and the resting section S2.

[0073] As shown in Figure 10, the denoised signal S corresponds to the pulse wave signal. Noise components such as body movement are removed from the denoised signal S. The data processing unit 35 detects the denoised signal S as a pulse wave signal. In the resting section S2, the denoised signal S has a more clearly detectable signal waveform compared to the green gradation detection value Dg. In the moving section S1, the denoised signal S is adjusted to a signal waveform corresponding to the pulse wave signal. Through the noise reduction process, the data processing unit 35 can detect the pulse wave signal in both the moving section S1 and the resting section S2.

[0074] The data processing unit 35 may calculate biological information such as pulse waves using the noise reduction signal S. The data processing unit 35 acquires the noise reduction signal S as a pulse wave signal. The data processing unit 35 calculates biological information such as pulse rate by calculating the period, amplitude, etc., of the pulse wave signal. The data processing unit 35 transmits the biological information, including the pulse wave signal, to the display control unit 37. The data processing unit 35 may store the biological information, etc., in the storage unit 41.

[0075] The data processing unit 35 acquires a pulse wave signal using one or any number of time series data Dn(t) of the nth facial feature point, from among the N time series data Dn(t) of the nth facial feature point, where n is from 1 to N.

[0076] Figure 11 shows an example of the control flow performed by the measuring device 10. Figure 11 shows a control flow using measurement point information. Figure 11 shows an example of the control flow performed in step S117 shown in Figure 7. Figure 11 shows a control flow in which one or more mesh points 131a are selected as measurement points based on the measurement point information, and time-series data of the measurement points are acquired. The measuring device 10 uses the time-series data of the selected mesh points 131a to acquire a pulse wave signal in step S119 shown in Figure 7.

[0077] In step S301, the measuring device 10 stores measurement point information. One of the mesh points 131a is pre-selected as a tracking measurement point. When multiple mesh points 131a are selected from all the mesh points 131a, each of the multiple mesh points 131a is selected as an analysis measurement point. One of the multiple analysis measurement points is a tracking measurement point. The tracking measurement points and analysis measurement points are pre-selected by the measurer M, the administrator of the bioanalysis program PG, etc. The measurement point information includes information relating to the tracking measurement point or the analysis measurement point. The information relating to the tracking measurement point is identification information that identifies the mesh point 131a that is the tracking measurement point. The information relating to the analysis measurement point is identification information that identifies the mesh point 131a that is the analysis measurement point. The identification information is, as an example, the code name shown in Figure 4. The measurement point information is stored in the storage unit 41 before the control flow shown in Figure 7 is executed.

[0078] In step S303, the measuring device 10 reads out the measurement point information. In step S115 of Figure 7, the data processing unit 35 reads out the measurement point information from the storage unit 41 after the acquisition of time-series data is completed.

[0079] After reading the measurement point information, the measuring device 10 acquires measurement point time-series data in step S305. The data processing unit 35 reads identification information corresponding to the tracking measurement point or the analysis measurement point using the measurement point information. The data processing unit 35 acquires time-series data of the mesh point 131a identified by the identification information. The data processing unit 35 acquires the time-series data of the mesh point 131a, which is a tracking measurement point, as tracking measurement point time-series data. The tracking measurement point time-series data includes a group of output values ​​for the tracking measurement point. The tracking measurement point time-series data is an example of measurement point time-series data and corresponds to an example of the light intensity detected at the tracking feature point. The data processing unit 35 acquires the time-series data of each mesh point 131a, which is an analysis measurement point, as analysis measurement point time-series data. The analysis measurement point time-series data includes a group of output values ​​for the analysis measurement point. The analysis measurement point time-series data is an example of measurement point time-series data and corresponds to an example of the light intensity detected at the measurement feature point.

[0080] The data processing unit 35 acquires the pulse wave signal in step S119 shown in Figure 7 using the time-series data of the tracked measurement points. The data processing unit 35 improves the measurement accuracy of the pulse wave signal by tracking predetermined mesh points 131a and acquiring the pulse wave signal using the output value group of the tracked mesh points 131a.

[0081] The data processing unit 35 may acquire the pulse wave signal in step S119 shown in Figure 7 using the time-series data of the analysis measurement points. The time-series data of the analysis measurement points includes the time-series data of the tracking measurement points. The data processing unit 35 acquires the output values ​​of the analysis measurement points included in the time-series data of the analysis measurement points. The data processing unit 35 calculates the average mesh point output value Bave using the mesh point output values ​​of the analysis measurement points for each image data. The average mesh point output value Bave is expressed by the following equation (8). Bave=(rave,gave,bave) (8) Here, rave is the average red gradation value calculated based on the output values ​​of the analysis measurement points, gave is the average green gradation value calculated based on the output values ​​of the analysis measurement points, and bave is the average blue gradation value calculated based on the output values ​​of the analysis measurement points.

[0082] The data processing unit 35 calculates the average mesh point output value time series data Bave(t) using the average mesh point output value Bave. The average mesh point output value time series data Bave(t) is an example of the time series data of the analysis measurement points. The average mesh point output value time series data Bave(t) is expressed by the following equation (9). Bave(t)=(rave(t),gave(t),bave(t)) (9)

[0083] The data processing unit 35 acquires the pulse wave signal in step S119 shown in Figure 7, using the average mesh point output value time series data Bave(t), which is the time series data of the analysis measurement points. The data processing unit 35 improves the measurement accuracy of the pulse wave signal by tracking multiple mesh points 131a and acquiring the pulse wave signal using the output value group of multiple mesh points 131a.

[0084] The measuring device 10 includes an imaging unit 11 that images the person being measured M and generates multiple image data, an image recognition processing unit 33 that identifies a face image included in the image data and identifies multiple mesh points 131a within the face image, and a data processing unit 35 that detects the pulse wave signal of the person being measured. The data processing unit 35 selects a tracking measurement point from the multiple mesh points 131a, tracks the tracking measurement point included in each of the multiple image data, and acquires a pulse wave signal based on the tracking measurement point time series data of each tracking measurement point in the multiple image data. The data processing unit 35 tracks predetermined mesh points 131a and acquires pulse wave signals using time-series data of tracked measurement points of the predetermined mesh points 131a, thereby improving the measurement accuracy of the pulse wave signals.

[0085] Preferably, the data processing unit 35 selects multiple analysis measurement points, including tracking measurement points, from among multiple mesh points 131a, tracks the analysis measurement points included in each of the multiple image data, calculates the analysis measurement point time series data for each of the multiple image data, and acquires a pulse wave signal using the multiple analysis measurement point time series data. The data processing unit 35 tracks multiple mesh points 131a and acquires a pulse wave signal using the time-series data of the analysis measurement points of the multiple mesh points 131a, thereby improving the accuracy of pulse wave signal measurement.

[0086] The bioanalysis program PG instructs the control unit 31, which is connected to the imaging unit 11 that images the subject M and generates multiple image data, to identify the face image included in the image data, to identify multiple mesh points 131a within the face image, to select the tracking measurement points included in the multiple mesh points 131a, to track the tracking measurement points included in each of the multiple image data, and to acquire the pulse wave signal based on the tracking measurement point time series data of each of the tracking measurement points in the multiple image data. The data processing unit 35 tracks predetermined mesh points 131a and acquires pulse wave signals using time-series data of tracked measurement points of the predetermined mesh points 131a, thereby improving the measurement accuracy of the pulse wave signals.

[0087] When the measuring device 10 acquires a pulse wave signal using measurement point information, it may acquire time-series data for all mesh points 131a, or it may not. The measuring device 10 may identify one or more mesh points 131a from which to acquire time-series data using measurement point information, and acquire time-series data for the identified mesh points 131a. The measuring device 10 acquires a pulse wave signal using the time-series data of the identified mesh points 131a.

[0088] Figure 12 shows an example of the control flow performed by the measuring device 10. Figure 12 shows the control flow for identifying measurement points using mesh point-related information MT. Figure 12 shows an example of the control flow performed in step S117 shown in Figure 7. Figure 12 identifies measurement points using mesh point coordinates included in the time-series data. The measuring device 10 uses the time-series data of the identified measurement points to acquire the pulse wave signal in step S119 shown in Figure 7.

[0089] In step S401, the measuring device 10 reads the mesh point-related information MT from the storage unit 41. The data processing unit 35 reads the mesh point-related information MT shown in Figure 6 as an example. The mesh point-related information MT includes information indicating mesh points 131a that are adjacent to each other and connected by mesh lines 131b.

[0090] After reading the mesh point related information MT, the measuring device 10 determines in step S403 whether the mesh point coordinates of two adjacent mesh points 131a have a predetermined positional relationship. The mesh point coordinates correspond to an example of coordinate information. Each mesh point 131a is set at a predetermined position within the face image region 121. When the face of the measurer M is facing the imaging unit 11, the mesh point coordinates of two adjacent mesh points 131a in the image data have a predetermined positional relationship. If the face of the measurer M changes from facing the imaging unit 11 to a different orientation, the mesh point coordinates of two adjacent mesh points 131a in the image data may not have the predetermined positional relationship.

[0091] The mesh point coordinates of any mesh point 131a are, for example, (x1, y1), and when the mesh point coordinates of the adjacent mesh point 131a adjacent to the right of the arbitrary mesh point 131a in the image data are set to (x2, y2), x1 and x2 have the relationship of x1 < x2. Here, the origin of the mesh point coordinates is set at the lower left of the image data. When the face of the measurer M faces left, x1 and x2 may have the relationship of x1 ≥ x2.

[0092] The data processing unit 35 selects one mesh point 131a among the plurality of mesh points 131a. The data processing unit 35 identifies the adjacent mesh point 131a adjacent to one mesh point 131a using the mesh point related information MT. The data processing unit 35 compares the mesh point coordinates of one mesh point 131a with the mesh point coordinates of the adjacent mesh point 131a for each image data. The data processing unit 35 determines whether one mesh point 131a and the adjacent mesh point 131a are in a predetermined positional relationship. The data processing unit 35 sequentially selects all the mesh points 131a as one mesh point 131a. The data processing unit 35 compares the mesh point coordinates of the selected mesh point 131a with the mesh point coordinates of the adjacent mesh point 131a identified using the mesh point related information MT for each mesh point 131a. The data processing unit 35 determines whether the selected mesh point 131a and the adjacent mesh point 131a are in a predetermined positional relationship. When the data processing unit 35 determines that one mesh point 131a and the adjacent mesh point 131a are in a predetermined positional relationship, the measuring device 10 proceeds to step S405 (step S403: YES). When the data processing unit 35 determines that one mesh point 131a and the adjacent mesh point 131a are not in a predetermined positional relationship, the measuring device 10 proceeds to step S407 (step S403: NO).

[0093] In step S405, the measuring device 10 sets one mesh point 131a as a measurement point. The set measurement point is either a tracking measurement point or an analysis measurement point. The data processing unit 35 selects a measurement point from among multiple mesh points 131a by setting one mesh point 131a as a measurement point. The data processing unit 35 acquires time-series data of the mesh point 131a set as a measurement point. Using the acquired time-series data, the data processing unit 35 acquires a pulse wave signal in step S119 shown in Figure 7.

[0094] In step S407, the measuring device 10 excludes one mesh point 131a from the measurement points. The data processing unit 35 does not use the time-series data of the mesh point 131a that was excluded from the measurement points to acquire the pulse wave signal.

[0095] The data processing unit 35 preferably acquires mesh point-related information MT relating to each of the multiple mesh points 131a and the adjacent mesh points 131a. The data processing unit 35 can identify adjacent mesh points 131a by acquiring mesh point-related information MT.

[0096] The data processing unit 35 preferably selects measurement points using mesh point-related information MT. The data processing unit 35 can easily select measurement points by using mesh point-related information MT.

[0097] The data processing unit 35 preferably identifies the coordinates of each of the multiple mesh points 131a and selects the analysis measurement points based on the mesh point coordinates. The measuring device 10 can exclude mesh points 131a that do not have a predetermined positional relationship depending on the orientation of the measurer M's face from the measurement points. The measuring device 10 can improve the detection accuracy of the pulse wave signal.

[0098] Figure 13 shows an example of the control flow performed by the measuring device 10. Figure 13 shows a control flow for identifying measurement points using time-series data. Figure 13 shows an example of the control flow performed in step S117 shown in Figure 7. Figure 13 identifies measurement points using mesh point output values ​​included in the time-series data. The measuring device 10 uses the time-series data of the identified measurement points to acquire a pulse wave signal in step S119 shown in Figure 7.

[0099] In step S501, the measuring device 10 reads out the light intensity threshold. The light intensity threshold is data used for comparison with the mesh point output value. As an example, the light intensity threshold represents the lower limit of the mesh point output value. The light intensity threshold is pre-set and stored in the storage unit 41. The data processing unit 35 reads out the light intensity threshold from the storage unit 41.

[0100] After reading the light intensity threshold, the measuring device 10 acquires the mesh point output value in step S503. The mesh point output value corresponds to an example of the light intensity for feature point detection. The data processing unit 35 acquires the mesh point output value included in the time series data. The data processing unit 35 acquires the mesh point output value for each mesh point 131a for each image data.

[0101] After acquiring the mesh point output value, the measuring device 10 determines in step S505 whether the mesh point output value is greater than the light intensity threshold. If the mesh point output value is less than the light intensity threshold, the detection accuracy of the pulse wave signal acquired using the mesh point output value decreases. The data processing unit 35 acquires time-series data that includes mesh point output values ​​greater than the light intensity threshold.

[0102] The data processing unit 35 selects one mesh point 131a from among multiple mesh points 131a. The data processing unit 35 compares the mesh point output value included in the time series data of the selected mesh point 131a with the light intensity threshold. The data processing unit 35 sequentially selects all mesh points 131a. For each mesh point 131a, the data processing unit 35 compares the mesh point output value included in the time series data of the selected mesh point 131a with the light intensity threshold. If the data processing unit 35 determines that the mesh point output value is greater than the light intensity threshold, the measuring device 10 proceeds to step S507 (step S505: YES). If the data processing unit 35 determines that the mesh point output value is less than the light intensity threshold, the measuring device 10 proceeds to step S509 (step S505: NO).

[0103] In step S507, the measuring device 10 sets a mesh point 131a whose mesh point output value included in the time-series data is greater than the light intensity threshold as a measurement point. The measurement point is either a tracking measurement point or an analysis measurement point. The data processing unit 35 selects one or more measurement points from among multiple mesh points 131a by setting a mesh point 131a whose mesh point output value included in the time-series data is greater than the light intensity threshold as a measurement point. The data processing unit 35 acquires the time-series data of the mesh point 131a set as a measurement point. Using the acquired time-series data, the data processing unit 35 acquires a pulse wave signal in step S119 shown in Figure 7.

[0104] In step S509, the measuring device 10 excludes mesh points 131a from the measurement points if the mesh point output value included in the time series data is smaller than the light intensity threshold. The data processing unit 35 does not use the time series data of the mesh points 131a that have been excluded from the measurement points to acquire the pulse wave signal.

[0105] The data processing unit 35 preferably acquires the output value of each of the multiple mesh points 131a and uses the multiple mesh point output values ​​to select the analysis measurement points. The measurement device 10 improves the detection accuracy of the pulse wave signal by selecting analysis measurement points using the mesh point output values.

[0106] Figure 13 shows, but is not limited to, the selection of measurement points by comparing the mesh point output value with the light intensity threshold. The light intensity threshold may also be an upper limit. The light intensity threshold may also be a value that sets a light intensity range that includes both an upper and lower limit. The data processing unit 35 may also select measurement points by comparing each mesh point output value with a statistical value. The statistical value is an example of a light intensity threshold. The statistical value may also be a value set based on the standard deviation, variance, median, quartiles, etc., of the mesh point output values ​​included in one time series data. The statistical value may also be a value set based on the standard deviation, variance, median, quartiles, etc., of the mesh point output values ​​included in the selected time series data or all time series data.

[0107] Figure 14 shows an example of the control flow performed by the measuring device 10. Figure 14 shows the control flow for identifying measurement points using mesh point-related information MT and time-series data. Figure 14 shows an example of the control flow performed in step S117 shown in Figure 7. Figure 14 identifies measurement points using mesh point-related information MT and mesh point output values ​​included in the time-series data. The measuring device 10 uses the time-series data of the identified measurement points to acquire pulse wave signals in step S119 shown in Figure 7.

[0108] In step S601, the measuring device 10 reads the mesh point-related information MT from the storage unit 41. The data processing unit 35 reads the mesh point-related information MT shown in Figure 6 as an example.

[0109] After reading the mesh point-related information MT, the measuring device 10 acquires the mesh point output value in step S603. The data processing unit 35 acquires the mesh point output value included in the time-series data. The data processing unit 35 acquires the mesh point output value for each mesh point 131a for each image data.

[0110] After acquiring the mesh point output value, the measuring device 10 determines in step S605 whether the mesh point output value of the adjacent mesh point 131a is within a predetermined range of light intensity difference. The data processing unit 35 selects one mesh point 131a from a plurality of mesh points 131a. The data processing unit 35 acquires the mesh point output value of the selected mesh point 131a. The data processing unit 35 identifies the adjacent mesh point 131a to the selected mesh point 131a using mesh point related information MT. The data processing unit 35 acquires the mesh point output value of the adjacent mesh point 131a. The data processing unit 35 calculates the difference value between the mesh point output value of the selected mesh point 131a and the mesh point output value of the adjacent mesh point 131a. The data processing unit 35 determines whether the difference value is within a predetermined range of light intensity difference. If the difference value is greater than the predetermined light intensity difference, the data processing unit 35 determines that the mesh point output value of the selected mesh point 131a is an abnormal value. The data processing unit 35 can improve the detection accuracy of the pulse wave signal by excluding mesh point output values ​​that may be abnormal.

[0111] If the data processing unit 35 determines that the difference value is within a predetermined range of light intensity difference, the measuring device 10 proceeds to step S607 (step S605: YES). If the data processing unit 35 determines that the difference value is not within a predetermined range of light intensity difference, the measuring device 10 proceeds to step S609 (step S605: NO).

[0112] In step S607, the measuring device 10 sets a mesh point 131a whose difference value is within a predetermined range of light intensity difference as a measurement point. The measurement point is either a tracking measurement point or an analysis measurement point. The data processing unit 35 selects one or more measurement points from among multiple mesh points 131a by setting a mesh point 131a whose difference value is within a predetermined range of light intensity difference as a measurement point. The data processing unit 35 acquires time-series data of the mesh points 131a set as measurement points. Using the acquired time-series data, the data processing unit 35 acquires a pulse wave signal in step S119 shown in Figure 7.

[0113] In step S609, the measuring device 10 excludes mesh points 131a whose difference value is not within a predetermined range of light intensity difference from the measurement points. The data processing unit 35 does not use the time-series data of the mesh points 131a that have been excluded from the measurement points to acquire the pulse wave signal.

[0114] Figure 14 shows that measurement points are selected using difference values, but is not limited to this. As an example, the data processing unit 35 calculates the distance between adjacent mesh points 131a using mesh point coordinates. The data processing unit 35 calculates the amount of light intensity displacement using the distance between two adjacent mesh points 131a and the mesh point output value. The data processing unit 35 may also select measurement points using the amount of light intensity displacement and a predetermined light intensity displacement threshold. [Explanation of Symbols]

[0115] 10... Measuring device, 11... Imaging unit, 13... Display unit, 15... Input unit, 31... Control unit, 33... Image recognition processing unit, 35... Data processing unit, 37... Display control unit, 41... Storage unit, 100... Acquired image, 121... Face image area, 131... Face mesh information, 131a... Mesh point, 131b... Mesh line, Db... Blue tone detection value, Dg... Green tone detection value, Dr... Red tone detection value, M... Measurer, MT... Mesh point related information, PG... Biological analysis program, S... Noise reduction signal, S1... Body movement section, S2... Resting section, TBL-01... First code name, TBL-02... Second code name, TBL-03... Third code name, TBL-04... Fourth code name.

Claims

1. An imaging unit that captures images of a living organism and generates multiple frame image data, A face recognition unit identifies a face image contained in the frame image data and identifies multiple feature points within the face image, It comprises a detection unit for detecting the pulse wave signal of the living organism, The detection unit is Select a tracking feature point from among the multiple feature points, and track the tracking feature point contained in each of the multiple frame image data. The pulse wave signal is acquired based on the amount of light detected at each of the tracking feature points in the multiple frame image data. A device for acquiring biological information.

2. The detection unit is Select a plurality of measurement feature points, including the tracking feature point, from among the plurality of feature points, and track the measurement feature points included in each of the plurality of frame image data. The measurement feature point detection light intensity of each of the measurement feature points in the multiple frame image data is calculated, and the pulse wave signal is acquired using the multiple measurement feature point detection light intensity amounts. A biological information acquisition device according to claim 1.

3. The detection unit acquires connection information relating to each of the feature points adjacent to each of the plurality of feature points. The biological information acquisition device according to claim 2.

4. The detection unit is The amount of light detected at each of the multiple feature points is obtained. Using the light intensity of multiple feature points, the measurement feature point is selected from among the multiple feature points. The biological information acquisition device according to claim 2.

5. The detection unit is Identify the coordinate information of each of the multiple feature points, Based on the coordinate information, select the measurement feature point. The biological information acquisition device according to claim 2.

6. The detection unit selects the measurement feature point using the linked information. A biological information acquisition device according to claim 3.

7. A computer connected to an imaging unit that captures images of living organisms and generates multiple frame image data, To identify the face image contained in the frame image data, To identify multiple feature points in the aforementioned facial image, Select a tracking feature point that is included in the multiple feature points, The tracking feature points included in each of the multiple frame image data are tracked. A pulse wave signal is acquired based on the amount of light detected at each of the tracking feature points in the multiple frame image data. A program for acquiring biometric information.

Images