Biological information processing program, biological information processing method, and biological information processing device

Abstract

A radar (R) outputs a plurality of detection signals obtained by emitting radar waves in a room (E), which is a measurement region. In a biological information processing device (100), an acquisition unit (101) acquires the detection signals to identify an area in which a human U as the subject is present. The acquisition unit (101) selects and acquires, within the identified area, a plurality of detection signals at a plurality of positions in which body movement and vital information of the human U have values indicating activity. A signal correction unit (102a) corrects the signal intensity of the detection signals on the basis of the signals in the most recent time range. A feature amount extraction unit (102) extracts feature amounts of the body movement and the vital information from the plurality of detection signals after correction. A determination unit (103) determines a biological state of the human U, for example, a respiratory state, on the basis of the extracted feature amounts. For example, when the respiratory state continuously decreases over time, the determination unit (103) determines that the human U is in a dangerous state and outputs an alarm.

Description

Biological information processing program, biological information processing method, and biological information processing device

[0001] The present invention relates to a biometric information processing program, a biometric information processing method, and a biometric information processing device.

[0002] By monitoring the position, posture, and vital signs of a human subject over time, it is possible to quickly detect abnormalities in the subject's biological information and respond promptly. For example, in medical and nursing care facilities and police station detention cells, there is a need to remotely monitor the behavior and health of people in private rooms.

[0003] For example, installing surveillance cameras to monitor people can provide a variety of information, but they may not function properly in dark places such as at night, and this can lead to problems with privacy violations. On the other hand, contact-based measurement using wearable devices can avoid privacy violations, but they require people to wear and manage the devices themselves, making them difficult to operate and preventing widespread adoption.

[0004] Furthermore, monitoring using radar technology such as millimeter-wave sensors is attracting attention because it allows for contactless measurement while protecting privacy. Millimeter-wave sensors can generally grasp human movement and behavior, such as direction and speed, based on radar transmission and reception signals, and obtain vital information such as respiratory rate and heart rate by analyzing reflected signals from stationary humans. Therefore, it is expected that radar technology can be used to monitor signal waveforms and heart and respiratory rates, thereby quickly detecting dangerous conditions such as respiratory arrest.

[0005] Conventional techniques include, for example, a technique for detecting biological information using radar, selecting range pins in a spectrum with large phase changes, and performing FFT (Fast Fourier Transformation) to determine whether a person has stopped breathing or not. Another technique involves characterizing a subject's entry into a room and range pins based on fluctuations in the signal strength of reflected radio waves, generating motion data tracking the subject's position, and determining dangerous conditions such as breathing based on temporal fluctuations in signal strength. Another technique involves removing unnecessary frequencies from received radar data to eliminate noise using the clutter effect, detecting one or more targets in specific range bins in the range data, and extracting the target's breathing rate based on the target's breathing signal. Another technique involves acquiring point cloud data of a person from a radar device and selecting the person's correct breathing rate by referring to the likelihood of the person's different positions (see, for example, Patent Documents 1 to 4 listed below).

[0006] JP 2023-123929 A JP 2020-81312 A U.S. Patent Application Publication No. 2015 / 0369911 JP 2024-40042 A

[0007] However, in conventional technology, the state of breathing, etc. is determined using the amount of displacement in the signal waveform of a specific point on the subject. In this case, for example, it is not possible to respond to changes in the amount of displacement or distribution of the intensity or phase of the detected signal due to changes in the subject's posture or position, etc., and a decrease in signal value due to changes in the subject's posture or position may result in an erroneous determination of respiratory cessation. In conventional technology, it was not possible to accurately determine the subject's biological information, such as respiratory cessation.

[0008] In one aspect, the present invention aims to improve the accuracy of detecting biological information of a subject.

[0009] According to one embodiment, a biometric information processing program, a biometric information processing method, and a biometric information processing device are proposed that include processing to identify an area where a subject is present based on a plurality of detection signals within a measurement area, select and acquire a plurality of detection signals from the identified area at a plurality of positions where the subject's body movement / vital information has values ​​indicating activity, extract feature values ​​of the body movement / vital information from the acquired plurality of detection signals, and determine the subject's biological condition based on the extracted feature values.

[0010] According to one aspect, it is possible to improve the accuracy of detecting biological information of a subject.

[0011] FIG. 1 is a diagram illustrating an example of a biometric information processing device according to an embodiment. FIG. 2 is a block diagram illustrating an example of the hardware configuration of the biometric information processing device. FIG. 3 is a diagram illustrating different cases of determining a dangerous state by measuring vital signs using radar. FIG. 4A is a diagram illustrating human detection and signal acquisition processing by an acquisition unit. FIG. 4B is a diagram illustrating signal correction and feature extraction processing by a feature extraction unit. FIG. 4C is a diagram illustrating determination processing by a determination unit. FIG. 5 is a diagram illustrating a determination state in response to a human state and a change in a detection signal. FIG. 6 is a diagram illustrating an example of human detection by radar. FIG. 7 is a diagram illustrating an example of signal processing of a detection signal by an acquisition unit. FIG. 8 is a diagram illustrating an example of signal correction processing performed by a signal correction unit. FIG. 9A is a diagram illustrating an example of feature extraction processing by a feature extraction unit. (Part 1) FIG. 9B is a diagram illustrating an example of feature extraction processing by the feature extraction unit. (Part 2) FIG. 10A is a diagram illustrating an example of determination processing by a determination unit. (Part 1) FIG. 10B is a diagram illustrating an example of determination processing by a determination unit. (Part 2) Fig. 11 is an explanatory diagram of human detection and position update, and Fig. 12 is a flowchart showing an example of a processing procedure of the biometric information processing device.

[0012] Hereinafter, embodiments of biometric information processing according to the present invention will be described in detail with reference to the drawings.

[0013] 1 is a diagram illustrating an example of a biological information processing device according to an embodiment. The biological information processing device 100 according to the embodiment is a computer that acquires a detection signal including biological information of a subject within a measurement region of a predetermined range, and determines a biological state of the subject, such as a state of breathing, from the detection signal.

[0014] The biometric information processing device 100 acquires detection signals from multiple positions within a measurement area using, for example, a non-contact millimeter-wave radar, and selects multiple detection signals within a range where the subject is present from the multiple detection signals acquired at predetermined time intervals, thereby responding to changes in the subject's position over time. The biometric information indicates the subject's body movements and various biological conditions, such as vital signs (e.g., pulse rate, heart rate, respiratory rate, body temperature, blood pressure, oxygen saturation, etc.). In the following embodiment, an example will be described in which the biometric information processing device 100 determines and notifies a dangerous state of respiratory arrest as the biological condition of the subject (human) based on the detection signals.

[0015] An overview of the functions and processing of the biometric information processing device 100 will be described using Figure 1. The biometric information processing device 100 includes an acquisition unit 101, a feature extraction unit 102, a determination unit 103, and a recording unit 105. The acquisition unit 101 acquires a detection signal from a radar R installed in a measurement area for detecting a subject (human) U, for example, in a room E. The radar R emits radar waves to a plurality of measurement points 1 to n (n = 15 in Figure 1) in a matrix pattern when viewed from a plane within the room E, and detects the human U in the room E in a non-contact manner based on the reflected waves from each measurement point. Each function of the biometric information processing device 100 performs the following processing.

[0016] (1) Acquisition Unit 101 The acquisition unit 101 acquires a detection signal of a person present in a measurement area. The biometric information processing device 100, for example, uses radar R to irradiate a plurality of measurement points 1 to n in a room E where a person is present with radar such as millimeter waves, and outputs a detection signal for each measurement point 1 to n based on the reflection of the radar. The acquisition unit 101 records the detection signal in the recording unit 105 (signal waveform (recording unit) 105a).

[0017] 1, the acquisition unit 101 extracts, for example, multiple detection signals at positions where a person U is present from the detection signals at multiple measurement points 1 to 15 at predetermined time intervals. The person U moves and changes posture over time. For example, at a certain time, the person U shown by the solid line is detected at multiple measurement points 2, 6 to 8, and 11 to 13, which are indicated by black circles. After this, at a point in time when time has passed, the person U shown by the line is detected to be located at different measurement points 10, 14 to 15.

[0018] The acquisition unit 101 selects, at predetermined time intervals, detection signals from the plurality of measurement points 1 to 15 that have signal waveforms containing body movement or breathing information (predetermined values ​​of predetermined indices) near the detection positions, thereby causing the measurement points to be acquired to track the movement of the person U. For example, among the n measurement points, the acquisition unit 101 selects and acquires multiple detection signals from multiple measurement points where the body movement and vital information of the person U has values ​​indicating activity. In the example of FIG. 1 , the "values ​​of body movement and vital information indicating activity" refer to values ​​at multiple measurement points 2, 6 to 8, and 11 to 13 indicated by black circles ● at a certain time that have valid information (e.g., greater than a predetermined value). Alternatively, the values ​​of multiple measurement points 2, 6 to 8, and 11 to 13 may be values ​​that are relatively larger (e.g., by a predetermined value or more) than the values ​​of the other measurement points 1, 3 to 5, 9, 10, 14, and 15. The acquisition unit 101 determines that the body movement and vital information of the detection signals from these multiple measurement points 2, 6 to 8, and 11 to 13 indicates activity. As a result, the acquisition unit 101 of the embodiment tracks and acquires detection signals having valid information corresponding to the presence of the human being U within the measurement area.

[0019] For example, the acquisition unit 101 acquires a detection signal including position information when a stationary human U is detected. The predetermined time is, for example, a time for determining hypopnea. In this case, the acquisition unit 101 selectively extracts multiple detection signals having a lot of information (e.g., having signal waveforms of amplitude and phase) from multiple measurement points near the position of the human U for each time for determining hypopnea, and performs signal processing to extract only the fluctuation components of each detection signal.

[0020] Determining the presence and moving / stationary state of a human U, and acquiring location information when a stationary human U is detected, are commonly performed using millimeter-wave radar R. For example, a biosignal detection technology has been proposed that simultaneously detects multiple people within a predetermined range and spatially separates the signals using a fast Fourier transform (FFT) to reduce noise signal interference and improve detection accuracy (see, for example, JP 2023-26124 A). Also proposed is a technology that removes invalid data based on the size of the radar's detection area and the signal's wave motion and strength, removes the average value of the signal's wave motion, eliminates offsets, and improves the accuracy of detecting stationary organisms (see, for example, JP 2023-80015 A). Furthermore, a technology has been proposed that determines the presence or absence of a living organism at each distance based on the amplitude and phase spectrum obtained by Fourier transforming the amplitude / phase distribution of the radar signal at each distance over a predetermined time, eliminates other stationary objects and noise, and accurately detects the location of the living organism (see, for example, JP 2021-30065 A). The acquisition unit 101 of the embodiment includes these existing technologies and uses radar to determine the presence of a human U and whether the human U is moving or stationary, and also acquires location information when a stationary human U is detected.

[0021] Here, the acquisition unit 101 of the embodiment uses signal waveforms of detection signals at multiple positions selected based on a predetermined index at each predetermined time as a new configuration not disclosed in the past. This makes it possible to track changes in the actual measurement position due to changes in the posture and body movement of the person U between different times, and to continuously detect the person U in the room E over time in a non-contact manner.

[0022] (2) Feature Extraction Unit 102 The feature extraction unit 102 includes a signal correction unit 102a. For example, the signal correction unit 102a reads the most recently acquired detection signal from the recording unit 105 (signal waveform 105a) and corrects the signal strength by dividing it by the average value of the detection signals (detection signals in a resting state) acquired up to a certain time before. The corrected detection signal is then stored again in the recording unit 105 (signal waveform 105a). This enables evaluation independent of measurement conditions. For example, the signal strength of a person U standing still at a distance from the radar R differs from that of a person U standing still at a distance from the radar R. However, by correcting the most recent detection signal with the detection signal in a resting state, the signal value is corrected to a similar value. This makes it possible to focus only on the relative change in the signal value.

[0023] The feature extraction unit 102 reads out the detection signal after signal correction by the signal correction unit 102a from the recording unit 105 (signal waveform 105a), performs signal processing, and extracts multiple feature amounts related to the biological information of the human U, such as the variance of amplitude and phase fluctuations, maximum and minimum values, and respiratory energy ratio. These feature amounts also include general evaluation indexes that are used when measuring the respiratory rate and heart rate, which are the biological state of the human U indicated by the biological information. The extracted feature amounts are recorded and stored in the recording unit 105.

[0024] (3) Determination Unit 103 The determination unit 103 reads the feature amounts extracted by the feature amount extraction unit 102 from the recording unit 105 and determines the biological state of the person, for example, the breathing or body movement state, by threshold determination or machine learning. For example, the determination unit 103 has a function of calculating a respiration score from the feature amounts and a function of detecting a hypopnea (decreased respiratory rate) state of the person based on the calculated score. For example, the calculated score is recorded and stored in the recording unit 105 (respiration score (recording unit) 105b). The determination unit 103 makes a determination using the signal value of the detection signal corrected as described above, thereby enabling the feature amounts and the hypopnea threshold to be set regardless of conditions. In this embodiment, the determination unit 103 determines a dangerous state of the person based on a state before respiratory arrest.

[0025] The determination unit 103 determines that a chronologically continuous occurrence of hypopnea is a dangerous state and issues an alarm. In the embodiment, the determination unit 103 performs multi-stage determination, for example, issuing an alarm when the time during which a score calculated based on multiple indices continues to exceed a threshold (the number of consecutive detections) exceeds another threshold, thereby preventing false alarms.

[0026] 1, the recording unit 105 is provided with a recording unit for a signal waveform 105a and a respiration score 105b. However, the recording unit 105 may be configured to record signals in each of the functional units, namely, the acquisition unit 101, the feature extraction unit 102, and the determination unit 103.

[0027] The biometric information processing device 100 having the above functions further periodically detects and updates the position of the human U, thereby ensuring that the detection signal is a signal originating from the human U. For example, if the human U is moving / moving significantly, or if the signal is determined to be from a source other than the human U when the human U is detected, the above processing is reset.

[0028] The biometric information processing device 100 is, for example, various information devices such as a PC (Personal Computer) or a server. Alternatively, the biometric information processing device 100 may be a mobile terminal such as a smartphone owned by the person U. The method for processing biometric information according to the embodiment can be realized, for example, by an information device executing a program. Furthermore, each function of the biometric information processing device 100 can be configured in the cloud.

[0029] (Example of Hardware Configuration of Biometric Information Processing Apparatus) Next, an example of the hardware configuration of the biometric information processing apparatus 100 will be described with reference to FIG.

[0030] Fig. 2 is a block diagram showing an example of the hardware configuration of a biometric information processing device. In Fig. 2, the biometric information processing device 100 has a CPU (Central Processing Unit) 201, a memory 202, a network I / F (Interface) 203, a recording medium I / F 204, and a recording medium 205. In addition, each component is connected to each other by a bus 200.

[0031] Here, the CPU 201 is a control unit that controls the entire biometric information processing device 100. The memory 202 includes, for example, a read-only memory (ROM), a random access memory (RAM), and a flash ROM. Specifically, for example, the flash ROM or ROM stores various programs, and the RAM is used as a work area for the CPU 201. The programs stored in the memory 202 are loaded into the CPU 201, causing the CPU 201 to execute the coded processes.

[0032] The network I / F 203 is connected to the network NW via a communication line and is connected to other computers via the network NW. The network I / F 203 controls the internal interface with the network NW and controls the input and output of data from other computers. The network I / F 203 is, for example, a modem or a LAN adapter.

[0033] The recording medium I / F 204 controls reading and writing of data from and to the recording medium 205 under the control of the CPU 201. The recording medium I / F 204 is, for example, a disk drive, a solid state drive (SSD), or a universal serial bus (USB) port. The recording medium 205 is a non-volatile memory that stores data written under the control of the recording medium I / F 204. The recording medium 205 is, for example, a disk, a semiconductor memory, or a USB memory. The recording medium 205 may be detachable from the biometric information processing device 100.

[0034] In addition to the above-mentioned components, the biometric information processing apparatus 100 may also include, for example, a keyboard, a mouse, a display, a printer, a scanner, a microphone, a speaker, etc. The biometric information processing apparatus 100 may also include a plurality of recording medium I / Fs 204 and recording media 205. The biometric information processing apparatus 100 may also not include the recording medium I / Fs 204 and the recording media 205.

[0035] The functions of the acquisition unit 101 to the determination unit 103 shown in Fig. 1 can be realized by the CPU 201 executing a program stored in the memory 202 or the recording medium 205 shown in Fig. 2. Furthermore, each of the functions of the acquisition unit 101 to the determination unit 103 shown in Fig. 1 has a recording unit that records processed data, and the recording unit can realize its function using the memory 202 or the recording medium 205 shown in Fig. 2.

[0036] (Comparison between the prior art and the embodiment) Next, the determination of biometric information (vital measurement) using radar will be explained by comparing the prior art with the embodiment using various figures. As disclosed in the technology of the above-mentioned patent documents, when a person moves, signal processing is performed on transmitted and received signals to detect distance, angle, and speed and track the person's position. Here, with the disclosed prior art, accurate vital measurement is not possible due to the influence of the person's movement, so only moving object detection is performed.

[0037] Furthermore, when a person is not moving, weak amplitude and phase signals are acquired through reflected signal analysis (e.g., Doppler analysis), and heart rate and respiratory rate are measured from the signal displacement. For example, vital signs are measured when a person is detected to be stationary. The measured heart rate and respiratory rate can then be used to determine the person's health condition and emotions. Radar-based vital sign measurement can detect dangerous conditions without contact with the person by monitoring the signal waveform and heart rate / respiratory rate.

[0038] 3A and 3B are explanatory diagrams for different cases of determining a dangerous state by measuring vital signs using radar. In the NG example of Fig. 3A, if the value (level) of the detection signal indicating the vital state of the radar is weak, the subject (human) is in a dangerous state and an alert is issued. However, even if the human is asleep and resting, the detection signal is weak, so it cannot be distinguished as a dangerous state and an alert is issued (false alert), which is a problem.

[0039] As shown in Figure 3(b), in the expected OK example, if the detection signal indicating the vital status of the radar is weak, the subject (human) is in danger and an alert is issued. If the human is asleep and resting, it is necessary not to issue an alert even if the detection signal is weak (preventing false alerts). To achieve this, it is necessary to accurately determine whether the human is in danger and issue an alert without issuing a false alert.

[0040] The key to detecting dangerous conditions is accurately distinguishing between "dangerous conditions" that should trigger an alert and "conditions where the subject is at rest but the signal is weak or unmeasurable due to posture or detection position," which should not trigger an alert. Since dangerous conditions are generally considered to occur only a few times a year, even a single false positive detection per day would make practical use difficult. While some of the above-mentioned prior art patent documents detect respiratory arrest, they make their determination solely based on a relative decrease or loss of signal value. As a result, signal changes due to posture changes, slight movements, or shifts in detection position are all deemed dangerous conditions, potentially resulting in frequent false positives. While it is possible to suppress false positives by setting strict thresholds, this also increases the likelihood of missed detections. Furthermore, when signals are extremely weak, human detection itself becomes difficult, making it difficult to ensure that the signal is truly human-derived.

[0041] In summary, conventional technologies have the following problems 1 to 3, for example: 1. In direct methods that determine a danger state when the amount of displacement of a signal at a specific point decreases, a decrease in the amount of displacement due to body movement or a change in posture is also considered a danger state, resulting in frequent erroneous determinations (since the likelihood of a danger state occurring in actual operation is low, it is essential to reduce erroneous determinations as much as possible). 2. It is difficult to distinguish whether an extremely weak signal, such as when breathing has stopped, is a human signal or a signal from another object, or a simple detection error. 3. It is not possible to accurately detect a "state in which human breathing is decreasing" (between normal and respiratory arrest).

[0042] In response to the above-mentioned problems, an embodiment of the present invention accurately determines whether a person's breathing is slowing down based on radar detection signals measured from a long distance without contact, thereby quickly and accurately determining whether a dangerous situation exists.

[0043] Hereinafter, each function of the biometric information processing apparatus 100 according to the embodiment described with reference to FIG. 1 will be described in comparison with the prior art.

[0044] 4A is an explanatory diagram of the human detection and signal acquisition process performed by the acquisition unit 101. The acquisition unit 101 performs the following processes.

[0045] 1. Determine the presence of human U and whether it is moving or stationary. This is commonly done with millimeter wave radar or the like. 2. Obtain location information when a stationary human U is detected. This is also commonly done with radar. 3. Obtain amplitude and phase signal waveforms from measurement points near the position of human U at predetermined time intervals, for example, at regular time intervals. 4. Selectively extract multiple amplitude and phase signal waveforms that contain a lot of information based on the evaluation index. 5. Perform signal processing to extract only the fluctuation components of each signal. For example, remove trends using a Savitzky-Golay filter or the like.

[0046] Conventional technology only estimates the position, respiratory rate, and heart rate of human U, but does not detect respiratory cessation based on signal values ​​or vital signs. Figure 4(a) shows changes in a person's position over time. Conventional technology acquires and tracks signal values ​​at specific fixed points indicated by black circles, and determines respiratory cessation based on a relative decrease or disappearance of the signal value. Such conventional technology determines respiratory cessation when the posture or state of human U at time t0 changes at time t1, weakening the signal from a specific position, or when a position where human U is not present is mistakenly detected as human U, resulting in frequent false alarms.

[0047] In contrast, in the embodiment, the acquisition unit 101 acquires signals by selecting a plurality of positions of the human U from among measurement points 1 to n (see FIG. 1) for each time t0, t1, .... The acquisition unit 101 selects, for each time, detection signals from the plurality of measurement points indicated by white circles, detection signals of signal waveforms from the plurality of measurement points that are effective information (predetermined values ​​for predetermined indicators) as body movement or breathing information near the detection positions, for example, body movement / vital information values ​​that indicate activity. This allows the acquired measurement points to follow the movement of the human U. In other words, it selects detection signals from the plurality of measurement points located at the position of the moving human U.

[0048] 4A(b) is a diagram showing the level of the detection signal over time. The horizontal axis represents time, and the vertical axis represents the level of the detection signal. In the conventional technology, the signal value at a specific point (position A) is acquired. Therefore, for example, as shown in FIG. 4A(a), if the posture or state of person U changes and they deviate from the specific point (position A), the signal at the specific point (position A) becomes weak, leading to a false alarm of respiratory arrest.

[0049] In contrast, in the embodiment, signals having valid information are tracked and acquired for each time. In the example of Fig. 4A(b), at times A to C, among the plurality of measurement points, detection signals of signal waveforms at a plurality of measurement points where the body movement or respiratory information has a predetermined value indicating activity are selected for each time. For convenience, Fig. 4A(b) shows only one signal waveform at one measurement point, but in reality, among the plurality of detection signals at a plurality of n measurement points at each of times A to C, detection signals of signal waveforms at a plurality of measurement points n1 where the body movement or respiratory information has a predetermined value indicating activity are selected for each time.

[0050] The acquisition unit 101 selectively extracts a plurality of detection signals having a lot of information (for example, having amplitude and phase signal waveforms) from among a plurality of measurement points near the position of the human U for each hypopnea determination time, and performs signal processing to extract only the fluctuation components of each detection signal. The extraction is performed by removing trends using, for example, a Savitzky-Golay filter. Note that in response to changes in the position and posture of the human U, some or all of the measurement points (detection signals) selected at times A to C may differ for times A, B, and C.

[0051] As described above, in the embodiment, unlike the conventional technology that acquires detection signals at specific positions where the positions are fixed, a plurality of detection signals having predetermined values ​​(signal waveforms) are selected and acquired for each time based on an index, thereby making it possible to follow changes in posture and body movements of the person U over time.

[0052] 4B is an explanatory diagram of the signal correction and feature extraction processing performed by the feature extraction unit 102. The feature extraction unit 102 performs the following processing.

[0053] 1. When the amplitude and phase fluctuation components of the detection signal are acquired at each time, the average value of the absolute values ​​of the amplitude / phase fluctuation components is saved. 2. The latest detection signal is corrected by dividing it by the average of the above average values ​​acquired up to a certain time before. 3. Multiple feature quantities such as variance, maximum / minimum, and respiratory energy ratio are calculated from the corrected signal.

[0054] First, the signal correction unit 102a of the feature extraction unit 102 corrects the signal strength by dividing the most recently acquired detection signal by the average value of detection signals acquired up to a certain time before (detection signals in a resting state), thereby enabling evaluation that is independent of measurement conditions.

[0055] Figure 4B(a) shows a radar waveform, with the horizontal axis representing time and the vertical axis representing the level for each condition. Before correction, the signal strength of the detection signal differs for each condition between a person U stationary at a distance from the radar R sensor and a person U stationary at a distance close to the radar R sensor. For this reason, the signal correction unit 102a corrects the most recent detection signal using, for example, a detection signal in a resting state (a past signal) as shown in Figure 4B(b). As a result, as shown after correction in Figure 4B(a), the signal level becomes approximately the same regardless of the condition (difference in distance from the sensor, etc.), making it possible to focus only on the relative change in the signal value.

[0056] The feature extraction unit 102 extracts multiple feature quantities, such as the variance of amplitude and phase fluctuations, maximum and minimum values, and respiratory energy ratio, from the detection signal corrected by the signal correction unit 102a. These feature quantities include general evaluation indices that are also used when measuring respiratory rate and heart rate. Furthermore, by correcting the detection signal as described above, it is possible to fix the feature quantities and thresholds for determining hypopnea, etc., regardless of conditions.

[0057] Figure 4B(c) shows the determination state using conventional technology. The horizontal axis represents time, and the vertical axis represents the detection signal and the determination state of respiratory arrest. In conventional technology, a simple determination is made based only on the magnitude of the phase fluctuation amount after frequency processing. Because only a binary determination is made based on a threshold value for a single index, fluctuations in the value of the detection signal frequently result in false detection (false determination) of respiratory arrest (0).

[0058] As described above, in the embodiment, signal correction of the detection signal enables determination and evaluation of the hypopnea state independent of the measurement conditions. Furthermore, signal correction also enables feature amounts and hypopnea thresholds to be set independent of the conditions. The feature amounts may be general evaluation indices that are also used when measuring respiratory rate and heart rate.

[0059] 4C is an explanatory diagram of the determination process performed by the determination unit 103. The determination unit 103 performs the following process for, for example, hypopnea / dangerous state.

[0060] 1. A respiration score is calculated from the feature values ​​using threshold judgment or machine learning, for example, a score between 0 and 1. The person's respiratory condition is then evaluated based on the score count, and if the score exceeds the threshold, it is determined to be in a state of hypopnea. 2. If hypopnea continues chronologically, it is determined to be a dangerous condition and an alert is issued.

[0061] In addition, the system periodically detects people and updates their location to ensure that the signal is human-originated. If a person moves or if the signal is determined to be non-human, the count is reset.

[0062] 4C(a) shows a respiration score based on the feature amounts. The horizontal axis represents time, and the vertical axis represents the respiration score. The determination unit 103 calculates the respiration score from the feature amounts output by the feature amount extraction unit 102. The determination unit 103 calculates the respiration score from the feature amounts by threshold determination or machine learning, and detects a state of hypopnea in a person based on the calculated score.

[0063] FIG. 4C(b) shows a determination state of a dangerous state based on the respiration score. The horizontal axis represents time, and the vertical axis represents the number of detected hypopneas in FIG. 4C(a). In the example of FIG. 4C(b), the determination unit 103 determines hypopnea at threshold 1 and determines a dangerous state at threshold 2. The determination unit 103 determines hypopnea and a dangerous state based on the number of consecutive hypopneas. For example, in the example of FIG. 4C(b), if the number of consecutive detected hypopneas (corresponding to the continuous time) is small and exceeds only threshold 1 (less than threshold 2), it is determined to be hypopnea (not a dangerous state). After this, when the number of consecutive hypopneas increases and exceeds both thresholds 1 and 2, it is determined to be a dangerous state.

[0064] As described above, in the embodiment, instead of a binary judgment based on a threshold for a single index as in the prior art, an alarm is issued when a score calculated based on multiple indexes exceeds a threshold for a certain period of time or more. This makes it possible to maintain a high rate of hypopnea detection while suppressing false positives through multi-stage judgment.

[0065] (Determination state based on change in detection signal) Figure 5 is a diagram showing the state of a person and the determination state in response to a change in the detection signal. The horizontal axis is time, and the vertical axis is signal (level). During a normal state (period A) in which the detection signal has a predetermined large value (level), both the prior art and the embodiment correctly determine a dangerous state as "no alarm is issued." The above-mentioned "value indicating activity in the subject's body movement and vital information" is a predetermined value as shown in periods A to B (B1, B3) in Figure 5, excluding period C.

[0066] Furthermore, when the value of the detection signal decreases and becomes difficult to measure due to a change in the posture or state of the person U (time B1), the person U remains in a normal state, but in the conventional technology, an "alert" of a dangerous state is mistakenly issued. In contrast, in the embodiment, as described above, another measurement point is selected in response to the change in posture, so it can be correctly determined that "no alert will be issued."

[0067] Furthermore, when a position where no human U is present is mistakenly detected as a human (time period B2), for example, when the signal becomes extremely small due to a detection error, the prior art would mistakenly issue an "alert" indicating a dangerous condition. In contrast, in the embodiment, as described above, the determination is not based solely on the signal level at a specific point, so it can correctly determine that "no alert will be issued."

[0068] Furthermore, when the value (level) of the detection signal changes downward due to decreased breathing of human U (time B3), both the prior art and the embodiment correctly judge the dangerous state and issue an "alert."

[0069] Furthermore, when the breathing of the person U stops and the value (level) of the detection signal becomes "no signal" (time C), neither the prior art nor the embodiment can detect only the person U. For example, it is difficult to distinguish the person U from other stationary objects.

[0070] 5, it can be seen that the embodiment can prevent erroneous alarms as in the prior art when the posture or state of human U changes (time period B1) and when a position where no human U is present is mistakenly detected as a human (time period B2). In the prior art, an alarm is issued during time period B, that is, when the posture or state of human U changes (time period B1), when a position where no human U is present is mistakenly detected as a human (time period B2), and when respiratory depression occurs (time period B3).

[0071] However, when human U stops breathing (time C), it is difficult to detect a human in both the conventional technology and the embodiment. For this reason, the embodiment makes it possible to correctly determine a "state in which a human's breathing is decreasing (time B3)" based on a detection signal. In the embodiment, the above-described configuration makes it possible to distinguish between each state that transitions to a dangerous state within time B between the normal state (time A) and respiratory arrest (time C) (no alert is issued at times B1 and B2, and an alert is issued at B3), thereby accurately determining a dangerous state at a more appropriate and accurate time.

[0072] (Example of Signal Processing by Each Functional Unit) Next, an example of signal processing by each functional unit of the bio-information processing apparatus 100 will be described.

[0073] Fig. 6 is a diagram showing an example of human detection by radar. Fig. 6 shows the states of radar R and human U in a plan view. Radar R is installed, for example, in the center of the ceiling of a room E or the like. This radar R detects a human U present in the room E in a non-contact manner. For example, radar R emits millimeter waves to a plurality of measurement points in a matrix shown by coordinates x and y within room E, and outputs a detection signal that is updated every 10 seconds with stationary position information of the reflected waves from the human U. The detection signal includes information on the number of counts for each position (measurement point) where the human U is stationary.

[0074] For convenience, the embodiment will describe an example in which one person U is detected in a room E. As shown in Fig. 6 , the count numbers corresponding to the positions of the people U are detected in clusters, so it is also possible to individually detect multiple people U in the room E. In this case, biometric information for the multiple people U can be acquired and the dangerous state can be individually determined.

[0075] FIG. 7 is a diagram illustrating an example of signal processing of detection signals by the acquisition unit. FIG. 7 shows detection signals selected by the acquisition unit 101. Regarding the detection information of multiple measurement points output by the radar R, the acquisition unit 101 selects detection signals from five measurement points with large signal deviations, for example, every five seconds, and extracts the signals. In the example of FIG. 7 , the acquisition unit 101 selects detection signals from five measurement points with large amplitude fluctuations and phase fluctuations, and the detection signals each have amplitude fluctuations and phase fluctuations with predetermined values ​​(levels) for a predetermined period of time. This allows the acquisition of detection signals from the position where the person U was present in the room E.

[0076] 8 is a diagram showing an example of signal correction processing performed by the signal correction unit 102a. The signal correction unit 102a performs correction to normalize the signal values ​​of the detection signals acquired at the five measurement points in FIG. 7 using data for a predetermined period, for example, the most recent 300 seconds.

[0077] The example in Fig. 8 shows that, out of 300 seconds, the average fluctuation of the detection signal value at time t = 5 (seconds) is "1.7" and the average fluctuation of the detection signal value at t = 10 is "1.4". The signal correction unit 102a then calculates correction data by dividing the detection signal value at time t = 300 by the average of the fluctuation averages of past data. This makes it possible to correct the intensity of the detection signal and extract multiple waveform feature amounts, and also makes it possible to fix thresholds for determining feature amounts and hypopnea, etc., regardless of conditions.

[0078] 9A and 9B are diagrams showing an example of feature extraction processing by the feature extraction unit. FIG. 9A shows a group of signals (amplitude fluctuation and phase fluctuation) corrected by the signal correction unit 102a. FIG. 9B shows feature amounts extracted by the feature extraction unit 102. The feature extraction unit 102 extracts feature amounts of signal amplitude and phase from the group of signals corrected by the signal correction unit 102a (FIG. 9A). As shown in FIG. 9B, for example, the feature extraction unit 102 extracts a phase fluctuation average, an amplitude fluctuation average, a phase energy ratio, and an amplitude energy ratio as multiple feature amounts.

[0079] 10A and 10B are diagrams showing an example of the determination process performed by the determination unit. Fig. 10A(a) shows scoring based on feature amounts performed by the determination unit 103, with the horizontal axis representing time and the vertical axis representing the score. In the case of determination using the threshold shown in Fig. 10A(a), the determination unit 103 compares the average fluctuations of the amplitude and phase signals indicated by the multiple feature amounts extracted by the feature amount extraction unit 102 (Fig. 9B) and four feature amounts of respiratory energy with thresholds to score the respiratory state. The score ranges from 0 to 1 in increments of 0.25, for example.

[0080] 10A(b) shows an example of a determination using machine learning. The determination unit 103 determines the possibility of a hypopnea state using a model such as a support vector machine (SVM) or a decision tree based on the waveforms of multiple feature amounts ( FIG. 9B ) extracted by the feature amount extraction unit 102, for example, 10 to 20 feature amounts, and scores the respiratory state. The score ranges, for example, from 0 to 1 point. The area with a high score surrounded by the dashed-dotted line in FIGS. 10A(a) and 10A(b) corresponds to the respiratory arrest area.

[0081] FIG. 10B shows a risk condition determination based on a score. The determination unit 103 determines hypopnea from the respiration score shown in FIG. 10A. In this determination, the risk condition is determined based on a state in which hypopnea continues. For example, if the respiration score shown in FIG. 10B(a) is greater than 0.5, the risk condition is determined to be a hypopnea state. Also, as shown in FIG. 10B(b), the vertical axis represents the number of hypopnea detections in the most recent 10 times. If the hypopnea state is detected six or more times in the most recent 10 times (50 seconds), the determination unit 103 determines the risk condition and outputs an alarm.

[0082] FIG. 11 is an explanatory diagram of human detection and position update. The biometric information processing device 100 periodically performs human detection and position update, such as at regular intervals. If a human U is detected at each measurement point during this update, the device continues processing related to the hypopnea determination. On the other hand, if a human U is not detected at each measurement point during the update, the device stops processing related to the hypopnea determination. For example, if a human U is not detected at all of the multiple measurement points, the device stops processing related to the hypopnea determination.

[0083] (Processing Example of Biometric Information Processing Device) Fig. 12 is a flowchart showing an example of a processing procedure of the biometric information processing device. The processing shown in Fig. 12 is executed by, for example, the control unit (CPU 201) shown in Fig. 2. The control unit acquires a signal sequence of a plurality of detection signals output by the radar R and executes detection of a human U (step S1201).

[0084] Then, the control unit determines whether the human U is stationary based on the detection signal (step S1202). If the human U is not stationary (step S1202: No), the control unit deletes the history of the signal and the breathing score recorded in the recording unit 105 (step S1203), and ends the above processing.

[0085] On the other hand, if the person U is stationary (step S1202: Yes), the control unit updates and acquires position information of the detection signal (step S1204). Next, the control unit searches for multiple feature points near the person U, acquires signal waveforms, and performs signal correction (step S1205). The control unit records the acquired detection signals in the recording unit 105 (signal waveforms 105a).

[0086] Next, the control unit extracts waveform features from the detection signal, calculates a respiration score, and uses the calculated respiration score to determine a dangerous state (step S1206). The control unit records the calculated respiration score in the recording unit 105 (respiration score 105b). In the embodiment, the processes of steps S1205 and S1206 are processes that are different from those of the conventional technology.

[0087] The control unit determines a dangerous state using the score of step S1206 (step S1207). For example, if it determines that the patient is in a hypopnea state (step S1207: Yes), the process proceeds to step S1208. If it determines that the patient is not in a hypopnea state (step S1207: No), the process proceeds to step S1210.

[0088] In step S1208, the control unit determines whether or not the hypopnea is continuing (step S1208). If the hypopnea is continuing (step S1208: Yes), the control unit outputs an alarm (step S1209) and ends the above process. On the other hand, if the hypopnea is not continuing (step S1208: No), the control unit proceeds to the process of step S1210.

[0089] In step S1210, the control unit waits until the next dangerous state is determined (step S1210), and then determines whether or not the human U has been redetected (step S1211). If the human U has been redetected (step S1211: Yes), the control unit returns to the processing of step S1201. On the other hand, if the human U has not been redetected (step S1211: No), the control unit determines whether or not the position of the human U has been updated (step S1212). If the position of the human has been updated (step S1212: Yes), the control unit returns to the processing of step S1204. On the other hand, if the position of the human has not been updated (step S1212: No), the control unit returns to the processing of step S1205.

[0090] As described above, according to the embodiment, the process includes identifying the area where the subject is located based on multiple detection signals within the measurement area, selecting and acquiring multiple detection signals from multiple positions within the identified area where the subject's body movement and vital signs indicate activity, extracting feature values ​​of the body movement and vital signs from the acquired multiple detection signals, and determining the subject's biological condition based on the extracted feature values. This improves the detection accuracy of the subject's biological information. Although the subject's posture and body movement change over time, the measurement position can be tracked and the subject can be continuously detected in a non-contact manner, thereby improving the detection accuracy of the subject's biological information.

[0091] In addition, in the embodiment, the extraction process may correct the signal strength of the detection signal based on the signal for the most recent time range, and extract the feature amounts of body movement and vital signs from the corrected detection signal. This allows the signal values ​​to be corrected to similar values, making it possible to focus only on relative changes in the signal values, and enabling evaluation that is not dependent on measurement conditions.

[0092] In addition, in the embodiment, the determination process may determine changes in the biological condition based on the time series of feature amounts of body movement and vital signs, which prevents the frequent occurrence of false alarms when determining based only on threshold values, and enables more realistic determination.

[0093] In addition, in the embodiment, the determination process may be performed by determining the respiratory condition of the subject based on the feature amount of vital signs, and determining that the subject is in a dangerous state when the respiratory condition continuously deteriorates over time. This allows the dangerous state based on the subject's respiratory condition to be determined at a more appropriate time, thereby improving accuracy.

[0094] In addition, in the embodiment, the determination process may be performed using a threshold or machine learning for the time series of feature amounts of body movement and vital signs. This makes it possible to more accurately determine changes in the biological state, such as a transition to a dangerous state, by determining, for example, a time series of states using a threshold or machine learning.

[0095] In addition, in the embodiment, human detection and location update may be performed periodically to confirm that the detection signal is a human-derived signal, thereby making it possible to determine the biological status of the subject based on the biological information as if the subject were a human.

[0096] In addition, in the embodiment, the multiple detection signals within the measurement area may be acquired from a radar that irradiates radar waves at multiple measurement points within the measurement area and outputs multiple detection signals based on the reflection of the waves. This makes it possible to acquire biometric information of a subject within the measurement area in a non-contact manner and to determine the biometric condition using the biometric information.

[0097] The biometric information processing method described in this embodiment can be realized by executing a prepared program on a computer such as a PC or a workstation. The biometric information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. The recording medium may be a hard disk, a flexible disk, a CD (Compact Disc)-ROM, an MO (Magneto Optical Disc), a DVD (Digital Versatile Disc), or the like. The biometric information processing program described in this embodiment may also be distributed via a network such as the Internet.

[0098] REFERENCE SIGNS LIST 100 Biometric information processing device 101 Acquisition unit 102 Feature extraction unit 102a Signal correction unit 103 Determination unit 105 Recording unit 201 CPU 202 Memory 203 Network I / F 204 Recording medium I / F 205 Recording medium E Room R Radar U Human (subject)

Claims

1. A biological information processing program that causes a computer to execute the following processes: identifying an area where a subject is present based on multiple detection signals within a measurement area; selecting and acquiring multiple detection signals from multiple positions within the identified area where the subject's body movement and vital information indicates activity; extracting feature amounts of body movement and vital information from the acquired multiple detection signals; and determining the biological condition of the subject based on the extracted feature amounts.

2. The bioinformation processing program according to claim 1, characterized in that the extraction process corrects the signal strength of the detection signal based on the signal of the most recent time range, and extracts the feature of body movement and vital signs from the corrected detection signal.

3. The biological information processing program according to claim 1, characterized in that the determination process determines changes in the biological condition based on the time series continuation of the feature amounts of body movement and vital signs.

4. The biometric information processing program according to claim 1, characterized in that the judgment process determines the subject's respiratory condition based on the vital signs' features, and judges the subject to be in a dangerous state when the respiratory condition deteriorates continuously over time.

5. The biometric information processing program according to claim 1, characterized in that the judgment process is performed using a threshold or machine learning for the time series continuation of the feature amounts of body movement and vital signs.

6. The biometric information processing program according to claim 1, characterized in that human detection and position update are periodically performed to confirm that the detection signal is a signal originating from a human.

7. The bioinformation processing program according to claim 1, characterized in that the multiple detection signals within the measurement area are obtained from a radar that irradiates radar waves at multiple measurement points within the measurement area and outputs multiple detection signals based on the reflection of the waves.

8. A biometric information processing method, characterized in that a computer performs the following processes: identifying an area where a subject exists based on a plurality of detection signals within a measurement area; selecting and acquiring a plurality of detection signals from the identified area at a plurality of positions where the subject's body movement / vital information has values ​​indicating activity; extracting feature amounts of body movement / vital information from the acquired plurality of detection signals; and determining the subject's biological condition based on the extracted feature amounts.

9. A biological information processing device characterized by having a control unit that performs the following processing: identifying an area where a subject is present based on multiple detection signals within a measurement area; selecting and acquiring multiple detection signals from multiple positions within the identified area where the subject's body movement / vital information indicates activity; extracting feature amounts of body movement / vital information from the acquired multiple detection signals; and determining the biological state of the subject based on the extracted feature amounts.

Images