Non-transitory computer-readable storage medium, biological information processing method, and biological

Abstract

To improve detection accuracy of biological information of an object person.SOLUTION: The radar R outputs a plurality of detection signals by irradiating the inside of the room E, which is a measurement region, with the radar R. In the biological information processing apparatus 100, the acquisition unit 101 acquires a detection signal, and specifies a range in which a human U who is a target exists. The acquisition unit 101 selects and acquires a plurality of detection signals at a plurality of positions where the body movement and vital sign information of the human U indicates an activity in the specified range. The signal correction unit 102a corrects the signal intensity of the detection signal based on the signal in the latest time range. The feature amount extraction unit 102 extracts the feature amounts of the body movement and the vital sign from the plurality of detection signals after the correction. The determination unit 103 determines a biological state, for example, a respiratory state of the human U based on the extracted feature amount. The determination unit 103 determines that the human U is in a dangerous state and outputs an alarm, for example, when the respiratory state continuously decreases in time series.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a biometric information processing program, a biometric information processing method, and a biometric information processing device. [Background technology]

[0002] By monitoring the position, posture, and vital signs of a human subject over time, it is possible to quickly detect abnormalities in the subject's biological information and respond promptly. For example, in medical and nursing care facilities and police station detention cells, there is a need to remotely monitor the behavior and health of people in private rooms.

[0003] For example, installing surveillance cameras to monitor people can provide a variety of information, but they may not function properly in dark places such as at night, and this can lead to problems with privacy violations. On the other hand, contact-based measurement using wearable devices can avoid privacy violations, but they require people to wear and manage the devices themselves, making them difficult to operate and preventing widespread adoption.

[0004] Monitoring using radar technology such as millimeter-wave sensors is attracting attention because it allows for contactless measurement while protecting privacy. Millimeter-wave sensors can generally grasp a person's movements and behavior, such as their direction and speed, based on radar transmission and reception signals, and can also obtain vital information such as breathing rate and heart rate by analyzing reflected signals from stationary people. Therefore, it is expected that radar technology can be used to monitor signal waveforms and heart and breathing rates, thereby quickly detecting dangerous conditions such as respiratory arrest.

[0005] Conventional techniques include, for example, a technique for detecting biological information using radar, selecting range pins in a spectrum with large phase changes, and performing FFT (Fast Fourier Transformation) to determine whether a person has stopped breathing or not. Another technique involves characterizing a subject's entry into a room and range pins based on fluctuations in the signal strength of reflected radio waves, generating motion data that tracks the subject's position, and determining dangerous conditions such as breathing based on temporal fluctuations in signal strength. Another technique involves removing unnecessary frequencies from received radar data to eliminate noise using the clutter effect, detecting one or more targets in specific range bins in the range data, and extracting the breathing rate based on the target's breathing signal. Another technique involves acquiring point cloud data of a person from a radar device and selecting the correct breathing rate for the person by referring to the likelihood of the person's different positions (see, for example, Patent Documents 1 to 4 listed below). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2023-123929 [Patent Document 2] Japanese Patent Publication No. 2020-81312 [Patent Document 3] US Patent Application Publication No. 2015 / 0369911 [Patent Document 4] Japanese Patent Application Laid-Open No. 2024-40042 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in conventional technology, the state of breathing, etc. is determined using the amount of displacement in the signal waveform of a specific point on the subject. In this case, for example, it is not possible to respond to changes in the amount of displacement or distribution of the intensity or phase of the detected signal due to changes in the subject's posture or position, etc., and a decrease in signal value due to changes in the subject's posture or position may result in an erroneous determination of respiratory cessation. In conventional technology, it was not possible to accurately determine the subject's biological information, such as respiratory cessation.

[0008] In one aspect, the present invention aims to improve the accuracy of detecting biological information of a subject. [Means for solving the problem]

[0009] According to one embodiment, a biometric information processing program, a biometric information processing method, and a biometric information processing device are proposed that include processing to identify an area where a subject is present based on a plurality of detection signals within a measurement area, select and acquire a plurality of detection signals from the identified area at a plurality of positions where the subject's body movement / vital information has values ​​indicating activity, extract feature values ​​of the body movement / vital information from the acquired plurality of detection signals, and determine the subject's biological condition based on the extracted feature values. [Effects of the Invention]

[0010] According to one aspect, it is possible to improve the accuracy of detecting biological information of a subject. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a biometric information processing apparatus according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of the hardware configuration of the biometric information processing device. [Figure 3] FIG. 3 is a diagram illustrating different cases of determining a dangerous state based on vital sign measurement using radar. [Figure 4A] FIG. 4A is an explanatory diagram of the human detection and signal acquisition process performed by the acquisition unit. [Figure 4B] FIG. 4B is an explanatory diagram of the signal correction and feature extraction processing performed by the feature extraction unit. [Figure 4C] FIG. 4C is an explanatory diagram of the determination process performed by the determination unit. [Figure 5] FIG. 5 is a diagram showing the state of a person and the state of determination in response to changes in the detection signal. [Figure 6] FIG. 6 is a diagram showing an example of human detection by radar. [Figure 7] FIG. 7 is a diagram illustrating an example of signal processing of the detection signal by the acquisition unit. [Figure 8] FIG. 8 is a diagram illustrating an example of signal correction processing performed by the signal correction unit. [Figure 9A] FIG. 9A is a diagram illustrating an example of feature extraction processing by the feature extraction unit (part 1). [Figure 9B] FIG. 9B is a diagram illustrating an example of feature extraction processing by the feature extraction unit (part 2). [Figure 10A] FIG. 10A is a diagram illustrating an example of a determination process performed by a determination unit (part 1). [Figure 10B] FIG. 10B is a diagram showing an example of a determination process by the determination unit (part 2). [Figure 11] FIG. 11 is an explanatory diagram of human detection and position update. [Figure 12] FIG. 12 is a flowchart illustrating an example of a processing procedure of the biometric information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of biometric information processing according to the present invention will be described in detail with reference to the drawings.

[0013] (An example of biometric information processing according to an embodiment) 1 is a diagram illustrating an example of a biometric information processing device according to an embodiment. The biometric information processing device 100 according to the embodiment is a computer that acquires a detection signal including biometric information of a subject within a predetermined measurement area and determines a biometric state of the subject, such as a state of breathing, from the detection signal.

[0014] The biometric information processing device 100 acquires detection signals at multiple positions within a measurement area using, for example, a non-contact millimeter wave radar, and selects multiple detection signals within the range in which the subject is present from the multiple detection signals acquired at predetermined time intervals, thereby responding to changes in the subject's position over time. The biometric information indicates the subject's body movements and various biological conditions, such as vital signs (e.g., pulse rate, heart rate, respiratory rate, body temperature, blood pressure, oxygen saturation, etc.). In the following embodiment, an example will be described in which the biometric information processing device 100 determines and notifies a dangerous state of respiratory arrest as the biological condition of the subject (human) based on the detection signals.

[0015] An overview of the functions and processing of the biometric information processing device 100 will be described using Fig. 1. The biometric information processing device 100 includes an acquisition unit 101, a feature extraction unit 102, a determination unit 103, and a recording unit 105. The acquisition unit 101 acquires a detection signal from a radar R installed in a measurement area for detecting a subject (human) U, for example, in a room E. The radar R emits radar waves to a plurality of measurement points 1 to n (n=15 in Fig. 1) in a matrix pattern when viewed on a plane in the room E, and detects the human U in the room E in a non-contact manner based on the reflected waves from each measurement point. Each function of the biometric information processing device 100 performs the following processing.

[0016] (1) Acquisition section 101 The acquisition unit 101 acquires a detection signal of a person present in a measurement area. The biometric information processing device 100 uses, for example, a radar R to irradiate a plurality of measurement points 1 to n in a room E where a person is present with a radar such as a millimeter wave, and outputs a detection signal of each measurement point 1 to n by the reflection of the radar. The acquisition unit 101 records the detection signal in the recording unit 105 (signal waveform (recording unit) 105a).

[0017] As shown in FIG. 1, the acquisition unit 101 extracts, for example, multiple detection signals at a position where a person U is present from the detection signals at multiple measurement points 1 to 15 at predetermined time intervals. The person U moves and changes his / her posture over time. For example, at a certain time, the person U shown by the solid line is detected at multiple measurement points 2, 6 to 8, and 11 to 13, which are indicated by black circles. After this, at a point in time when time has passed, the person U shown by the line is detected to be located at different measurement points 10, 14 to 15.

[0018] The acquisition unit 101 selects, at predetermined time intervals, detection signals from the plurality of measurement points 1 to 15 that have signal waveforms containing body movement or breathing information (predetermined values ​​of predetermined indices) near the detection positions, thereby allowing the acquired measurement points to track the movement of the person U. For example, among the n measurement points, the acquisition unit 101 selects and acquires a plurality of detection signals from a plurality of measurement points where the body movement and vital information of the person U has values ​​indicating activity. In the example of FIG. 1, the "values ​​of body movement and vital information indicating activity" refer to values ​​at the plurality of measurement points 2, 6 to 8, 11 to 13 indicated by black circles ● at a certain time that have valid information (e.g., values ​​greater than a predetermined value). Alternatively, the values ​​may be values ​​at the plurality of measurement points 2, 6 to 8, 11 to 13 that are relatively greater (e.g., by a predetermined value or more) than the values ​​of the other measurement points 1, 3 to 5, 9, 10, 14, and 15. The acquisition unit 101 determines that the body movement and vital information of the detection signals from the plurality of measurement points 2, 6 to 8, 11 to 13 indicates activity. As a result, the acquiring unit 101 of the embodiment tracks and acquires a detection signal having valid information corresponding to the presence of the human U within the measurement area.

[0019] For example, the acquisition unit 101 acquires a detection signal including position information when a stationary human U is detected. The predetermined time is, for example, a time for determining hypopnea. In this case, the acquisition unit 101 selectively extracts multiple detection signals having a lot of information (for example, having signal waveforms of amplitude and phase) from multiple measurement points near the position of the human U for each time for determining hypopnea, and performs signal processing to extract only the fluctuation components of each detection signal.

[0020] Determining the presence and movement / stationary status of a human U and acquiring location information when a stationary human U is detected is commonly performed using millimeter-wave radar R. For example, a biosignal detection technology has been proposed that simultaneously detects multiple people within a predetermined range and spatially separates the signals using a fast Fourier transform (FFT) to reduce noise signal interference and improve detection accuracy (see, for example, JP 2023-26124 A). Another technology has been proposed that removes invalid data based on the size of the radar's detection area and the signal's amplitude and intensity, removes the average value of the signal's amplitude, and eliminates offsets, thereby improving the accuracy of detecting stationary organisms (see, for example, JP 2023-80015 A). Another technology has been proposed that determines the presence or absence of a living organism at each distance based on the amplitude and phase spectrum obtained by Fourier transforming the amplitude / phase distribution of the radar signal at each distance over a predetermined time, thereby eliminating other stationary objects and noise and accurately detecting the location of the living organism (see, for example, JP 2021-30065 A). The acquisition unit 101 of the embodiment includes these existing technologies and uses radar to determine the presence of a human U and whether the human U is moving or stationary, and acquires location information when a stationary human U is detected.

[0021] Here, the acquisition unit 101 of the embodiment uses signal waveforms of detection signals at multiple positions selected based on a predetermined index at each predetermined time as a new configuration not disclosed in the past. This makes it possible to track changes in the actual measurement position due to changes in the posture and body movement of the person U between different times, and to continuously detect the person U in the room E over time in a non-contact manner.

[0022] (2) Feature Extraction Unit 102 The feature extraction unit 102 includes a signal correction unit 102a. The signal correction unit 102a corrects the signal strength by, for example, reading the most recently acquired detection signal from the recording unit 105 (signal waveform 105a) and dividing it by the average value of the detection signals acquired up to a certain time ago (detection signals in a resting state). The corrected detection signal is then stored again in the recording unit 105 (signal waveform 105a). This enables evaluation independent of measurement conditions. For example, the signal strength of a person U standing still at a distance from the radar R differs from that of a person U standing still at a position close to the radar R. However, by correcting the most recent detection signal with the detection signal in a resting state, the signal value is corrected to a similar value. This makes it possible to focus only on the relative change in the signal value.

[0023] The feature extraction unit 102 reads out the detection signal after signal correction by the signal correction unit 102a from the recording unit 105 (signal waveform 105a) and performs signal processing to extract multiple feature amounts related to the biological information of the human U, such as the variance of amplitude and phase fluctuations, maximum and minimum values, and respiratory energy ratio. These feature amounts also include general evaluation indices used when measuring the respiratory rate and heart rate, which are the biological state of the human U indicated by the biological information. The extracted feature amounts are recorded and stored in the recording unit 105.

[0024] (3) Determination unit 103 The determination unit 103 reads out the feature amounts extracted by the feature amount extraction unit 102 from the recording unit 105 and determines the biological state of the person, for example, the breathing or body movement state, by threshold determination or machine learning. For example, the determination unit 103 has a function to calculate a respiration score from the feature amounts and a function to detect a hypopnea (decrease in respiratory rate) state of the person based on the calculated score. For example, the calculated score is recorded and stored in the recording unit 105 (respiration score (recording unit) 105b). The determination unit 103 can set the feature amounts and the threshold for hypopnea regardless of conditions by making a determination using the signal value of the detection signal corrected as described above. In the embodiment, the dangerous state of the person is determined based on the state before respiratory arrest.

[0025] The determination unit 103 determines that a chronologically continuous hypopnea is a dangerous state and issues an alarm. In the embodiment, the determination unit 103 performs multi-stage determination, for example, issuing an alarm when the time during which a score calculated based on multiple indices continues to exceed a threshold (the number of consecutive detections) exceeds another threshold, thereby preventing false alarms.

[0026] 1, the recording unit 105 is provided with recording units for the signal waveform 105a and the respiration score 105b. However, the recording unit 105 may be configured to record signals in the respective functional units of the acquisition unit 101, the feature extraction unit 102, and the determination unit 103.

[0027] The biometric information processing device 100 having the above functions further periodically performs detection and position updates of the human U to ensure that the detection signal is a signal originating from the human U. For example, if the human U is moving / moving significantly, or if the signal is determined to be from a person other than the human U when the human U is detected, the above processing is reset.

[0028] The biometric information processing device 100 is, for example, various information devices such as a PC (Personal Computer) or a server. Alternatively, the biometric information processing device 100 may be a mobile terminal such as a smartphone owned by a person U. The method for processing biometric information according to the embodiment can be realized by, for example, an information device executing a program. Furthermore, each function of the biometric information processing device 100 can also be configured in the cloud.

[0029] (Example of hardware configuration of biometric information processing device) Next, an example of the hardware configuration of the biometric information processing device 100 will be described with reference to FIG.

[0030] Fig. 2 is a block diagram showing an example of the hardware configuration of a biometric information processing device. In Fig. 2, the biometric information processing device 100 has a CPU (Central Processing Unit) 201, a memory 202, a network I / F (Interface) 203, a recording medium I / F 204, and a recording medium 205. Furthermore, each component is connected to each other by a bus 200.

[0031] Here, the CPU 201 is a control unit that controls the entire biometric information processing device 100. The memory 202 has, for example, a read-only memory (ROM), a random access memory (RAM), and a flash ROM. Specifically, for example, the flash ROM or the ROM stores various programs, and the RAM is used as a work area for the CPU 201. The programs stored in the memory 202 are loaded into the CPU 201, causing the CPU 201 to execute coded processes.

[0032] The network I / F 203 is connected to the network NW via a communication line, and is connected to other computers via the network NW. The network I / F 203 manages the internal interface with the network NW and controls the input and output of data from other computers. The network I / F 203 is, for example, a modem or a LAN adapter.

[0033] The recording medium I / F 204 controls reading / writing of data from / to the recording medium 205 under the control of the CPU 201. The recording medium I / F 204 is, for example, a disk drive, a solid state drive (SSD), a universal serial bus (USB) port, etc. The recording medium 205 is a non-volatile memory that stores data written under the control of the recording medium I / F 204. The recording medium 205 is, for example, a disk, a semiconductor memory, a USB memory, etc. The recording medium 205 may be detachable from the biometric information processing device 100.

[0034] In addition to the above-mentioned components, the biometric information processing device 100 may also include, for example, a keyboard, a mouse, a display, a printer, a scanner, a microphone, a speaker, etc. The biometric information processing device 100 may also include a plurality of recording medium I / Fs 204 and recording media 205. The biometric information processing device 100 may also not include the recording medium I / Fs 204 and the recording media 205.

[0035] The functions of the acquisition unit 101 to the determination unit 103 shown in Fig. 1 can be realized by the CPU 201 executing a program stored in the memory 202 or the recording medium 205 shown in Fig. 2. Furthermore, each of the functions of the acquisition unit 101 to the determination unit 103 shown in Fig. 1 has a recording unit that records processed data, and the recording unit can be realized using the memory 202 or the recording medium 205 shown in Fig. 2.

[0036] (Comparison between the prior art and the embodiment) Next, the determination of biological information (vital measurement) using radar will be explained by comparing the conventional technology with the embodiment using various drawings. As disclosed in the technology of the above-mentioned patent documents, when a person moves, signal processing is performed on the transmitted and received signals to detect the distance, angle, and speed and track the person's position. Here, with the disclosed conventional technology, accurate vital measurement is not possible due to the influence of the person's movement, so only moving object detection is performed.

[0037] Furthermore, when a person is not moving, weak amplitude and phase signals are acquired through reflected signal analysis (e.g., Doppler analysis), and heart rate and respiratory rate are measured from the signal displacement. For example, vital signs are measured when a person is detected to be stationary. The measured heart rate and respiratory rate can then be used to determine the person's health condition and emotions. Radar-based vital sign measurement can detect dangerous conditions without contact with the person by monitoring the signal waveform and heart rate and respiratory rate.

[0038] Figure 3 is a diagram explaining different cases of determining a dangerous state by measuring vital signs using radar. In the NG example of Figure 3(a), if the value (level) of the detection signal indicating the vital state of the radar is weak, it is assumed that the subject (human) is in a dangerous state and an alert is issued. However, even if the human is asleep and at rest, the detection signal is weak, so it cannot be distinguished as a dangerous state and an alert is issued (false alert), which is a problem.

[0039] As shown in Figure 3(b), in the expected OK example, if the radar detection signal indicating the vital status is weak, the subject (human) is in danger and an alert is issued. If the human is asleep and resting, it is necessary not to issue an alert (prevent false alerts) even if the detection signal is weak. To achieve this, it is necessary to accurately determine whether the human is in danger and issue an alert without issuing false alerts.

[0040] The key to detecting dangerous conditions is accurately distinguishing between "dangerous conditions" that should trigger an alert and "conditions where the subject is at rest but the signal is weak or unmeasurable due to posture or detection position," which should not trigger an alert. Since dangerous conditions are generally considered to occur only a few times a year, even a single false positive detection per day would make practical use difficult. While some of the above-mentioned prior art patent documents detect respiratory arrest, they rely solely on a relative decrease or loss of signal value. Therefore, signal changes due to posture changes, slight movements, or misaligned detection positions may all be interpreted as dangerous conditions, resulting in frequent false positives. While it is possible to suppress false positives by setting strict thresholds, this also increases the likelihood of missed detections. Furthermore, when signals are extremely weak, human detection itself becomes difficult, making it difficult to verify that the signal is truly human-derived.

[0041] In summary, the conventional techniques have the following problems 1 to 3, for example. 1. In the direct method of determining a danger state when the signal displacement at a specific point decreases, a decrease in displacement due to body movement or changes in posture is also considered a danger state, resulting in frequent erroneous determinations (since the possibility of a danger state occurring in actual operation is low, it is essential to reduce erroneous determinations as much as possible). 2. It is difficult to distinguish extremely weak signals, such as those from respiratory arrest, from human signals, signals from other objects, or simple detection errors. 3. It is not possible to accurately detect the "state of a person's breathing slowing down" (between normal and respiratory arrest).

[0042] In response to the above-mentioned problems, an embodiment of the present invention accurately determines a "state in which a person's breathing is decreasing" based on a radar detection signal measured non-contact from a long distance, thereby quickly and accurately determining a dangerous state.

[0043] Hereinafter, each function of the biometric information processing apparatus 100 according to the embodiment described with reference to FIG. 1 will be described in comparison with the prior art.

[0044] (signal acquisition) 4A is an explanatory diagram of the human detection and signal acquisition process performed by the acquisition unit 101. The acquisition unit 101 performs the following processes.

[0045] 1. Determine the presence and moving / stationary status of a human U. This is commonly done with millimeter wave radars. 2. Obtain location information when a stationary human U is detected. This is also commonly done with radar. 3. Amplitude and phase signal waveforms are acquired from measurement points near the position of the person U at predetermined time intervals, for example, at regular time intervals. 4. Based on the evaluation index, multiple amplitude and phase signal waveforms with a large amount of information are selectively extracted. 5. Extract only the fluctuation components of each signal through signal processing. For example, remove trends using a Savitzky-Golay filter.

[0046] Conventional technology only estimates the position, respiratory rate, and heart rate of human U, but does not detect respiratory arrest based on signal values ​​or vital information. Figure 4(a) shows changes in a person's position over time. Conventional technology acquires and tracks signal values ​​at specific points with fixed positions indicated by black circles, and determines respiratory arrest based on a relative decrease or disappearance of the signal value. Such conventional technology may determine respiratory arrest when the posture or state of human U at time t0 changes at time t1, weakening the signal from a specific position, or when a position where human U is not present is mistakenly detected as human U, resulting in frequent false alarms.

[0047] In contrast, in the embodiment, the acquisition unit 101 selects a plurality of positions of the human U from measurement points 1 to n (see FIG. 1) at each time t0, t1, ... and acquires signals. For the detection signals of the plurality of measurement points indicated by white circles, the acquisition unit 101 selects, for each time, detection signals of signal waveforms of the plurality of measurement points that have information effective as body movement or breathing information (predetermined values ​​for predetermined indicators) near the detection positions, for example, body movement and vital information values ​​that indicate activity. In this way, the measurement points to be acquired are made to follow the movement of the human U. In other words, the acquisition unit 101 selects detection signals of the plurality of measurement points located at the position of the moving human U.

[0048] Fig. 4A(b) is a diagram showing the level of the detection signal over time. The horizontal axis represents time, and the vertical axis represents the level of the detection signal. In conventional technology, the signal value at a specific point (position A) is acquired. Therefore, for example, as shown in Fig. 4A(a), if the posture or state of person U changes and they deviate from the specific point (position A), the signal at the specific point (position A) becomes weak, leading to a false alarm that respiratory arrest has occurred.

[0049] In contrast, in the embodiment, signals having valid information are tracked and acquired for each time. In the example of Fig. 4A(b), at times A to C, among the plurality of measurement points, detection signals of signal waveforms at a plurality of measurement points where the body movement or respiratory information has a predetermined value indicating activity are selected for each time. For convenience, Fig. 4A(b) shows only one signal waveform at one measurement point, but in reality, among the plurality of detection signals at a plurality of n measurement points at each of times A to C, detection signals of signal waveforms at a plurality of measurement points n1 where the body movement or respiratory information has a predetermined value indicating activity are selected for each time.

[0050] The acquisition unit 101 selectively extracts multiple detection signals having a lot of information (e.g., having amplitude and phase signal waveforms) from multiple measurement points near the position of the person U at each hypopnea determination time, and performs signal processing to extract only the fluctuation components of each detection signal. The extraction is performed by removing trends using, for example, a Savitzky-Golay filter. Note that, in response to changes in the position and posture of the person U, some or all of the multiple measurement points (detection signals) selected at times A to C may differ between times A, B, and C.

[0051] As described above, in the embodiment, unlike the prior art in which a detection signal is acquired from a specific fixed position, a plurality of detection signals having predetermined values ​​(signal waveforms) are selected and acquired for each time based on an index, thereby enabling tracking of changes in posture and body movements of the person U over time.

[0052] (Feature extraction) 4B is an explanatory diagram of the signal correction and feature extraction processing performed by the feature extraction unit 102. The feature extraction unit 102 performs the following processing.

[0053] 1. When the amplitude and phase fluctuation components of the detection signal are acquired at each time, the average value of the absolute values ​​of the amplitude / phase fluctuation components is saved. 2. Correct the latest detection signal by dividing it by the average value of the above average values ​​obtained up to a certain time before. 3. Calculate multiple features such as variance, maximum / minimum, and respiratory energy ratio from the corrected signal.

[0054] First, the signal correction unit 102a of the feature extraction unit 102 corrects the signal strength by dividing the most recently acquired detection signal by the average value of the detection signals (detection signals in a resting state) acquired up to a certain time before, which enables evaluation that is independent of measurement conditions.

[0055] FIG. 4B(a) shows a radar waveform, with the horizontal axis representing time and the vertical axis representing the level for each condition. Before correction, the signal strength of the detection signal differs for each condition between a person U standing still at a distance from the radar R sensor and a person U standing still close to the radar R sensor. For this reason, the signal correction unit 102a corrects the most recent detection signal using, for example, a detection signal in a resting state (a past signal) as shown in FIG. 4B(b). This results in a signal level that is roughly the same regardless of the condition (difference in distance from the sensor, etc.), as shown after correction in FIG. 4B(a), making it possible to focus only on the relative change in signal value.

[0056] The feature extraction unit 102 extracts multiple feature values, such as the variance of amplitude and phase fluctuations, maximum and minimum values, and respiratory energy ratio, from the detection signal corrected by the signal correction unit 102a. These feature values ​​include general evaluation indices that are also used when measuring respiratory rate and heart rate. Furthermore, by correcting the detection signal as described above, it becomes possible to fix the feature values ​​and thresholds for determining hypopnea, etc., regardless of conditions.

[0057] Figure 4B(c) shows the determination state using conventional technology. The horizontal axis represents time, and the vertical axis represents the detection signal and the determination state of respiratory arrest. Conventional technology makes a simple determination based only on the magnitude of the phase fluctuation amount after frequency processing. Because it only performs a binary determination based on a threshold for a single indicator, fluctuations in the value of the detection signal frequently result in false detection (false determination) of respiratory arrest (0).

[0058] As described above, in the embodiment, signal correction of the detection signal enables the determination and evaluation of the hypopnea state independent of the measurement conditions. Furthermore, signal correction makes it possible to set the feature amount and the hypopnea threshold value independent of the conditions. The feature amount may be a general evaluation index that is also used when measuring the respiratory rate and heart rate.

[0059] (judgement) 4C is an explanatory diagram of the determination process performed by the determination unit 103. The determination unit 103 performs the following process for, for example, hypopnea / dangerous state.

[0060] 1. A respiration score is calculated from the feature amount using threshold judgment or machine learning, for example, a score between 0 and 1. The human respiratory state is then evaluated based on the score count, and if the score exceeds the threshold, the person is judged to be in a hypopnea state. 2. If respiratory depression continues over time, it is determined to be a dangerous condition and an alarm is sounded.

[0061] In addition, the system periodically performs human detection and location updates to ensure that the signal is human-originated. If a human moves or if the signal is determined to be non-human, the count is reset.

[0062] FIG. 4C(a) shows a respiration score based on the feature amounts. The horizontal axis represents time, and the vertical axis represents the respiration score. The determination unit 103 calculates the respiration score from the feature amounts output by the feature amount extraction unit 102. The determination unit 103 calculates the respiration score from the feature amounts by threshold determination or machine learning, and detects a state of hypopnea in a person based on the calculated score.

[0063] FIG. 4C(b) shows the state of determining a dangerous state based on the respiration score. The horizontal axis represents time, and the vertical axis represents the number of detected hypopneas in FIG. 4C(a). In the example of FIG. 4C(b), the determination unit 103 determines whether a hypopnea occurs at threshold 1, and whether a dangerous state occurs at threshold 2. The determination unit 103 determines whether a hypopnea occurs or whether a dangerous state occurs based on the number of consecutive hypopneas. For example, in the example of FIG. 4C(b), if the number of consecutive detected hypopneas (corresponding to the continuous time) is small and exceeds only threshold 1 (less than threshold 2), it is determined to be hypopnea (not a dangerous state). After this, when the number of consecutive hypopneas increases and exceeds both thresholds 1 and 2, it is determined to be a dangerous state.

[0064] As described above, in the embodiment, instead of a binary judgment based on a threshold for a single index as in the prior art, an alarm is issued when a score calculated based on multiple indexes exceeds a threshold for a certain period of time or more. This makes it possible to maintain a high rate of hypopnea detection while suppressing false positives through multi-stage judgment.

[0065] (Determination state based on change in detection signal) FIG. 5 is a diagram showing the state of a person and the judgment state in response to changes in the detection signal. The horizontal axis is time, and the vertical axis is signal (level). During the normal state (period A) when the detection signal has a predetermined large value (level), both the conventional technology and the embodiment correctly judge the dangerous state as "no alarm is issued." The above-mentioned "values ​​indicating activity in the subject's body movement and vital information" are as shown in periods A to B (B1, B3) in FIG. 5, excluding period C, when the detection signal has a predetermined value.

[0066] Furthermore, when the value of the detection signal decreases and becomes difficult to measure due to a change in the posture or state of the person U (time B1), the person U remains in a normal state, but in the conventional technology, an "alert" of a dangerous state is mistakenly issued. In contrast, in the embodiment, as described above, another measurement point is selected in response to the change in posture, so it can be correctly determined that "no alert is issued."

[0067] Furthermore, when a position where no human U is present is mistakenly detected as a human (time period B2), for example, when the signal becomes extremely small due to a detection error, the prior art would mistakenly "issue an alarm" indicating a dangerous condition. In contrast, in the embodiment, as described above, the determination is not based solely on the signal level at a specific point, so it can correctly determine that "no alarm will be issued."

[0068] Furthermore, when the value (level) of the detection signal changes downward due to decreased breathing of the person U (time B3), both the prior art and the embodiment correctly judge that the dangerous state is dangerous and issue an "alert."

[0069] Furthermore, when the breathing of the person U stops and the value (level) of the detection signal becomes "no signal" (time C), neither the conventional technology nor the embodiment can detect only the person U. For example, it is difficult to distinguish the person U from other stationary objects.

[0070] 5, according to the embodiment, it is possible to prevent erroneous alarms like those in the prior art when the posture or state of human U changes (time period B1) and when a position where human U is not present is mistakenly detected as a human (time period B2). In the prior art, an alarm is issued during time period B, that is, when the posture or state of human U changes (time period B1), when a position where human U is not present is mistakenly detected as a human (time period B2), and also when respiratory depression occurs (time period B3).

[0071] However, when human U stops breathing (time C), it is difficult to detect a human in both the conventional technology and the embodiment. For this reason, the embodiment makes it possible to correctly determine the "state in which a human's breathing is decreasing (time B3)" based on a detection signal. In the embodiment, the above-mentioned configuration makes it possible to distinguish between each state that transitions to a dangerous state within time B between the normal state (time A) and respiratory arrest (time C) (no alarm is issued at times B1 and B2, and an alarm is issued at B3), thereby accurately determining the dangerous state at a more appropriate and accurate time.

[0072] (Example of signal processing for each functional unit) Next, an example of signal processing by each functional unit of the bio-information processing device 100 will be described.

[0073] FIG. 6 is a diagram showing an example of human detection by radar. FIG. 6 shows the state of radar R and human U in a plan view. Radar R is installed, for example, in the center of the ceiling of a room E or the like. This radar R detects human U present in room E in a non-contact manner. For example, radar R emits millimeter waves to a plurality of measurement points in a matrix shown by coordinates x and y within room E, and outputs a detection signal that is updated every 10 seconds with stationary position information of the reflected waves from human U. The detection signal includes information on the number of counts for each position (measurement point) where human U is stationary.

[0074] For convenience, the embodiment will describe an example of detecting one person U in a room E. As shown in Fig. 6, since the count numbers corresponding to the position of the person U are detected in clusters, it is also possible to detect multiple people U individually in the room E. In this case, biometric information for the multiple people U can be obtained and the dangerous state can be determined individually.

[0075] FIG. 7 is a diagram showing an example of signal processing of detection signals by the acquisition unit. FIG. 7 shows detection signals selected by the acquisition unit 101. For example, the acquisition unit 101 selects detection signals from five measurement points with large signal deviations every five seconds from the detection information of multiple measurement points output by the radar R, and extracts the signals. In the example of FIG. 7, the acquisition unit 101 has selected detection signals from five measurement points with large amplitude fluctuations and phase fluctuations, and the detection signals each have amplitude fluctuations and phase fluctuations with predetermined values ​​(levels) for a predetermined period of time. This makes it possible to acquire detection signals from the position where the person U was present in the room E.

[0076] Fig. 8 is a diagram showing an example of signal correction processing performed by the signal correction unit 102a. The signal correction unit 102a performs correction on the detection signals at the five measurement points acquired in Fig. 7, normalizing the signal values ​​using data for a predetermined period, for example, the most recent 300 seconds.

[0077] 8 shows that, out of 300 seconds, the average fluctuation of the detection signal value at time t=5 (seconds) is "1.7" and the average fluctuation of the detection signal value at t=10 is "1.4." The signal correction unit 102a then calculates correction data by dividing the detection signal value at time t=300 by the average of the fluctuation averages of past data. This makes it possible to correct the intensity of the detection signal and extract multiple waveform feature amounts, and also makes it possible to fix thresholds for determining feature amounts and hypopnea, etc., regardless of conditions.

[0078] 9A and 9B are diagrams showing an example of feature extraction processing by the feature extraction unit. FIG. 9A shows a group of signals (amplitude fluctuation amount and phase fluctuation amount) corrected by the signal correction unit 102a. FIG. 9B shows feature amounts extracted by the feature extraction unit 102. The feature extraction unit 102 extracts feature amounts of signal amplitude and phase from the group of signals corrected by the signal correction unit 102a (FIG. 9A). As shown in FIG. 9B, for example, the feature extraction unit 102 extracts a phase fluctuation average, an amplitude fluctuation average, a phase energy ratio, and an amplitude energy ratio as multiple feature amounts.

[0079] 10A and 10B are diagrams showing an example of the determination process by the determination unit. FIG. 10A(a) shows scoring based on feature amounts by the determination unit 103, with the horizontal axis representing time and the vertical axis representing the score. In the case of determination using the threshold shown in FIG. 10A(a), the determination unit 103 compares the average fluctuations of the amplitude and phase signals indicated by the multiple feature amounts extracted by the feature amount extraction unit 102 (FIG. 9B) and four feature amounts of respiratory energy with thresholds to score the respiratory state. The score ranges from 0 to 1 in increments of 0.25, for example.

[0080] 10A(b) shows an example of determination using machine learning. The determination unit 103 determines the possibility of a hypopnea state using a model such as a support vector machine (SVM) or a decision tree based on the waveforms of multiple feature amounts (FIG. 9B) extracted by the feature amount extraction unit 102, for example, 10 to 20 feature amounts, and scores the respiratory state. The score ranges, for example, from 0 to 1 point. The area with a high score surrounded by the dashed-dotted line shown in FIGS. 10A(a) and (b) corresponds to the respiratory arrest area.

[0081] FIG. 10B shows a risk condition determination based on a score. The determination unit 103 determines hypopnea from the respiration score shown in FIG. 10A. In this determination, the risk condition is determined based on a state in which hypopnea continues. For example, if the respiration score shown in FIG. 10B(a) is greater than 0.5, the risk condition is determined to be a hypopnea state. Also, as shown in FIG. 10B(b), the vertical axis represents the number of hypopnea detections in the most recent 10 times, and the determination unit 103 determines that the risk condition is a risk condition and outputs an alarm if the hypopnea state occurs six or more times in the most recent 10 determinations (50 seconds).

[0082] FIG. 11 is an explanatory diagram of human detection and position update. The biometric information processing device 100 periodically performs human detection and position update, such as at regular intervals. If a human U is detected at each measurement point during this update, the device continues processing related to hypopnea determination. On the other hand, if a human U is not detected at each measurement point during the update, the device stops processing related to hypopnea determination. For example, if a human U is not detected at all of the multiple measurement points, the device stops processing related to hypopnea determination.

[0083] (Example of processing by a biometric information processing device) Fig. 12 is a flowchart showing an example of a processing procedure of the biometric information processing device. The processing shown in Fig. 12 is executed by, for example, the control unit (CPU 201) shown in Fig. 2. The control unit acquires a signal sequence of a plurality of detection signals output by the radar R and executes detection of a human U (step S1201).

[0084] Then, the control unit determines whether the human U is stationary or not based on the detection signal (step S1202). If the human U is not stationary (step S1202: No), the control unit deletes the history of the signal and breathing score recorded in the recording unit 105 (step S1203), and ends the above processing.

[0085] On the other hand, if the person U is stationary (step S1202: Yes), the control unit updates and acquires position information of the detection signal (step S1204). Next, the control unit searches for multiple feature points near the person U, acquires signal waveforms, and performs signal correction (step S1205). The control unit records the acquired detection signals in the recording unit 105 (signal waveforms 105a).

[0086] Next, the control unit extracts waveform features from the detection signal, calculates a respiration score, and uses the calculated respiration score to determine a dangerous state (step S1206). The control unit records the calculated respiration score in the recording unit 105 (respiration score 105b). In the embodiment, the processes of steps S1205 and S1206 are distinctive processes that differ from those of the prior art.

[0087] The control unit determines a dangerous state using the score of step S1206 (step S1207). For example, if it determines that the patient is in a hypopnea state (step S1207: Yes), the process proceeds to step S1208, and if it determines that the patient is not in a hypopnea state (step S1207: No), the process proceeds to step S1210.

[0088] In step S1208, the control unit determines whether or not the hypopnea is continuing (step S1208). If the hypopnea is continuing (step S1208: Yes), the control unit outputs an alarm (step S1209) and ends the above process. On the other hand, if the hypopnea is not continuing (step S1208: No), the control unit proceeds to the process of step S1210.

[0089] In step S1210, the control unit waits until the next dangerous state is determined (step S1210), and then determines whether or not the human U has been detected again (step S1211). If the human U has been detected again (step S1211: Yes), the control unit returns to the processing of step S1201. On the other hand, if the human U has not been detected again (step S1211: No), the control unit determines whether or not the position of the human U has been updated (step S1212). If the position of the human has been updated (step S1212: Yes), the control unit returns to the processing of step S1204. On the other hand, if the position of the human has not been updated (step S1212: No), the control unit returns to the processing of step S1205.

[0090] As described above, according to the embodiment, the process includes identifying the area where the subject is located based on multiple detection signals within the measurement area, selecting and acquiring multiple detection signals from multiple positions within the identified area where the subject's body movement and vital signs indicate activity, extracting feature values ​​of the body movement and vital signs from the acquired multiple detection signals, and determining the subject's biological condition based on the extracted feature values. This improves the detection accuracy of the subject's biological information. Although the subject's posture and body movement change over time, the system can track changes in the measurement position and continuously detect the subject in a non-contact manner, thereby improving the detection accuracy of the subject's biological information.

[0091] In addition, in the embodiment, the extraction process may correct the signal strength of the detection signal based on the signal from the most recent time range, and extract the feature amounts of body movement and vital signs from the corrected detection signal. This allows the signal values ​​to be corrected to similar values, making it possible to focus only on relative changes in the signal values, and enabling evaluation that is not dependent on measurement conditions.

[0092] In addition, in the embodiment, the determination process may be configured to determine changes in the biological condition based on the time series of feature amounts of body movement and vital signs, thereby preventing the frequent occurrence of false alarms when determining based only on threshold values, and enabling more realistic determinations.

[0093] In addition, in the embodiment, the determination process may be performed by determining the respiratory condition of the subject based on the feature amount of vital signs, and determining that the subject is in a dangerous state when the respiratory condition continuously deteriorates over time. This allows the dangerous state based on the subject's respiratory condition to be determined at a more appropriate time, thereby improving accuracy.

[0094] In addition, in the embodiment, the determination process may be performed using a threshold or machine learning for the time series of feature amounts of body movement and vital signs. This makes it possible to more accurately determine changes in biological conditions, such as a transition to a dangerous state, by determining the time series of states using a threshold or machine learning.

[0095] In addition, in the embodiment, human detection and location updates may be periodically performed to confirm that the detection signal is a human signal, thereby making it possible to determine the biological status of the subject based on the biological information as if they were a human.

[0096] In addition, in the embodiment, the multiple detection signals within the measurement area may be acquired from a radar that irradiates radar waves at multiple measurement points within the measurement area and outputs multiple detection signals based on the reflection of the waves. This makes it possible to acquire biometric information of a subject within the measurement area in a non-contact manner and to determine the biometric condition using the biometric information.

[0097] The biometric information processing method described in this embodiment can be realized by executing a prepared program on a computer such as a PC or a workstation. The biometric information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. The recording medium may be a hard disk, a flexible disk, a CD (Compact Disc)-ROM, an MO (Magneto Optical disc), a DVD (Digital Versatile Disc), or the like. The biometric information processing program described in this embodiment may also be distributed via a network such as the Internet.

[0098] The following additional notes are provided regarding the above-described embodiment.

[0099] (Appendix 1) Based on multiple detection signals within the measurement area, the area in which the subject is present is identified, Selecting and acquiring a plurality of the detection signals at a plurality of positions where the body movement / vital information of the subject indicates activity within the specified range, Extracting feature quantities of body movement and vital signs from the acquired multiple detection signals; determining a biological state of the subject based on the extracted feature amount; A biological information processing program that causes a computer to execute processing.

[0100] (Note 2) The extraction process corrects the signal intensity of the detection signal based on the signal in the most recent time range, and extracts the feature amount of body movement and vital signs from the corrected detection signal. 2. The biometric information processing program according to claim 1,

[0101] (Appendix 3) The determination process determines a change in the biological condition based on the time series of the feature amounts of body movement and vital signs. 2. The biometric information processing program according to claim 1,

[0102] (Appendix 4) The determination process includes determining the respiratory condition of the subject based on the feature amount of the vital signs, and determining that the subject is in a dangerous state when the respiratory condition continuously deteriorates over time. 2. The biometric information processing program according to claim 1,

[0103] (Appendix 5) The judgment process is performed using a threshold or machine learning for the time series of the feature amounts of body movement and vital signs. 2. The biometric information processing program according to claim 1,

[0104] (Appendix 6) Human detection and location updates are periodically performed to verify that the detection signal is human-originated. 2. The biometric information processing program according to claim 1,

[0105] (Appendix 7) The plurality of detection signals within the measurement area are obtained from a radar that irradiates radar waves at a plurality of measurement points within the measurement area and outputs a plurality of detection signals based on the reflection of the waves. 2. The biometric information processing program according to claim 1,

[0106] (Appendix 8) Identify the area where the subject is located based on multiple detection signals within the measurement area, Selecting and acquiring a plurality of the detection signals at a plurality of positions where the body movement / vital information of the subject indicates activity within the specified range, Extracting feature quantities of body movement and vital signs from the acquired multiple detection signals; determining a biological state of the subject based on the extracted feature amount; A biometric information processing method characterized in that the processing is executed by a computer.

[0107] (Appendix 9) Based on multiple detection signals within the measurement area, the area in which the subject exists is identified, Selecting and acquiring a plurality of the detection signals at a plurality of positions where the body movement / vital information of the subject indicates activity within the specified range, Extracting feature quantities of body movement and vital signs from the acquired multiple detection signals; determining a biological state of the subject based on the extracted feature amount; A biological information processing device comprising a control unit for performing processing. [Explanation of symbols]

[0108] 100 Biometric information processing device 101 Acquisition Department 102 Feature Extraction Unit 102a Signal correction section 103 Judgment section 105 Recording section 201 CPU 202 memory 203 Network I / F 204 Recording Media I / F 205 Recording Media Room E R radar U Human (subject)

Claims

1. Identifying the area where the subject is present based on multiple detection signals within the measurement area; Selecting and acquiring a plurality of the detection signals at a plurality of positions where the body movement / vital information of the subject indicates activity within the specified range; extracting feature amounts of body movement and vital signs from the acquired plurality of detection signals; determining a biological state of the subject based on the extracted feature amount; A biological information processing program that causes a computer to execute processing.

2. The extraction process includes correcting the signal intensity of the detection signal based on a signal within a most recent time range, and extracting the feature amount of body movement and vital signs from the corrected detection signal.

2. The biometric information processing program according to claim 1.

3. The determination process determines a change in the biological condition based on a time series of the feature amounts of body movement and vital signs.

2. The biometric information processing program according to claim 1.

4. The determination process includes determining a respiratory condition of the subject based on the feature amount of vital signs, and determining that the subject is in a dangerous state when the respiratory condition continuously deteriorates in a time series.

2. The biological information processing program according to claim 1.

5. The determination process is performed using a threshold or machine learning for the time series of the feature amounts of body movement and vital signs.

2. The biological information processing program according to claim 1.

6. periodically performing human detection and location updates to verify that the detection signals are human-originated; 2. The biometric information processing program according to claim 1.

7. The plurality of detection signals within the measurement area are acquired from a radar that irradiates radar waves at a plurality of measurement points within the measurement area and outputs a plurality of detection signals based on the reflections of the waves.

2. The biological information processing program according to claim 1.

8. Based on the multiple detection signals within the measurement area, the area in which the subject is present is identified, Selecting and acquiring a plurality of the detection signals at a plurality of positions where the body movement / vital information of the subject indicates activity within the specified range; extracting feature amounts of body movement and vital signs from the acquired plurality of detection signals; determining a biological state of the subject based on the extracted feature amount; A biometric information processing method characterized in that the processing is executed by a computer.

9. Identifying the area where the subject is present based on multiple detection signals within the measurement area; Selecting and acquiring a plurality of the detection signals at a plurality of positions where the body movement / vital information of the subject indicates activity within the specified range; extracting feature amounts of body movement and vital signs from the acquired plurality of detection signals; determining a biological state of the subject based on the extracted feature amount; A biological information processing device comprising a control unit for performing processing.

Images