Information processing program, information processing method, and information processing apparatus

Abstract

To improve the accuracy of measuring vital signs in people. [Solution] The computer performs a process to continue the second analysis of the radio signal on the radio signal if the extracted waveform satisfies predetermined conditions. This process involves first analyzing the radio signal corresponding to an object moving within a measurement space to obtain the activity level of the object, second analyzing the radio signal to obtain the signal waveform of the object from the radio signal by detecting smaller movements than those detected in the first analysis, identifying the vital range from the acquired activity level based on a pre-stored correspondence between the activity level and the vital range, extracting the waveform corresponding to the identified vital range from the signal waveform, and continuing the second analysis of the radio signal if the extracted waveform satisfies predetermined conditions.

Description

Technical Field

[0006] , , , , , , , , ,

[0007] ,

[0001] The present invention relates to an information processing program, an information processing method, and an information processing apparatus.

Background Art

[0002] In medical and nursing facilities, detention facilities in police stations, etc., there is an increasing demand for technology to remotely monitor the behavior and health status of people in individual rooms. In remote monitoring, the use of action recognition technology using cameras is effective.

[0003] However, not only is it impossible to ensure sufficient action recognition accuracy at night, but there is also a risk of compromising the privacy of the monitored person. Although wearable terminals may be used as an alternative to cameras by means of acceleration sensors and vital sensors, handling such as charging is also necessary, and it is not something that can always be worn, especially for the elderly and dementia patients.

[0004] Therefore, sensing using a millimeter-wave radar has attracted attention. A millimeter-wave radar can measure the position, action, and vital signs of a person by irradiating a radio wave signal and analyzing the reflected signal from the object. Since it does not record a person's face or living space as an image like a camera, privacy-conscious sensing becomes possible.

[0005] In the prior art, there is a known technique for measuring the respiratory rate of humans and animals non-contact using a radar and monitoring apnea syndrome during sleep, etc.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] Because millimeter-wave radar sensing analyzes the movement of objects using radio signals, it is difficult to determine whether an object is a person based on its appearance, unlike with cameras. Therefore, vital signs may be measured for objects other than people. For example, if vital signs are measured for a curtain swaying in the wind, just as they would for a person, abnormal vital sign readings may trigger unnecessary alerts and notifications, potentially increasing the workload of the monitoring personnel.

[0008] In one aspect, the objective is to provide an information processing program, information processing method, and information processing device that can improve the accuracy of measuring a person's vital signs. [Means for solving the problem]

[0009] In one embodiment, an information processing program obtains the activity level of an object moving within a measurement space by analyzing a first radio wave signal. When the activity level of the object is no longer obtainable, it obtains the signal waveform of the object by analyzing a second radio wave signal that detects smaller movements than those detected by the analysis of the first radio wave signal. Based on a pre-stored correspondence between the activity level of an organism and its vital range, the program identifies the vital range from the activity level of the object, extracts the waveform corresponding to the vital range from the signal waveform, and, if the waveform satisfies predetermined conditions, causes the computer to continue the analysis of the second radio wave signal for the object. [Effects of the Invention]

[0010] In one respect, it can provide an information processing program that can improve the accuracy of measuring a person's vital signs. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a diagram illustrating an example of the configuration of a vital sign measurement system according to an embodiment. [Figure 2] Figure 2 is an example of a functional block diagram showing the functional configuration of the information processing device 11 according to the embodiment. [Figure 3]Figure 3 shows an example of the displacement DB222 in the embodiment. [Figure 4] Figure 4 is an example of a functional block diagram showing the functional configuration of the radar device 13 according to the embodiment. [Figure 5] Figure 5 is a diagram illustrating the overview of the tracking process for the object being measured according to the example. [Figure 6] Figure 6 shows the object to be measured before the activity according to the example. [Figure 7] Figure 7 shows the object to be measured after the activity according to the example. [Figure 8] Figure 8 shows an example of the correspondence relationship DB220 between activity levels and corresponding vital signs in the embodiment. [Figure 9] (a) is an example of a signal waveform received from the curtain 60 according to the embodiment. (b) is an example of a waveform extracted by applying the frequency filter 221 to the signal waveform received from the curtain 60 according to the embodiment. [Figure 10] (a) is an example of a signal waveform received from person 61 according to the embodiment. (b) is an example of a waveform extracted by applying the frequency filter 221 to the signal waveform received from person 61 according to the embodiment. [Figure 11] Figure 11 is an example of a flowchart illustrating the flow of the activity level acquisition process according to the embodiment. [Figure 12] Figure 12 is an example of a flowchart illustrating the signal waveform analysis process according to the embodiment. [Figure 13] Figure 13 is an example of a flowchart illustrating the flow of the person identification process according to the embodiment. [Figure 14] Figure 14 is a diagram illustrating an example of the hardware configuration of an information processing device according to an embodiment. [Modes for carrying out the invention]

[0012] Hereinafter, embodiments of an information processing program, an information processing method, and an information processing apparatus according to the present invention will be described in detail while referring to the drawings. Note that these embodiments are merely examples for implementing the present invention and do not limit the present invention. Further, in this embodiment, an embodiment of measuring the heart rate, which is one of the vital signs, by analyzing radar is described, but the types of vital signs and the analysis targets are not limited to the contents of this embodiment.

Embodiment

[0013] [Description of System] FIG. 1 is a diagram for explaining an example of the configuration of a vital sign measurement system according to this embodiment. In the vital sign measurement system 10, an information processing apparatus 11 and a radar apparatus 13 are communicably connected to each other via a network 12. The vital sign measurement system 10 determines whether a person 14 to be measured is a person, for example, in a private room of a hospital or a nursing facility, and measures the vital signs of the person 14.

[0014] The information processing apparatus 11 acquires a radio wave signal of the person 14 from the radar apparatus 13 via the network 12, performs signal processing, and acquires data on the activity amount and the signal waveform. The information processing apparatus 11 is an apparatus that determines whether the person 14 to be measured is a person by analyzing the data and measures the vital signs of the person 14. As an example, the information processing apparatus 11 is a personal computer, a smartphone, a tablet terminal, or the like.

[0015] The network 12 is a communication network such as the Internet or an intranet, regardless of whether it is wired or wireless. As an example, the network 12 is configured by a plurality of intranets, the Internet and an intranet, etc. via devices such as a gateway.

[0016] The radar apparatus 13 is an apparatus that irradiates a measurement space with radio waves such as millimeter-wave radar, and acquires information on the measurement object by receiving the reflected wave. The radar apparatus 13 transmits data of the radio wave signal to the information processing apparatus 11 via the network 12.

[0017] Person 14 is a person who is being measured in a private room in a hospital or nursing home.

[0018] The information processing device 11 in this embodiment has multiple analysis processing methods when analyzing the data of the received radio wave signal. The information processing device 11 switches the analysis processing method depending on whether the person 14 is moving or not.

[0019] The information processing device 11 calculates the difference in frequency between the transmitted wave and the reflected wave up to the person 14, as well as the frequency shift due to the Doppler effect, if the person 14 is moving. The information processing device 11 calculates the distance, angle, and velocity to the person 14 by processing the signal from the frequency difference and shift.

[0020] Furthermore, the information processing device 11 acquires point cloud data, which is a set of reflection points corresponding to the person 14 (hereinafter referred to as the "point cloud"), based on information such as the distance, angle, and speed to the person 14. The information processing device 11 analyzes the point cloud data and obtains the activity level of the moving person 14. This is one example of the "first analysis" in the claims. The specific process for obtaining the activity level will be described later.

[0021] Immediately after the radar device 13 emits a radio signal such as a millimeter-wave radar, the information processing device 11 determines whether or not point cloud data corresponding to the person 14 has been generated. If point cloud data has been generated, it acquires the point cloud data, performs analysis, and executes a process to obtain the activity level.

[0022] If point cloud data has not been generated, the information processing device 11 determines that the movement of the person 14 is too small to acquire point cloud data, and performs a process to acquire a signal waveform by acquiring a weak signal originating from vital signs using a method such as reflected signal analysis (Doppler analysis) and processing the weak signal.

[0023] The information processing device 11 measures vital signs such as heart rate and respiratory rate of person 14 by analyzing the acquired signal waveform. This is one example of the "second analysis" in the claims. The specific processing for measuring vital signs will be described later.

[0024] Furthermore, the information processing device 11 determines whether the person 14 is a person or not by performing an analysis that applies a frequency filter to the signal waveform. The analysis will be described later. In Figure 1, the object to be measured is a person 14, but the object to be measured is not limited to humans, and may be a living organism other than a human, or an object that does not have vital signs, such as an electronic device.

[0025] Figure 2 is an example of a functional block diagram showing the functional configuration of an information processing device 11 according to an embodiment. The information processing device 11 includes a communication unit 20, a control unit 21, and a storage unit 22.

[0026] The communication unit 20 communicates with other devices. This is achieved, for example, by a communication interface such as a network interface card (NIC).

[0027] The control unit 21 controls the entire processing of the information processing device 11. The control unit 21 receives point cloud data of the object to be measured that exists in the measurement space from the radar device 13. The control unit 21 performs analysis processing on the received point cloud data. In this embodiment, when the object to be measured is moving, the control unit 21 acquires point cloud data generated at the location of the object to be measured for each frame. The control unit 21 calculates the centroid position of the point cloud for each frame using a method such as clustering. The control unit 21 determines whether the object is the same based on the centroid position of the point cloud in the current frame and the next frame, and tracks the centroid position of the point cloud. The determination of whether it is the same object may be made by, for example, determining that it is the same object if the distance between frames of the centroid position of the point cloud is less than or equal to a predetermined threshold. The control unit 21 tracks the centroid position of the point cloud for each object until point cloud data can no longer be acquired, and acquires the activity amount from the accumulated amount of movement. The method for accumulating the amount of movement and acquiring the activity amount will be described later.

[0028] The storage unit 22 stores various data or various programs executed by the control unit. The storage unit 22 is implemented by a main memory device such as Random Access Memory (RAM) or an auxiliary memory device such as a Hard Disk Drive (HDD). The storage unit 22 includes a correspondence DB 220, a frequency filter 221, a movement amount DB 222, and signal waveform data 223.

[0029] The Correspondence Relationship DB220 is a table that stores the correspondence between activity levels and corresponding vital ranges. Activity level refers to the magnitude of an object's movement, determined from at least one of the time or distance it travels, such as the maximum movement of the object being measured per unit time. In this embodiment, the maximum movement per minute is used as the activity level. The vital range is the range of vital values ​​corresponding to a person's activity level. The Correspondence Relationship DB220 may be modified by the user as appropriate based on medical knowledge and data. Details of the Correspondence Relationship DB220 will be described later with reference to Figure 8.

[0030] The frequency filter 221 is a filter used to extract waveforms in the frequency range corresponding to the vital range in the correspondence DB 220, and is generated by the control unit 21. In this embodiment, the frequency filter 221 is a bandpass filter that allows only frequency components in a specific range to pass through and attenuates low-frequency or high-frequency components. The control unit 21 generates each frequency filter 221 corresponding to the correspondence DB 220.

[0031] The Movement DB222 is a table that stores the accumulated value of the movement of the object being measured over a predetermined time period. The Movement DB222 will be described later using Figure 3.

[0032] The signal waveform data 223 is time-series data obtained by analyzing weak signals originating from vital signs. This analysis, for example, is a reflected signal analysis (Doppler analysis) performed by the control unit 21 on a stationary object being measured. Examples of the signal waveform data 223 are shown in Figures 9 and 10.

[0033] The control unit 21 applies a frequency filter 221 to the acquired signal waveform data 223 and performs analysis to determine whether the object being measured is a person or not. The analysis in this embodiment is as follows, for example: The control unit 21 reads out the frequency filter 221 and the signal waveform data 223 stored in the memory unit 22. The control unit 21 applies the frequency filter 221 to the signal waveform data 223 and extracts waveforms in the frequency range corresponding to the vital range. Subsequently, the control unit 21 determines whether the extracted waveform has an amplitude greater than or equal to a predetermined threshold, and whether the period, which is the time interval of the peak value of the waveform, corresponds to the heart rate range identified by the correspondence DB 220. The specific determination process will be described later.

[0034] Figure 3 shows an example of the movement amount DB222 according to the embodiment. The movement amount DB222 defines the object, the amount of movement (m), the start time of accumulation, and the end time of accumulation.

[0035] An object is a collection of point clouds that exist within the measurement space. In this embodiment, the movement amount DB222 assumes that there are three types of objects: "A", "B", and "C". The control unit 21 similarly generates movement amount data for each object even if there are four or more types of objects in the measurement space.

[0036] The amount of movement (m) represents the distance the object has moved during the cumulative time, and is acquired by the control unit 21 by tracking the centroid position of the point cloud. The cumulative time is the time taken to accumulate the amount of movement. In this embodiment, the maximum amount of movement per minute is used as the activity amount, so the accumulation time for the amount of movement is set to 60 seconds.

[0037] The start time of accumulation is the time when the accumulation of movement amounts begins, and the end time of accumulation is the time when the accumulation of movement amounts ends.

[0038] The control unit 21 generates records for all patterns of the accumulated distance (m) over a 60-second period. For example, suppose object "A" moves within the measurement space for 62 seconds, from 12:00:00 to 12:01:02. In this case, there are three patterns for the accumulation over the 60-second period: 12:00:00 to 12:01:00, 12:00:01 to 12:01:01, and 12:00:02 to 12:01:02. The control unit 21 generates records of the distance moved by object "A" for all three patterns.

[0039] The control unit 21 generates records of the amount of movement for object "A". Specifically, the control unit 21 generates records of the accumulated amount of movement for object "A" over a 60-second period for all patterns, such as 38(m) for the period from 12:00:00 to 12:01:00, 40(m) for the period from 12:00:01 to 12:01:01, and 39(m) for the period from 12:00:02 to 12:01:02.

[0040] The control unit 21 generates records of the amount of movement for object "B". Specifically, the control unit 21 generates records of the accumulated amount of movement over 60 seconds for object "B", similar to object "A" as described above, for all patterns, such as the amount of movement from 12:00:00 to 12:01:00 being 39 (m), the amount of movement from 12:00:01 to 12:01:01 being 40 (m), and the amount of movement from 12:00:02 to 12:01:02 being 41 (m).

[0041] The control unit 21 generates a record of the amount of movement for object "C". The amount of movement for object "C" has only been accumulated for 30 seconds, from 12:00:00 to 12:00:30, and has not reached the 60-second accumulation time. In this case, the control unit 21 generates 12(m), which is the accumulated value of the amount of movement for the 30 seconds from 12:00:00 to 12:00:30, as the record for object "C".

[0042] Here, the time of the first frame in which point cloud data is acquired is defined as 2220, the time when the process of accumulating the movement of objects begins (hereinafter referred to as the "accumulation process start time"). In the movement DB222, the accumulation process start time for object "A" is 12:00:00 (2220a), the accumulation process start time for object "B" is 12:00:00 (2220b), and the accumulation process start time for object "C" is 12:00:00 (2220c).

[0043] Furthermore, the time of the last frame in which point cloud data was acquired is defined as the time 2221 (hereinafter referred to as "completion time") when the process of accumulating the movement of objects is completed. In the movement DB222, the completion time of the accumulation process for object "A" is 12:01:02 (2221a), the completion time of the accumulation process for object "B" is 12:01:02 (2221b), and the completion time of the accumulation process for object "C" is 12:00:30 (2221c).

[0044] Figure 4 is an example of a functional block diagram showing the functional configuration of a radar device 13 according to this embodiment. In this embodiment, the radar device 13 uses a frequency-modulated continuous wave (FMCW) millimeter-wave radar. The FMCW method is a method that transmits a continuous wave of a certain frequency while frequency modulating it over a certain period of time, receives the reflected wave that has been reflected back from the target object, and performs frequency analysis using methods such as FFT (Fast Fourier Transform) to acquire time-series data such as the position and speed of movement of the target object. The radar device 13 has a communication unit 40, a control unit 41, a transmission unit 42, and a reception unit 43.

[0045] The communication unit 40 communicates with other devices. In this embodiment, the communication unit 40 communicates with the information processing device 11. The communication unit 40 is implemented by a communication interface such as a network interface card (NIC).

[0046] The control unit 41 controls the entire processing of the radar device 13. The control unit 41 performs frequency analysis on the transmitted radio wave signal and the received reflected radio wave signal using methods such as FFT to generate point cloud data of the object to be measured. The control unit 41 transmits the generated point cloud data to the information processing device 11 via the communication unit 40. The control unit 41 is implemented, for example, by a processor having a central processing unit (CPU) and RAM.

[0047] The transmitting unit 42 receives control from the control unit 41 and transmits radio signals such as radar signals. The transmitting unit 42 is implemented, for example, by an antenna or a duplexer.

[0048] The receiving unit 43 receives the reflected radio wave signal. The receiving unit 43 is implemented, for example, by an antenna or a duplexer.

[0049] Figure 5 is a diagram illustrating the overview of the tracking process for the object being measured. The radar device 13 generates point cloud data of the location of the moving object for each frame. In this embodiment, the radar device 13 irradiates the measurement space 50 with a radio wave signal and receives the reflected radio wave signal. The radar device 13 generates point cloud data for each frame, which is information about the set of reflection points corresponding to the object being measured.

[0050] Radar-based point cloud data generation involves, for example, using radar to detect an object and acquire point cloud data, performing a interpolation process based on the time distance between the frame containing the point cloud data and the current frame to generate feature data, and then using a neural network to generate keypoint data based on the feature data.

[0051] The information processing device 11 clusters the point clouds by their positional relationships for each frame and determines the centroid for each object. In Figure 5, there are two sets of point clouds, which are designated as object A and object B. The centroid position of the point cloud of object A obtained by clustering is shown by rectangle 51, and the centroid position of the point cloud of object B is shown by triangle 52.

[0052] The information processing device 11 tracks the centroid position of the point cloud of the object being measured by repeatedly determining whether the centroid points are the same based on the distance between frames and performing this operation over time. For example, the determination of whether they are the same object may be made by determining that they are the same object if the distance between frames of the centroid positions of the point cloud is less than or equal to a predetermined threshold.

[0053] The information processing device 11 performs tracking, and if point cloud data is no longer generated, it determines that the movement of the person 14 has become too small to acquire point cloud data, and switches to an analysis process that acquires a signal waveform by acquiring a weak signal derived from vital signs and processing the weak signal.

[0054] Figure 6 shows the object to be measured before the activity according to the embodiment. In this embodiment, a radar device 13, a curtain 60, and a person 61 are present in the measurement space 50.

[0055] Figure 7 shows the object to be measured after the activity according to the embodiment. The curtain before the activity is shown as 60a, and the curtain after the activity is shown as 60b. The person before the activity is shown as 61a, and the person after the activity is shown as 61b. Assume that the curtain 60a sways due to factors such as wind and moves along trajectory 70 to the position of curtain 60b. Assume that the person 61a moves from the position of person 61a to the position of person 61b by moving, such as walking or running, and following trajectory 71.

[0056] In Figure 6, a curtain is used as an example of an object other than a person to be measured, but other objects such as the leaves of a houseplant or electronic devices moving within the measurement space may also be used. Furthermore, in this embodiment, there are two objects to be measured: the curtain 60 and the person 61, but there may be three or more objects in the measurement space 50.

[0057] The radar device 13 emits radio waves, such as millimeter-wave radar signals, into the measurement space 50 and receives the reflected signals. The radar device 13 performs frequency analysis on the transmitted and received radio waves using techniques such as FFT (Fast Fourier Transform) to generate point cloud data of the curtain 60 and person 61 present in the measurement space 50, and transmits the point cloud data to the information processing device 11.

[0058] The information processing device 11 tracks the centroid position of the point cloud data for both the curtain 60 and the person 61 for each frame and performs integration processing of the amount of movement. In this embodiment, the point cloud detected from the position corresponding to the curtain 60 corresponds to the curtain 60, and the point cloud detected from the position corresponding to the person 61 corresponds to the person 61.

[0059] The control unit 21 proceeds to vital sign measurement processing of the object to be measured if point cloud data acquisition ceases. However, the conditions for proceeding to vital sign measurement processing are not limited to these. In this embodiment, the control unit 21 uses the time of the latest frame in which point cloud data was acquired as the end time of the integration process, and uses the centroid position of the point cloud at the end time of the integration process as a candidate for the position of the object to be measured (hereinafter referred to as "position candidate").

[0060] Figure 8 shows an example of a DB220 showing the correspondence between activity level and corresponding vital sign ranges in this embodiment. In this embodiment, the vital sign range is the heart rate range, but other vital sign information may be used. The correspondence DB220 is a table consisting of the maximum movement per minute (m) used as the activity level and the heart rate range (bpm) associated with the activity level.

[0061] If the maximum movement of the object being measured per minute is less than 24 m, the heart rate range after activity will be 85-110 bpm. If the maximum movement of the object being measured per minute is 24 m or more but less than 48 m, the heart rate range after activity will be 110-125 bpm. If the maximum movement of the object being measured per minute is 48 m or more, the heart rate range after activity will be 125-150 bpm.

[0062] In Figure 8, the values ​​and units for the maximum movement per minute and heart rate range are merely examples and may be designed to be changed by the user as appropriate. In this embodiment, the activity level uses the maximum movement per minute, but the average movement per minute, the number of points per minute, the average movement speed per minute, etc., may also be used, or other types of indicators may be used as the activity level. Furthermore, two or more indicators may be used as the activity level.

[0063] Next, the process by which the control unit 21 acquires the activity amount will be explained. The control unit 21 acquires the activity amount from the movement amount DB 222. The control unit 21 acquires the maximum movement amount among all records included from the start time of the accumulation process 2220 to the end time of the accumulation process 2221 as the activity amount. From here on, the method of acquiring the activity amount will be explained using Figure 3, with object "A" being a curtain 60 and object "B" being a person 61.

[0064] The control unit 21 extracts the maximum value of the movement amount from all records generated for each object in the movement amount DB 222. For the movement amount of object "A", three records are generated: 38(m), 40(m), and 39(m). The control unit 21 acquires the maximum value of 40(m) as the activity amount of the curtain 60. For the movement amount of object "B", three records are generated: 39(m), 40(m), and 41(m). The control unit 21 acquires the maximum value of 41(m) as the activity amount of person 61.

[0065] Furthermore, although not present in this embodiment, if an object "C" with a cumulative movement time of less than 60 seconds exists within the measurement space 50, the control unit 21 acquires 12(m) as the activity amount of object "C". Note that if the cumulative movement time is less than 60 seconds, the control unit 21 may choose not to acquire the activity amount and instead associate it with the lowest vital range value in the correspondence DB 220. Also, it is not necessary to use a method that acquires the maximum movement amount per minute from all records. For example, if the maximum value is exceeded every second, the value of the maximum movement amount per minute may be updated, and if the maximum value has not been updated for 3 minutes or more, the maximum movement amount per minute may be acquired from the data of the most recent 3 minutes.

[0066] Next, we will explain the process of extracting waveforms corresponding to the vital range from the signal waveform. Figure 9 is a diagram illustrating an example of the process for the curtain 60, and Figure 10 is a diagram illustrating an example of the process for the person 61.

[0067] Figure 9(a) shows an example of a signal waveform received from the curtain 60 according to the embodiment. Signal waveform 90 is an example of visualizing the signal waveform data 223 received from the curtain 60 as a figure. Figure 9(b) shows an example of a waveform extracted by applying the frequency filter 221 to the signal waveform received from the curtain 60 according to the embodiment. In both Figure 9(a) and (b), the vertical axis is displacement and the horizontal axis is time (seconds).

[0068] The control unit 21 acquires the signal waveform 90 at curtain 60b, which is a candidate position for the curtain. The control unit 21 reads out the frequency filter 221 of the frequency range corresponding to the heart rate range identified from the activity level of curtain 60. As mentioned above, in this embodiment, curtain 60 is object "A" in the movement amount DB222 in Figure 3, and the activity level of curtain 60 is 40 (m). Therefore, the control unit 21 refers to the correspondence DB220 and identifies the heart rate range corresponding to the activity level as 110 to 125 (bpm).

[0069] The control unit 21 reads out a frequency filter 221 corresponding to the identified heart rate range 110-125 (bpm). The control unit 21 applies the frequency filter 221 to the signal waveform 90 to extract a waveform 91 having signal components in a specific frequency range.

[0070] Figure 10(a) shows an example of a signal waveform received from person 61 according to the embodiment. Signal waveform 100 is an example of visualizing the signal waveform data 223 received from person 61 as a figure. Figure 10(b) shows an example of a waveform extracted by applying a frequency filter 221 to the signal waveform received from person 61 according to the embodiment. In both Figure 10(a) and (b), the vertical axis is displacement and the horizontal axis is time (seconds).

[0071] The control unit 21 acquires the signal waveform 100 at person 61, which is a candidate for the person's position. The control unit 21 reads out the frequency filter 221 of the frequency range corresponding to the heart rate range identified from the activity level of person 61. In this embodiment, person 61 is object "B" in the movement amount DB222 in Figure 3, and the activity level of person 61 is 41 (m). Therefore, the control unit 21 refers to the correspondence DB220 and identifies the heart rate range corresponding to the activity level as 110 to 125 (bpm).

[0072] The control unit 21 reads out the frequency filter 221 corresponding to the heart rate range 110 to 125 (bpm). The control unit 21 applies the frequency filter 221 to the signal waveform 100 to extract the waveform 101 having signal components in a specific frequency range.

[0073] The following describes the determination process performed by the control unit 21 to determine whether the object to be measured in the measurement space is a person or not. The control unit 21 performs a first determination process regarding the amplitude of the waveform extracted by applying a frequency filter. The control unit 21 determines whether the extracted waveform has an amplitude greater than or equal to a predetermined threshold.

[0074] In this embodiment, the control unit 21 detects all local maximums and local minimums (hereinafter, the two together will be referred to as "extreme values") of the extracted waveforms 91 and 101. A local maximum is the largest value within the range including that value, and a local minimum is the smallest value within the range including that value. In this embodiment, the range including the local maximum is defined as one period of the waveform, but it may be a different range.

[0075] The maximum value in waveform 91 is shown as maximum value 92, and the minimum value in waveform 91 is shown as minimum value 93. Similarly, the maximum value in waveform 101 is shown as maximum value 102, and the minimum value in waveform 101 is shown as minimum value 103.

[0076] The control unit 21 calculates the difference in displacement between the median and the maximum value, and the difference in displacement between the median and the minimum value. The median is the value that is in the middle when the displacement values ​​are arranged in ascending order. In Figure 9(b), waveform 91 shows the median displacement as M1 with a dashed line, and in Figure 10(b), waveform 101 shows the median displacement as M2 with a dashed line.

[0077] The control unit 21 calculates the difference in displacement between the median value M1 and the extreme value in waveform 91, and calculates the difference in displacement between the median value M2 and the extreme value in waveform 101. Then, the control unit 21 adds up all the calculated difference in displacement between the median and the extreme values, divides by the number of detected extreme values, and multiplies by 2 to calculate the average amplitude. If the calculated average amplitude is less than a predetermined threshold, the control unit 21 terminates the determination process.

[0078] If the average value of the calculated amplitude is greater than or equal to a predetermined threshold, the control unit 21 performs a determination process to determine whether the periodicity described below corresponds to the heart rate range. For example, the control unit 21 calculates the periodicity of the waveform by calculating all the time intervals of the maximum values ​​of the extracted waveform and calculating the average value of the multiple calculated time intervals.

[0079] Specifically, the control unit 21 calculates the time interval between the detected maximum value and the next occurrence of the maximum value in the time series. Note that the periodicity of the waveform may be calculated using the minimum value, or using both the maximum and minimum values. Alternatively, a method that does not use extreme values ​​may be used; for example, frequency analysis may be performed using a Fourier transform or similar technique to identify the frequency component with the strongest frequency spectrum intensity, and the periodicity of the waveform may be calculated from its reciprocal.

[0080] If the periodicity of the calculated waveform does not correspond to the heart rate range, the control unit 21 determines that the object being measured is not a person and terminates the process.

[0081] The control unit 21 determines that the object being measured is a person if the periodicity of the calculated waveform corresponds to the heart rate range, and continues to measure the person's vital signs. The control unit 21 outputs candidate locations for the person as the person's location and continues to output heart rate data by methods such as counting the number of peaks per minute.

[0082] So far, we have explained the determination process using extreme values, but other determination methods may also be used. For example, a periodic trigonometric function may be fitted, and the amplitude or periodicity of the waveform may be determined using indicators such as the degree of fitting or error.

[0083] [flowchart] Figure 11 is an example of a flowchart illustrating the flow of the activity level acquisition process according to the embodiment. The radar device 13 irradiates the measurement space with radio wave signals such as millimeter-wave radar and acquires the radio wave signals (S100).

[0084] The control unit 21 determines whether or not point cloud data generated around the position of the moving object can be acquired (S101). If point cloud data can be acquired (S101Yes), the amount of movement is accumulated by tracking the centroid position of the point cloud until point cloud data of the object to be measured can no longer be acquired (S102). If point cloud data cannot be acquired (S101No), the process proceeds to step S107 shown in Figure 12.

[0085] If the accumulated movement amount is 60 seconds or more (S103Yes), the control unit 21 generates a record of the movement amount for all patterns (S104). If the accumulated movement amount is not 60 seconds or more (S103No), the control unit 21 accumulates the movement amount from the start time of the accumulation process 2220 to the end time of the accumulation process 2221 and generates one record of the movement amount (S105).

[0086] The control unit 21 obtains the maximum value of the movement amount from the generated movement amount DB222 among the records of the same object as the activity amount. After obtaining the activity amount, the system proceeds to step S107 (S106) shown in Figure 12.

[0087] Figure 12 is an example of a flowchart illustrating the signal waveform analysis process according to the embodiment. The control unit 21 acquires the signal waveform at the candidate location of the object to be measured (S107). The control unit 21 reads the correspondence DB 220 and identifies the vital range corresponding to the acquired activity level (S108). The control unit 21 reads the frequency filter 221 corresponding to the identified vital range (S109). The control unit 21 performs a person determination process to determine whether the object to be measured is a person or not using the waveform extracted by applying the frequency filter 221 (S200). The flowchart for the person determination process will be described later with reference to Figure 13.

[0088] If the control unit 21 determines that the object being measured is a person (S110Yes), it outputs a candidate for the person's location as the location, and also outputs the person's vital sign, the heart rate (S111). The control unit 21 may also output and display the person's location and vital signs on a monitor connected to the information processing device 11. The control unit 21 continues to output heart rate data by methods such as counting the number of peaks per minute (S112). If the control unit 21 determines that the object being measured is not a person (S110No), it terminates the process.

[0089] [Flowchart for Person Identification Process] Figure 13 is an example of a flowchart illustrating the flow of the person determination process according to the embodiment. Referring to Figure 13, the process flow in which the information processing device 11 determines whether or not the object to be measured is a person will be explained.

[0090] The control unit 21 applies the readout frequency filter 221 to the signal waveform acquired by methods such as reflected signal analysis, and extracts waveforms having components in a specific frequency range (S201). The control unit 21 performs a first determination process regarding the amplitude of the waveform extracted by applying the frequency filter (S202). The control unit 21 determines whether the extracted waveform has an average amplitude greater than or equal to a predetermined threshold.

[0091] If the average value of the amplitude of the extracted waveform is less than the threshold (S202No), the control unit 21 determines that the object being measured is not a person (S205) and terminates the person determination process. If the average value of the amplitude of the extracted waveform is greater than or equal to the threshold (S202Yes), the control unit 21 proceeds to the second determination process.

[0092] Next, the control unit 21 performs a second determination process regarding periodicity for waveforms that have an average value of amplitudes above a threshold (S203). The control unit 21 calculates the time interval of the waveform's maximum value and calculates the waveform's periodicity by taking the average of the multiple calculated time intervals. If the periodicity of the extracted waveform does not correspond to the heart rate range (S203No), the control unit 21 determines that the object being measured is not a person (S205) and terminates the person determination process. If the periodicity of the extracted waveform corresponds to the heart rate range (S203Yes), the control unit 21 determines that the object being measured is a person (S204) and terminates the person determination process.

[0093] The flow of the person detection process is merely an example, and the order of the first detection process, which determines whether the average value of the amplitude is above a threshold, and the second detection process, which determines whether the periodicity of the waveform corresponds to the heart rate range, may be reversed. Alternatively, only one of either amplitude or periodicity may be checked.

[0094] As explained above, the information processing device 11 acquires the activity level of an object moving within the measurement space by analyzing a first radio wave signal. When the activity level of an object is no longer acquired, the information processing device 11 acquires the signal waveform of the object by analyzing a second radio wave signal that detects smaller movements than those detected by the analysis of the first radio wave signal. Based on the correspondence between the activity level of an organism and the vital range of the organism, which is stored in advance, the information processing device 11 identifies the vital range from the activity level of the object. The information processing device 11 extracts the waveform corresponding to the identified vital range from the signal waveform. If the waveform satisfies predetermined conditions, the information processing device 11 continues the analysis of the second radio wave signal for the object. As a result, the information processing device 11 performs conditional judgment on the waveform acquired from the object, preventing it from continuing the analysis of radio waves for objects that do not meet the conditions. Therefore, even if objects other than people are present in the measurement space, people can be identified, thereby improving the accuracy of vital sign measurement for people.

[0095] Furthermore, according to this embodiment, the information processing device 11 may apply a filter with a frequency range corresponding to the vital range to the signal waveform and extract the waveform of the component corresponding to the frequency range. As a result, when the information processing device 11 determines whether or not a person is present, it uses the waveform of the component of the frequency range corresponding to the vital range rather than making a determination directly from the signal waveform, thereby improving the accuracy of the determination.

[0096] Furthermore, according to the embodiment, the information processing device 11 may calculate the activity level of an object using at least one of the following: average movement per unit time, maximum movement per unit time, point cloud volume per unit time, and average movement speed per unit time. This allows the information processing device 11 to quantitatively evaluate the activity level of an object and to identify the corresponding vital range from the activity level.

[0097] Furthermore, according to the embodiment, the information processing device 11 may satisfy the above condition if at least one of the following conditions is met: the waveform displacement is above a predetermined threshold, or the periodicity calculated based on the waveform corresponds to the vital range. This allows the information processing device 11 to determine whether the acquired waveform is obtained from a person or from an object other than a person, thereby improving the accuracy of measuring a person's vital signs.

[0098] Furthermore, according to the embodiment, the information processing device 11 may calculate periodicity based on the time interval of the extreme values ​​of the waveform. Alternatively, the information processing device 11 may perform frequency analysis on the waveform and calculate the value of the frequency with the strongest frequency spectral intensity as the periodicity. This allows the information processing device 11 to quantitatively evaluate the periodicity of the waveform and determine whether the acquired waveform is obtained from a person or from an object other than a person, thereby improving the accuracy of vital sign measurement of a person.

[0099] Furthermore, according to the embodiment, the information processing device 11 may perform the following processes: analyze the first radio wave signal to acquire point cloud data generated at the location of an object moving within the measurement space; calculate the centroid position of the point cloud based on the positional relationship of the point cloud; track the centroid position of the point cloud in a time series; and acquire the activity level based on the trajectory obtained by the tracking. This makes it possible to quantitatively evaluate the activity level and to identify the corresponding vital range from the activity level.

[0100] [Hardware configuration of the information processing device 11] Figure 14 is a diagram illustrating an example of the hardware configuration of an information processing device according to an embodiment. As an example, the information processing device 11 includes a CPU 140, RAM 141, input / output interface 142, communication interface 143, and HDD 144 as its components, and these components are connected via a bus 145.

[0101] The CPU 140 is a processor that operates the processes that perform each function of the control unit 21. The CPU 140 may have one or more processor cores. The information processing device 11 may include processors other than the CPU, and may include multiple types of processors.

[0102] RAM 141 operates as the main memory of the information processing unit 11 and stores programs read from auxiliary storage devices such as HDD 144 and data used to execute programs. The information processing unit 11 may have other memory besides RAM, and may have multiple memory sources.

[0103] The input / output interface 142 is an interface that inputs signals to the information processing device 11 and outputs signals from the information processing device 11. For example, the input / output interface 142 receives signals from input devices such as a keyboard or mouse connected to the information processing device 11. Multiple input and output devices may be connected to the information processing device 11 via the input / output interface 142.

[0104] The communication interface 143 is an interface for connecting the information processing device 11 to a network. For example, the standard for the communication interface 143 may be a wired LAN communication standard such as Ethernet (registered trademark), a wireless LAN communication standard such as WiFi (registered trademark), or a mobile communication wireless standard such as Local 5G.

[0105] The HDD144 operates as an auxiliary storage device for storing programs and data used by the operating system (OS) and control unit 21 of the information processing device 11 that perform the functions indicated by the OS and control unit 21. The information processing device 11 may also be equipped with auxiliary storage devices other than the HDD, such as a Solid State Drive (SSD), and may be equipped with multiple auxiliary storage devices.

[0106] Although embodiments of the present invention have been described above, the embodiments of the present invention are not limited to those described herein. The present invention may be implemented in various different forms other than those described above. The processing procedures, processing methods, names of each part, and various data shown in the embodiments and drawings are merely examples and may be changed as appropriate unless otherwise specified.

[0107] The functional and hardware configurations shown in the examples and drawings are merely examples and do not necessarily have to be configured or arranged exactly as shown. Unless otherwise specified, they may be changed as needed. For example, the components of the functional and hardware configurations may be distributed or integrated into any unit, such as a function.

[0108] The information processing program according to this embodiment is not limited to being executed by the information processing device 10. For example, the information processing program according to this embodiment may be executed across multiple devices.

[0109] Furthermore, the information processing program according to the embodiment may be distributed via a network such as the Internet. Also, the information processing program according to the embodiment may be recorded on a computer-readable recording medium and sold. Examples of computer-readable recording media include Compact Disc Read-only memory (CD-ROM), Digital Versatile Disc (DVD), USB memory, floppy disk, and Magneto-Optical Disk (MO). The information processing program according to the embodiment may be read from a computer-readable recording medium by a computer and executed by the computer. [Explanation of Symbols]

[0110] 10. Vital signs measurement system 11. Information Processing Device 12 Networks 13 Radar equipment 14 People 20 Communications Department 21 Control Unit 22 Memory section 220 Correspondence Database 221 Frequency Filter 222 Movement amount DB 223 Signal waveform data

Claims

1. By first analyzing the radio wave signals corresponding to an object moving within the measurement space, the activity level of the object is obtained. A second analysis, which detects smaller movements than those detected in the first analysis, is performed to obtain the signal waveform of the object from the radio signal. By referring to the pre-stored correspondence between activity levels and vital ranges, the vital range corresponding to the acquired activity level is identified. The waveform corresponding to the identified vital range is extracted from the signal waveform. If the extracted waveform satisfies predetermined conditions, the second analysis of the radio wave signal is continued. An information processing program characterized by having a computer perform the processing.

2. The second analysis described above is performed when the activity level of the object is no longer acquired. The information processing program according to claim 1.

3. The extraction process involves applying a filter of the frequency range corresponding to the vital range to the signal waveform and extracting the waveform of the component corresponding to the frequency range. The information processing program according to claim 1.

4. The activity level of the object is calculated using at least one of the following: average movement per unit time, maximum movement per unit time, point cloud volume per unit time, and average movement speed per unit time. The information processing program according to claim 1.

5. The above conditions are met if at least one of the following is satisfied: the amplitude of the waveform is greater than or equal to a predetermined threshold, or the periodicity calculated based on the waveform corresponds to the vital range. The information processing program according to claim 1.

6. The periodicity is calculated based on the time interval of the extreme values ​​of the waveform. The information processing program according to claim 5.

7. A frequency analysis is performed on the waveform, and the value of the frequency with the strongest frequency spectral intensity is calculated as the periodicity. The information processing program according to claim 5.

8. The first analysis involves acquiring point cloud data generated at the location of an object moving within the measurement space, calculating the centroid position of the point cloud based on the positional relationship of the point cloud, tracking the centroid position of the point cloud over time, and acquiring the activity level based on the trajectory obtained from the tracking. The information processing program according to claim 1.

9. By first analyzing the radio wave signals corresponding to an object moving within the measurement space, the activity level of the object is obtained. A second analysis, which detects smaller movements than those detected in the first analysis, is performed to obtain the signal waveform of the object from the radio signal. By referring to the pre-stored correspondence between activity levels and vital ranges, the vital range corresponding to the acquired activity level is identified. The waveform corresponding to the identified vital range is extracted from the signal waveform. If the extracted waveform satisfies predetermined conditions, the second analysis of the radio wave signal is continued. An information processing method characterized by having a computer perform the processing.

10. By first analyzing the radio wave signals corresponding to an object moving within the measurement space, the activity level of the object is obtained. A second analysis, which detects smaller movements than those detected in the first analysis, is performed to obtain the signal waveform of the object from the radio signal. By referring to the pre-stored correspondence between activity levels and vital ranges, the vital range corresponding to the acquired activity level is identified. The waveform corresponding to the identified vital range is extracted from the signal waveform. If the extracted waveform satisfies predetermined conditions, the second analysis of the radio wave signal is continued. An information processing device having a control unit.

Images