Living body detection device and living body detection method

Abstract

A living body detection device is for acquiring living body information of a target occupant, and comprises a control unit (11) for acquiring living body information by using a radio wave sensor configured to have a variable directivity. The control unit acquires position information of a reference site of the target occupant on the basis of data received from the radio wave sensor or from another on-board sensor different from the radio wave sensor, and adjusts the orientation of the directivity of the radio wave sensor according to the position information of the reference site.

Description

Living body detection device and living body detection method CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2024-106880 filed in Japan on July 2, 2024, and the contents of the original application are incorporated by reference in their entirety.

[0002] The disclosure in this specification relates to a living body detection device and a living body detection method.

[0003] Patent document 1 discloses a biometric information acquisition device that has a radio wave sensor that measures the distance to a person by sending and receiving radio waves, and acquires biometric information with high accuracy by appropriately adjusting the output value of the radio wave sensor according to the distance.

[0004] Japanese Patent Application Laid-Open No. 2022-116583

[0005] Although Patent Document 1 adjusts output based on distance, it does not control the direction of the directivity of the radio wave sensor. Therefore, depending on the posture of the target occupant, there is a possibility that radio waves reflected from parts other than the intended part may be received. Furthermore, in an environment with many reflecting objects, such as the inside of a vehicle (a so-called multipath environment), the transmitted radio waves may be reflected by various objects and received by the radio wave sensor. In other words, objects other than the target occupant may also be detected. Therefore, the detection accuracy of biometric information may be degraded.

[0006] An object of the present disclosure is to provide a living body detection device and a living body detection method that can improve the detection accuracy of biological information.

[0007] The biometric detection device disclosed herein is a biometric detection device that acquires biometric information of a target occupant, and is equipped with a control unit that acquires the biometric information using a radio wave sensor configured with variable directivity, and the control unit acquires position information of a reference part of the target occupant based on data received from the radio wave sensor or another vehicle sensor different from the radio wave sensor, and adjusts the direction of the directivity of the radio wave sensor according to the position information of the reference part.

[0008] The biometric detection method disclosed herein is a biometric detection method executed by a processor for acquiring biometric information of a target occupant, and includes: acquiring position information of a reference part of the target occupant based on data received from a radio wave sensor configured with variable directivity or another vehicle-mounted sensor other than the radio wave sensor; adjusting the direction of the directivity of the radio wave sensor according to the position information of the reference part; and acquiring the biometric information based on the output data of the radio wave sensor.

[0009] According to these methods, the directivity of the radio wave sensor is determined according to the position of the reference part. This allows the directivity to be in the appropriate direction. Therefore, the influence of radio waves reflected from directions other than the desired direction can be reduced. Therefore, it is possible to improve the detection accuracy of biological information.

[0010] The various aspects disclosed in this specification employ different technical means to achieve their respective objectives. The reference numerals in parentheses in the claims and in this section are intended to exemplify correspondences with the following embodiments and are not intended to limit the technical scope. The objectives, features, and advantages disclosed in this specification will become more apparent by reference to the following detailed description and the accompanying drawings.

[0011] It is a diagram showing a living body detection system. It is a diagram of a frequency spectrum. It is a diagram of a frequency spectrum. It is a diagram for explaining a method of acquiring a target position. It is a diagram for explaining a method of acquiring a target position. It is a flowchart showing an example of processing by a control unit. It is a diagram showing the configuration of an antenna.

[0012] In the following description, components having the same function may be given the same reference numerals, and a detailed description thereof may be omitted. Also, components having the same function may be given the same or similar names, and a detailed description thereof may be omitted. When only a portion of the configuration is mentioned, the description given elsewhere may apply to the other portions.

[0013] 1 is a diagram showing an example of a schematic configuration of a living body detection system 100 including a living body detection device 1. As shown in FIG.

[0014] The living body detection device 1 is mounted on a vehicle and used. The living body detection device 1 is a device that acquires the biological information of a target occupant. In other words, the living body detection device 1 is a device that measures the biological information of the target occupant. The target occupant is an occupant of the vehicle. Hereinafter, the occupant to be measured will also be referred to as the target occupant. The target occupant may be a person sitting in a specific seat. In this disclosure, the seat that defines the target occupant will also be referred to as the target seat. Note that the living body to be measured is not limited to a human being, but may also be an animal (pet) such as a dog or cat. In the following description, the term "target occupant" may be replaced with "target body" or "measurement target."

[0015] In this embodiment, the biological information acquired by the biological detection device 1 is the respiratory rate. The biological detection device 1 may acquire biological information other than the respiratory rate. The biological detection device 1 may be configured to measure the heart rate instead of the respiratory rate, or may be configured to measure both the respiratory rate and the heart rate. The respiratory rate here refers to the number of breaths taken per certain period of time, and may be rephrased as the respiratory cycle or the respiratory rate. The heart rate also refers to the number of heartbeats per certain period of time, and may be rephrased as the pulse cycle or the pulse rate. The biological detection device 1 may acquire pulse rate, body temperature, etc. as biological information.

[0016] The living body detection device 1 is used to detect a decrease in the driver's level of alertness (i.e., the degree of drowsiness) or an abnormal physical condition (a so-called dead man). The living body detection device 1 may be used to detect whether an infant has been left behind. The living body detection device 1 may be used to determine whether an occupant is an infant or not. The living body detection device 1 may record changes in the physical condition of an occupant and be used for health support.

[0017] <System Configuration Including the Living Body Detection Device> As shown in FIG. 1, the living body detection device 1 is connected to a radio wave sensor 2, a camera 3, and an ADASECU 4 when in use.

[0018] <Radio Wave Sensor> The radio wave sensor 2 is a sensor that generates and outputs data related to the biometric information of a target occupant by transmitting and receiving radio waves in a predetermined frequency band. The radio wave sensor 2 is configured to have variable directivity. The radio wave sensor 2 is disposed in a position where it can detect the biometric information of all occupants in the vehicle cabin. The radio wave sensor 2 is disposed on the ceiling or the like inside the vehicle. Here, as an example, the radio wave sensor 2 is attached to the rearview mirror. The radio wave sensor 2 may also be disposed in any other position, such as near the overhead console. The radio wave sensor 2 may be disposed on the ceiling along the center line of the vehicle body extending in the fore-and-aft direction of the vehicle, or may be disposed in a position offset to the left or right. The radio wave sensor 2 may be attached to any position on the ceiling. Furthermore, the radio wave sensor 2 is not limited to being disposed on the ceiling, but may also be disposed on the instrument panel, pillar, or the like.

[0019] The radio wave sensor 2 is a millimeter-wave radar. As an example, the radio wave sensor 2 is a millimeter-wave radar using the FMCW (Frequency Modulated Continuous Wave) method. In other embodiments, the radio wave sensor 2 may be a Doppler millimeter-wave radar. The radio wave sensor 2 may be a pulse-type radar. For example, the radio wave sensor 2 may be a UWB (Ultra Wide Band) radar that uses impulse waves used in UWB communication. The frequency of the transmission wave of the radio wave sensor 2 is selected appropriately depending on the usage situation. The frequency of the transmission wave of the radio wave sensor 2 is referred to as the operating frequency below. As an example, the operating frequency is approximately 24 GHz. The frequency band of the transmission wave of the radio wave sensor 2 is referred to as the operating frequency band below.

[0020] The radio wave sensor 2 transmits radio waves (transmission waves) at an operating frequency toward the occupant. More preferably, the radio wave sensor 2 is mounted in an attitude and position that transmits radio waves toward a target region of the occupant. The target region here refers to a body region used to acquire biometric information. In this embodiment, the target region is the chest. Additionally or alternatively, the target region may be the abdomen. In the present disclosure, the position of the target region within the vehicle is also referred to as a target position T.

[0021] The radio wave sensor 2 receives reflected waves, which are radio waves reflected by the occupant. The radio wave sensor 2 transmits observation data obtained by receiving the reflected waves to the living body detection device 1. The observation data may be data indicating the reception strength of the reflected waves over time. In other words, the data indicating the reception strength of the reflected waves over time is waveform data of the reflected waves. The observation data may also include waveform data of the transmitted waves. The living body detection device 1 obtains the distance between the radio wave sensor 2 and the occupant as a distance measurement value based on the observation data input from the radio wave sensor 2. When the radio wave sensor 2 obtains reflected waves from the occupant's chest, the living body detection device 1 can obtain the distance between the radio wave sensor 2 and the chest as a distance measurement value.

[0022] The position of the occupant's chest can fluctuate by several millimeters as the occupant breathes. The respiratory cycle is approximately 4 seconds for adults and 0.8 seconds for infants. That is, the position of the chest can fluctuate in a cycle of several seconds. Therefore, the respiratory rate can be estimated by observing the change in the measured distance over time. Furthermore, the position of the occupant's chest can fluctuate by several tens of micrometers as the occupant's heartbeats. Therefore, the heart rate can also be estimated by observing the change in the measured distance over time. Note that the heart rate may be estimated from the interval at which the change in the frequency of the reflected wave (in other words, the Doppler shift) associated with heartbeats is observed.

[0023] The radio wave sensor 2 has a signal processing circuit 20, multiple antennas 21, and a switch 21s. The antenna 21 is an antenna for transmitting and receiving radio waves in the operating frequency band. The antenna 21 is connected to the signal processing circuit 20. The antenna 21 radiates a signal input from the signal processing circuit 20 as a radio wave. The antenna 21 inputs the received radio waves to the signal processing circuit 20.

[0024] The antenna 21 is a directional antenna. The multiple antennas 21 are arranged so that each antenna has a directivity (main beam) directed in a different direction. The switch 21s is a switch that turns on and off the electrical connection between the antenna 21 and the signal processing circuit 20. By switching the switch 21s on and off, it is possible to select the antenna 21 to be used for transmission and reception. In other words, the radio wave sensor 2 is configured so that the directivity can be changed by switching the switch 21s on and off.

[0025] The signal processing circuit 20 includes a transmitting circuit and a receiving circuit. The transmitting circuit is a circuit that generates a signal to be transmitted and outputs it to the antenna 21. The receiving circuit is a circuit that performs predetermined signal processing on the signal received by the antenna 21. The signal processing circuit 20 performs appropriate signal processing on the transmitted signal and the received signal, and transmits the signals to the processor 11. The signal processing circuit 20 selects the antenna 21 to be used for transmission and reception based on the signal transmitted from the biological detection device 1.

[0026] The signal processing circuit 20 may include a calculation unit that calculates biological information (heart rate, respiratory rate) based on the transmitted and received radio waves. The signal processing circuit 20 may transmit the calculated biological information to the processor 11.

[0027] <Camera> The camera 3 is a component that captures an image of a target occupant. The camera 3 is, for example, a stereo camera. The camera 3 may be a ToF (Time of Flight) camera. The camera 3 is included in the other on-board sensors described below. The camera 3 is disposed in a position where it can capture images of all occupants in the vehicle cabin. The camera 3 is disposed near the radio wave sensor 2. For convenience, one of the two cameras that make up the stereo camera will be referred to as the first camera, and the other as the second camera. The first camera and the second camera may be arranged side by side in the vehicle width direction, separated by a predetermined distance. The stereo camera as the camera 3 may have the first and second cameras on either side of the radio wave sensor 2. The camera 3 and the radio wave sensor 2 may be configured as an integrated sensor module.

[0028] The camera 3 may be disposed at any position on the ceiling. Here, as an example, the camera 3 is attached to the rearview mirror. The camera 3 may also be disposed at any position, such as near the overhead console. The camera 3 does not have to be disposed near the radio wave sensor 2. The camera 3 may be disposed in a location where it is easy to capture an image of the target occupant.

[0029] The image of the target occupant captured by the camera 3 is hereinafter referred to as a captured image. The camera 3 transmits captured image data to the living body detection device 1 as information for the processor 11 to identify the position of a predetermined part (hereinafter also referred to as a reference part) of the target occupant. The stereo camera serving as the camera 3 may transmit the captured images of each camera constituting the stereo camera, i.e., the image captured by the first camera and the image captured by the second camera, to the living body detection device 1.

[0030] The reference part may be any part of the target occupant. The reference part may be the top of the head, the eyes, the chin, the mouth, or the nose. In this embodiment, the reference part is the top of the head. In this embodiment, the processor 11 estimates the target position T based on the position of the reference part. In other words, the position information of the reference part is used by the living body detection device 1 to identify the target position T. In this embodiment, the reference part is a part different from the target part, but the reference part may be the same as the target part. The camera 3 may be configured to identify the target position T by image recognition and provide it to the processor 11.

[0031] <ADASECU> The ADASECU 4 is an ECU that executes control to assist the driver in driving operations. ADAS is an abbreviation for Advanced Driving Assistant System. ECU is an abbreviation for Electronic Control Unit. The ADASECU 4 may notify or warn the driver based on the biological information transmitted from the biological detection device 1. The biological detection device 1 is connected to the ADASECU 4, but the function thereof may be built into the ADASECU 4.

[0032] The living body detection device 1 may be connected to any other ECU. The living body detection device 1 may be mounted on any other ECU. The living body detection device 1 may be connected to a cockpit ECU. The cockpit ECU is an ECU that controls a meter device, a navigation device, an air conditioning device, etc.

[0033] <Biodetection Device> The biodetection device 1 has a processor 11, RAM 12, storage 13, and I / O 14. The processor 11 corresponds to a control unit. The processor 11 is an arithmetic core that performs arithmetic processing based on data received from the radio wave sensor 2 or the camera 3. The processor 11 may be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). RAM is an abbreviation for Random Access Memory.

[0034] The storage 13 is a rewritable non-volatile memory. The storage 13 may be realized by at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The storage 13 may include multiple types of storage media, such as a read-only memory (ROM) and a flash memory. The storage 13 stores a living body detection program to be executed by the processor 11. Execution of the living body detection program by the processor 11 corresponds to execution of a living body detection method.

[0035] The living body detection device 1 is connected to a radio wave sensor 2 and a camera 3. The camera 3 corresponds to another on-board sensor different from the radio wave sensor 2. Hereinafter, the other on-board sensor different from the radio wave sensor 2 may also be simply referred to as the other on-board sensor. Examples of the other on-board sensor include a weight sensor, an infrared sensor, and a seat belt sensor.

[0036] The weight sensor is a sensor that detects the weight of an occupant sitting in a seat. Infants are generally lighter than adults. Therefore, the processor 11 may identify the seat in which the occupant is present and whether the occupant is an infant or not from the output signal of the weight sensor for each seat. The infrared sensor is a sensor that detects the temperature and movement within the vehicle cabin. Infants typically have a higher body temperature and are more active than adults. Therefore, the processor 11 may identify whether the target occupant is an infant or not using such characteristics obtained by the infrared sensor. The seat belt sensor is a sensor that detects the fastening status of the seat belt. The processor 11 may estimate the seat in which the occupant is sitting from the output signal of the seat belt sensor. The other in-vehicle sensors may be sensors other than those described above.

[0037] The processor 11 acquires observation data from the signal processing circuit 20 and performs processing to calculate biological information based on the observation data. As described above, the observation data may be received waveform data of reflected waves. Based on the observation data, the processor 11 may generate data (hereinafter referred to as distance FFT data) by performing an FFT on an IF (intermediate frequency) signal obtained by mixing transmitted and received waves. The distance FFT data indicates the frequency spectrum of the IF signal and indirectly indicates the location of a reflecting object. The location of a reflecting object is, specifically, the distance to the reflecting object. If multiple reflecting objects are present, the observation data may be data with peaks at multiple frequencies. For convenience, the frequency at which a reflection point exists and a peak occurs in the frequency spectrum is also referred to as a peak detection point. In the distance FFT results, if the frequency at the peak detection point is f, the speed of light is c, and the distance is d, then the relationship d = f / 2c holds. Alternatively, the signal processing circuit 20 may generate distance FFT data, and the processor 11 may receive that data as observation data.

[0038] The processor 11 also acquires information for identifying the position of the reference part of the target occupant from the radio wave sensor 2 or another on-board sensor. In this embodiment, the processor 11 identifies the position of the reference part based on a captured image acquired from the camera 3. The captured image corresponds to data (information) received from the camera 3, which is another on-board device, for identifying the position of the reference part. In this way, the information for identifying the position of the reference part may be information that indirectly indicates the position of the reference part.

[0039] For example, the processor 11 may analyze the captured image to obtain the position coordinates of the reference part as seen from the camera 3. The processor 11 may identify the position coordinates of the reference part (here, the top of the head) from parallax information of two images captured by a stereo camera serving as the camera 3. The position information of the reference part can be used to identify the target position T.

[0040] Furthermore, the processor 11 may be configured to acquire physique information of the target occupant, specifically, shoulder width ("W" in FIG. 6) and sitting height ("H" in FIG. 6), by analyzing the captured image. For example, the processor 11 may acquire the shoulder width (W) and sitting height (H) by analyzing the captured image acquired from the camera 3. For example, the processor 11 may identify the position coordinates of the left and right shoulders from parallax information of two images captured by a stereo camera serving as the camera 3, and calculate the shoulder width. The processor 11 may also calculate the sitting height based on the position coordinates of the seat surface and the position coordinates of reference parts, which will be described later.

[0041] The processor 11 may acquire a camera-to-target distance (R), which is the distance from the camera 3 to the target location, using the reference location as a reference. The processor 11 may calculate a sensor-to-target distance, which is the distance from the radio wave sensor 2 to the target position T, from the camera-to-target distance. The processor 11 estimates the target position T using the camera 3. A detailed method for acquiring the target position T using the camera 3 will be described later.

[0042] Furthermore, the processor 11 adjusts the direction of the directivity of the radio wave sensor 2 in accordance with the acquired position information of the reference portion or the target position T. For example, the processor 11 estimates the target position T based on the position information of the reference portion, and adjusts the direction of the directivity of the radio wave sensor 2 so that it faces the target position T. This adjustment may be control that reduces the angular difference between the direction in which the target portion exists as viewed from the radio wave sensor 2 (hereinafter, target direction) and the center of the directivity, and does not have to be control that completely matches the two.

[0043] The processor 11 acquires biological information based on the observation data acquired from the radio wave sensor 2. For example, the processor 11 acquires a distance measurement value corresponding to the distance from the radio wave sensor 2 to a reflecting object based on the observation data, and stores the distance measurement value together with the acquisition time in the RAM 12. The processor 11 may then acquire biological information such as the heart rate or respiratory rate from time series data of the distance measurement values ​​obtained by performing sampling a predetermined number of times. The sampling interval may be sufficiently short compared to the period of breathing or heartbeat (for example, 50 milliseconds or 100 milliseconds).

[0044] In this embodiment, the processor 11 sets the chest as the target region and measures the respiratory rate as biological information based on time-series data of the chest position. When measuring the respiratory rate, the processor 11 may set the abdomen as the target region. In general, the abdomen vibrates frequently in response to breathing. Therefore, when measuring the respiratory rate as biological information, setting the abdomen as the target region can improve the detection accuracy of the biological information.

[0045] The processor 11 may be configured to measure the heart rate as biometric information, either additionally or alternatively. The processor 11 may measure the heart rate as biometric information based on time-series data of the chest position, with the chest as the target location. The chest is located close to the heart. Therefore, the amplitude of the chest position is likely to change relatively in response to pulsation. Alternatively, the Doppler shift associated with pulsation is likely to cause fluctuations in the frequency of the reflected wave. When measuring the heart rate as biometric information, using the chest as the target location can improve the detection accuracy of the biometric information. In this way, the target location may be changed depending on the item to be measured.

[0046] The processor 11 may measure biometric information from multiple candidate target areas (e.g., the chest and abdomen), evaluate both, and select the biometric information that produces better results.

[0047] As a method of evaluating the biological information, for example, time series data of the distance from the radio wave sensor 2 to the target location may be obtained based on the distance FFT data, and the time series data may be further subjected to FFT to obtain a frequency spectrum, and evaluation may be performed using the ratio of the square root of the sum of the squares (S / N ratio) of the desired signal band components to the signal components outside the desired signal band. The desired signal band components may be frequency components corresponding to the respiratory cycle (0.2 Hz to 1 Hz) or frequency components corresponding to the heartbeat cycle (1.0 to 2.5 Hz).

[0048] As an application example, the processor 11 may be configured to select, from among multiple peak detection points shown in the observation data, a peak detection point corresponding to a reflection point from the target location, based on information about the distance from the radio wave sensor 2 to the target location (i.e., the sensor-target distance) determined by analyzing the video from the camera 3. In other words, peak detection points resulting from reflectors other than the target location may be excluded based on the sensor-target distance. For example, if the sensor-target distance is estimated to be 55 cm and peak detection points are observed at 54 cm, 64 cm, and 80 cm, the peak detection point corresponding to 54 cm may be considered the distance measurement value for the target location, and the others may be considered noise.

[0049] This configuration reduces the risk of regarding distance measurements for noise points as distance measurements from the target area. For convenience, a predetermined range from the sensor-target distance determined based on the output of the camera 3 is referred to as the effective range. The above configuration corresponds to a configuration in which peak detection points outside the effective range are considered to be noise. The effective range corresponds to an example of a data section used to identify biological information.

[0050] As a further application example, the processor 11 may acquire supplemental information of the target occupant based on data input from the radio wave sensor 2 or other on-board sensors. The supplemental information of the target occupant is information indicating the attributes of the target occupant, which is different from the biometric information to be measured. The attributes of the target occupant may be information such as gender, age, and weight. In other words, the attributes of the target occupant are also information indicating the physical characteristics of a person. The supplemental information of the target occupant may be information that can determine the gender, age, weight, etc. of the target occupant. Hereinafter, information regarding the gender of the target occupant will be referred to as gender information. The gender information may be information that indicates the gender of the target occupant, or information that can determine the gender.

[0051] The processor 11 may obtain supplemental information (specifically, gender information) about the target occupant by analyzing the captured image provided by the camera 3. The processor 11 determines the physical (or biological) gender of the target occupant through image recognition or facial recognition. If the processor 11 determines that the target occupant is male, it may set the target area to the abdomen. Men often practice abdominal breathing. Therefore, if the target occupant is male, setting the target area to the abdomen can improve the accuracy of detecting the respiratory rate.

[0052] If the processor 11 determines that the target occupant is a woman, the processor 11 may set the target region to the abdomen. Women often breathe using their chest. Therefore, if the target occupant is a woman, the accuracy of detecting the respiration rate can be improved by setting the target region to the chest.

[0053] The processor 11 may determine whether the target occupant is an infant by analyzing the image provided by the camera 3. If the processor 11 determines that the target occupant is an infant, it may set the target region to the abdomen. Infants often exhibit diaphragmatic breathing. Therefore, if the target occupant is an infant, setting the target region to the abdomen can improve the accuracy of detecting the respiratory rate.

[0054] When measuring the respiratory rate, the processor 11 may set both the abdomen and the chest as target regions. The processor 11 evaluates the acquired abdominal biological information and chest biological information and selects one of the biological information based on the evaluation result. The evaluation of the biological information may be performed by any method.

[0055] As an example, the processor 11 collects, i.e., samples, observation data when the directivity is directed toward the abdomen and when the directivity is directed toward the chest. This allows the processor 11 to obtain abdomen-focused data, which is observation data when the directivity is directed toward the abdomen, and chest-focused data, which is observation data when the directivity is directed toward the chest. Each piece of data may be the result of FFT processing performed on the IF signal, and indicates a frequency spectrum corresponding to the distance to the reflecting object.

[0056] For example, Fig. 2 shows abdominal focus data, and Fig. 3 shows chest focus data. The horizontal axis in Figs. 2 and 3 indicates frequency, and the vertical axis indicates spectral intensity. Note that the peak detection point outside the effective range (f1 in the figures) indicates noise such as reflected components from vehicle components near the radio wave sensor 2. f2 indicates reflected components from the target area.

[0057] 2 and 3, the abdomen-focused data shown in Fig. 2 has a better S / N ratio. The processor 11 may compare the abdomen-focused data with the chest-focused data and use the result with the better S / N ratio to calculate the respiratory rate. The S / N ratio may be evaluated based on the ratio of the square root of the sum of squares (S / N ratio) of the components within the effective range to the other signal components.

[0058] Whether the target occupant is abdominally breathing or thoracically breathing cannot be determined definitively depending on gender, age, etc. Therefore, by taking measurements with the abdomen as the target site and measurements with the chest as the target site as described above, evaluating the biological information from each, and selecting the biological information, it becomes possible to measure the respiratory rate with high accuracy.

[0059] 4 and 5, a method for the processor 11 to calculate the coordinates of the target position T based on the captured image acquired from the camera 3 and the camera-to-target distance will be described. In this example, the target occupant is a person P.

[0060] The processor 11 determines the physique of the target occupant through image recognition. The physique determination is a process of calculating shoulder width and sitting height. The processor 11 recognizes the top of the head Ht and shoulders Hs of the target occupant through image recognition. The processor 11 also recognizes the seat surface S through image recognition. The processor 11 calculates the shoulder width and sitting height based on the top of the head Ht, shoulders Hs, and seat surface S. The sitting height is the length from the top of the head Ht to the seat surface S. The sitting height may also be the length from the eyes to the seat surface S. The shoulder width is the distance from one shoulder Hs to the other shoulder Hs.

[0061] The processor 11 determines, within the captured image, a position that is in the center of the shoulder width and that is a predetermined offset value L below the top of the head Ht, as the target position T. The target position T does not have to be in the center of the shoulder width. It may be a position offset to the left or right of the target occupant. The offset value L is set arbitrarily.

[0062] The offset value L may be an absolute value of the length or a ratio to the sitting height. The offset value L may be changed depending on the biological information to be measured, i.e., whether to measure the respiration rate or the heart rate. When measuring the respiration rate, the offset value L may be set to 50 cm so that the target site is the abdomen. When measuring the heart rate, the offset value L may be set to 40 cm so that the target site is the chest. The offset value L used for measuring the respiration rate may be a value that is a predetermined amount longer than the offset value L used for measuring the heart rate.

[0063] The offset value L may be changed depending on the target region. For example, if the offset value L when the chest is the target region is 40 cm, the offset value L for the abdomen may be set to 50 cm or 60 cm. The offset value L for the abdomen may be a value that is a predetermined amount longer than the offset value L for the chest.

[0064] The offset value L may be changed according to the distance between the camera 3 and the target occupant. For example, if the distance between the camera 3 and the target occupant is large, the offset value L may be increased. The offset value L may be changed according to the seat in which the occupant is sitting. For example, if the distance between the camera 3 and the seat in which the target occupant is sitting is large, the offset value L may be increased.

[0065] The value of the offset value L may be changed depending on the attributes of the target occupant. The value of the offset value L may be changed depending on the gender of the target occupant. If the target occupant is male, the value of the offset value L may be set larger than if the target occupant is female. The value of the offset value L may be changed depending on whether the target occupant is an infant or not. If the target occupant is an infant, the value of the offset value L may be set smaller than if the target occupant is not an infant. The gender of the target occupant may be determined by image recognition, etc. Whether the target occupant is an infant or not may also be determined by image recognition, etc.

[0066] Data in which an offset value L corresponding to each individual is defined may be stored in the storage 13. In this case, the processor 11 may identify the target occupant by image recognition or personal authentication (such as face recognition) and use setting data corresponding to the result.

[0067] The processor 11 may calculate a camera-to-reference portion distance, which is the distance from the camera 3 to the reference portion, and estimate the target position in the camera coordinate system using the camera-to-reference portion distance in combination with the shoulder width and sitting height. The processor 11 may calculate a camera-to-target distance in combination with the shoulder width and sitting height, and estimate the target position in the camera coordinate system. The processor 11 may use or correct the offset value L using the camera-to-target distance.

[0068] If the camera 3 is a camera capable of measuring distances, such as a stereo camera or a ToF camera, the position coordinates of feature points such as the right shoulder, left shoulder, eyes, top of the head, nose, and mouth in the camera coordinate system can also be identified. The processor 11 may estimate the target position in the camera coordinate system using data defining the relative position of the target part with respect to one of these feature points.

[0069] The camera coordinate system may be a Cartesian coordinate system with the mounting position of the camera 3 as the origin. Data defining the relative position of the target part with respect to the feature point may be different for each individual. The processor 11 may distinguish between the gender of the occupant by image recognition or personal authentication (such as face recognition) and use data corresponding to the result.

[0070] The processor 11 converts the target position in the camera coordinate system into the radio wave sensor coordinate system. The radio wave sensor coordinate system is a polar coordinate system with its origin at the attachment position of the radio wave sensor 2. Through the coordinate conversion, the processor 11 obtains the target position and the sensor-to-target distance in the radio wave sensor coordinate system.

[0071] <Processing Flow> A flow showing an example of processing performed by the processor 11 will be described using Figure 6. The processing flow starts when the vehicle ignition switch is turned ON. The processing flow may be repeated until the ignition switch is turned OFF. Alternatively, the processing flow of Figure 6 may be executed for a predetermined time in response to an event such as opening or closing a door or locking the vehicle. This makes it possible to detect, for example, a child being left behind in a vehicle.

[0072] In step 11, the processor 11 determines the attributes of the target occupant. In this embodiment, the processor 11 acquires supplemental information about the target occupant based on data input from the camera 3 (i.e., the captured image). The processor 11 determines the gender of the target occupant based on the captured image. In step 11, the processor 11 may additionally or alternatively determine whether the target occupant is a child based on the captured image. Note that the processing of step 11 may be omitted.

[0073] In step S12, the processor 11 determines the physique and distance of the target occupant. The processor 11 determines the physique and distance of the target occupant based on the captured image acquired from the camera 3. The processor 11 determines the offset value L according to the physique and distance of the target occupant. The processor 11 may adjust the offset value L according to the attributes of the target occupant acquired in step S11.

[0074] In step 13, as described above, the processor 11 estimates a position that is below the reference part (here, the top of the head) by the offset value L as the target position T. The processor 11 may identify the target position T in the camera coordinate system based on at least one of the physique information, distance information, and attribute information of the target occupant observed using the camera 3.

[0075] In step 14, the processor 11 transforms the target position T in the camera coordinate system into the radio wave sensor coordinate system. The processor 11 acquires the target position T in the radio wave sensor coordinate system. At the same time, the processor 11 acquires the sensor-target distance.

[0076] In step 15, the processor 11 adjusts the direction of the directivity of the radio wave sensor 2 so that it is directed toward the target position T. The signal processing circuit 20 selects the antenna 21 whose directivity is directed toward the target position T.

[0077] In step 16, the processor 11 controls the radio wave sensor 2 to transmit and receive radio waves. The processor 11 acquires biometric information of the target occupant based on the radio waves transmitted and received by the radio wave sensor 2. Here, the processor 11 selects a peak point of interest based on the sensor-to-target distance, and acquires the biometric information based on time-series data of the distance information of the selected peak point.

[0078] Summary of the embodiment According to the present embodiment, the directivity of the radio wave sensor 2 is determined according to the position of the reference body part. Therefore, the directivity can be set in an appropriate direction. This reduces the influence of radio waves reflected from directions other than the desired direction. Therefore, it is possible to improve the detection accuracy of biological information.

[0079] In this embodiment, the directivity of the radio wave sensor 2 is adjusted so that it is directed toward the target position T. This reduces the influence of radio waves reflected from areas other than the target location, thereby improving the accuracy of detecting biological information.

[0080] In this embodiment, the sensor used to measure the biometric information (radio wave sensor 2) is different from the sensor used to acquire information for identifying the position of the reference body part (camera 3). This makes it possible to use sensors suitable for both measuring the biometric information and acquiring the position information of the reference body part. This makes it possible to improve the accuracy of detecting the biometric information.

[0081] In this embodiment, the processor 11 acquires information for identifying the position of the reference portion from the camera 3. The processor 11 identifies the position of the reference portion based on the image captured by the camera 3, thereby enabling the processor 11 to improve the accuracy of identifying the position information of the reference portion.

[0082] In this embodiment, the processor 11 selects the reflected wave to be used for measuring the biological information based on the sensor-target distance. This allows the selection of a radio wave suitable for measuring the biological information, thereby improving the detection accuracy of the biological information.

[0083] <Modifications> The radio wave sensor 2 does not have to be placed in a position where it can measure all occupants in the vehicle cabin. For example, if the target occupant is the driver only, it may be placed in a position where it can measure only the driver. The radio wave sensor 2 may be placed in a sun visor above the driver's seat. The radio wave sensor 2 may be placed on the steering wheel. The same applies to the placement of the camera 3. The camera 3 does not have to be placed in a position where it can capture images of all occupants in the vehicle cabin.

[0084] The multiple antennas 21 may be adaptive array antennas. The signal processing circuit 20 may independently control the amplitude and phase of excitation of each antenna 21 to direct the directivity toward the target position T. The radio wave sensor 2 may have variable directivity using digital beamforming. The antenna 21 of the radio wave sensor 2 may be an array antenna in which antenna elements are arranged two-dimensionally. For example, as shown in FIG. 7 , the radio wave sensor 2 may have three rows of transmitting antenna elements and four rows of receiving antenna elements. Such a radio wave sensor 2 may be realized with the aid of a MIMO (Multiple-Input-Multiple-Output) radar. The radio wave sensor 2 may have antenna elements arranged in a matrix, and the directivity may be adjustable not only in the vertical direction but also in the horizontal direction.

[0085] Alternatively, the radio wave sensor 2 may use a single directional antenna as a transmitting and receiving antenna. In this case, the orientation of the single directional antenna may be changed by physically moving the attitude of the antenna using an actuator such as a motor.

[0086] The radio wave sensor 2 may also have a separate transmitting antenna and a receiving antenna. In this case, the directivities of both the transmitting antenna and the receiving antenna may be directed toward the target position T. The directivity of either the transmitting antenna or the receiving antenna may be directed toward the target position T.

[0087] The radio wave sensor 2 may calculate the position of the reference region and provide it to the processor 11. That is, the processor 11 may acquire the biometric information and the positional information of the reference region from one radio wave sensor 2. This makes it possible to miniaturize the entire biometric detection system.

[0088] The biological detection system 100 may include a radio wave sensor (hereinafter referred to as the second radio wave sensor 5) different from the radio wave sensor 2 (hereinafter referred to as the first radio wave sensor). In this case, the second radio wave sensor 5 may calculate the position of the reference site. The second radio wave sensor 5 may also detect the sensor-to-target distance. The second radio wave sensor 5 may be a radio wave sensor that operates using a different method than the first radio wave sensor. As an example, the second radio wave sensor 5 is a pulse-type millimeter-wave radar. In other words, information on the measurement of biological information and information on the position of the reference site are obtained from radio wave sensors that use different methods. This makes it possible to employ sensors that are suitable for measuring biological information and obtaining information on the position of the reference site. This makes it possible to improve the accuracy of biological information detection.

[0089] A plurality of values ​​of the camera-to-target distance or the sensor-to-target distance may be stored in advance in the storage 13 as set values ​​for each seat. The camera-to-target distance for each seat may be stored in a storage medium included in the camera 3, rather than in the storage 13. Similarly, the sensor-to-target distance for each seat may be stored in a storage medium included in the camera 3, rather than in the storage 13. Hereinafter, the camera-to-target distance may be appropriately replaced with the sensor-to-target distance.

[0090] When using the stored camera-target distance value, the processor 11 may read and use the camera-target distance value corresponding to the seat in which the target occupant is seated. The seat in which the target occupant is seated may be identified using the detection results of a seat belt sensor, a weight sensor, or the camera 3. For example, R for a person seated in the driver's seat may be set to 60 cm, and the camera-target distance for a person seated in a rear seat may be set to 140 cm, for example.

[0091] In a configuration using a pre-registered camera-to-target distance, the camera-to-target distance may be changed depending on the type of biometric information to be measured. For example, the camera-to-target distance for measuring heart rate may be set to be about 5 cm shorter than the camera-to-target distance for measuring respiration rate.

[0092] When using stored values ​​of the camera-target distance, the value of the camera-target distance may be changed depending on the attributes of the target occupant. For example, the value of the camera-target distance may be changed depending on the gender of the target occupant. The camera-target distance value for a male may be set to a predetermined amount (e.g., 10 cm) smaller than the camera-target distance value for a female. The camera-target distance value for a child may be set to a value different from the camera-target distance value for an adult. Each setting value may be determined by testing.

[0093] The storage 13 may store the camera-to-target distance for each seat in advance. The processor 11 may obtain the camera-to-target distance associated with the seat in which the target occupant is sitting from the storage 13. In this case, it is not necessary to measure the camera-to-target distance. This reduces the calculation load on the processor 11. Then, the received radio waves are selected based on the camera-to-target distance. Therefore, it is possible to select radio waves (in other words, data intervals) suitable for measuring biometric information, thereby improving the detection accuracy of biometric information.

[0094] The processor 11 may be configured to convert the camera-to-target distance into the sensor-to-target distance and then select a data section to be used for extracting biometric information. The processor 11 may also be configured to estimate the target position T from the camera-to-reference portion distance and physique information, instead of the camera-to-target distance. The above-mentioned description of the camera-to-target distance may be replaced with the camera-to-reference portion distance. The camera-to-reference portion distance may be actually measured using parallax information or ToF, or may be registered in advance in the storage 13. The storage 13 may have registered therein a plurality of patterns of camera-to-reference portion distances according to seat, gender, and whether or not the subject is an infant.

[0095] The above describes a mode in which the beam is adjusted after the target position T is identified, but identifying the target position T may be an optional element. The processor 11 may change the beam adjustment direction directly from the position of the reference part. If the position of the top of the head is lower than the base position registered in advance, the beam may be lowered. If the position of the top of the head detected by the camera 3 is higher than the base position, the beam may be directed upward.

[0096] When the target location is set to the chest, the processor 11 may operate as follows: When the processor 11 acquires position information about the head as the reference location, the processor 11 may adjust the direction of the directivity of the radio wave sensor 2 so that the beam is directed at least downward from the head. When the processor 11 acquires position information about the waist as the reference location, the processor 11 may adjust the direction of the directivity of the radio wave sensor 2 so that the beam is directed upward from the waist. In this way, the processor 11 may adjust the direction of the directivity of the radio wave sensor 2 depending on the relationship between the reference location and the target location.

[0097] Additionally, the processor 11 may perform frequency analysis or signal analysis by focusing on frequency components derived from respiration or heartbeat in the received signal of the reflected wave. For example, in the case of respiratory rate, the processor may be configured to extract only frequency components between 0.2 and 1.0 Hz from the received signal of the reflected wave and perform analysis processing to identify the respiratory rate. In the case of heart rate, the processor may be configured to extract only frequency components between 1.0 and 2.5 Hz from the received signal of the reflected wave and perform analysis processing to identify the heart rate. Various methods may be used to estimate the heart rate or respiratory rate from waveform data. Extracting specific frequency components from the entire received waveform data in this way also corresponds to selecting a data section to be used to identify biological information.

[0098] The camera 3 in the above-described embodiment is not limited to a camera capable of measuring distances, such as a stereo camera or a ToF camera, but may also be a monocular optical camera. In this case, the processor 11 may estimate the shoulder width W based on the in-image distance (e.g., the number of pixels in the horizontal direction) from the right shoulder to the left shoulder in the captured image. Similarly, the processor 11 may estimate the sitting height based on the number of pixels in the vertical direction from the seating surface to the top of the head. The processor 11 may identify the target direction, which is the direction in which the target part exists as seen from the camera 3, based on the position of the target part in the image and the optical axis direction of the camera 3. The processor 11 may estimate the target direction as seen from the radio wave sensor 2 by performing coordinate transformation on the target direction as seen from the camera 3, and control the directivity based on the estimation result.

[0099] Some of the functions of the processor 11 may be provided by the radio wave sensor 2, the camera 3, or the ADASECU 4. The arrangement of the functions can be changed as appropriate. The ADASECU 4 is an optional element and may be omitted. The living body detection device 1 and the radio wave sensor 2 or the camera 3 may be configured as an integrated unit. The living body detection device 1, the radio wave sensor 2, and the camera 3 may be configured as an integrated unit. A module including the living body detection device 1 and the radio wave sensor 2 may be configured as a device that can be retrofitted to a vehicle (removable if necessary). For example, the module including the living body detection device 1 may be configured so that it can be retrofitted to an assist grip provided on a headrest or ceiling.

[0100] (Disclosure of Technical Ideas) This specification discloses multiple technical ideas described in the following multiple clauses. Some clauses may be described in a multiple dependent form, with the subsequent clause alternatively referring to the preceding clause. Furthermore, some clauses may be described in a multiple dependent form, with the subsequent clause referring to another multiple dependent clause. These multiple dependent clauses define multiple technical ideas.

[0101] (Technical Idea 1) A living body detection device that acquires biometric information of a target occupant, comprising a control unit (11) that acquires the biometric information using a radio wave sensor configured with variable directivity, wherein the control unit acquires position information of a reference body part of the target occupant based on data received from the radio wave sensor or another on-board sensor different from the radio wave sensor, and adjusts the direction of the directivity of the radio wave sensor in accordance with the position information of the reference body part. (Technical Idea 2) The living body detection device according to Technical Idea 1, wherein the control unit acquires a target position that is the position of a target body part of the target occupant based on the position information of the reference body part, and adjusts the direction of the directivity of the radio wave sensor to face the target position. (Technical Idea 3) The living body detection device according to Technical Idea 1 or 2, wherein the control unit is configured to acquire the position information of the reference body part based on data received from the other on-board sensor. (Technical Idea 4) The living body detection device according to Technical Idea 3, wherein the other on-board sensor is a second radio wave sensor different from the first radio wave sensor, and the second radio wave sensor detects the position of the reference body part using a method different from that of the first radio wave sensor. (Technical Idea 5) A biological detection device described in any one of Technical Ideas 1 to 4, wherein the radio wave sensor transmits radio waves and receives reflected waves that are returned after the radio waves are reflected by an object, and the control unit: acquires distance information from the other vehicle-mounted sensor to be used to calculate a sensor-target distance, which is the distance between the radio wave sensor and the target part of the target occupant; calculates the sensor-target distance from the distance information; acquires data of the reflected waves from the radio wave sensor; selects a data section from the reflected wave data to be used to identify the biological information; and acquires the biological information based on the selected data section.(Technical Idea 6) The living body detection device according to any one of Technical Ideas 1 to 4, wherein the radio wave sensor transmits radio waves and receives reflected waves that are returned after the radio waves are reflected by an object, and the control unit: acquires data of the reflected waves from the radio wave sensor; acquires the sensor-target distance, which is the distance between the radio wave sensor and a target part of the target occupant, from a storage unit that stores the sensor-target distance; selects a data section of the reflected wave data to be used for identifying the biological information based on the sensor-target distance; and acquires the biological information based on the selected data section. (Technical Idea 7) The living body detection device according to Technical Idea 1 or 2, wherein the control unit is configured to acquire position information of the reference part based on data received from the radio wave sensor. (Technical Idea 8) The living body detection device according to any one of Technical Ideas 1 to 7, wherein the control unit measures a heart rate as the biological information, and the target part of the target occupant is the chest. (Technical Idea 9) The living body detection device according to any one of Technical Ideas 1 to 7, wherein the control unit measures a respiratory rate as the biological information, and a target area of ​​the target occupant is the abdomen. (Technical Idea 10) The living body detection device according to any one of Technical Ideas 1 to 7, wherein the control unit measures a respiratory rate as the biological information, and obtains gender information of the target occupant based on data input from the radio wave sensor or the other in-vehicle sensor, and switches between setting the target area of ​​the target occupant to the abdomen or the chest depending on the gender of the target occupant. (Technical Idea 11) The living body detection device according to any one of Technical Ideas 1 to 7, wherein the control unit measures a respiratory rate as the biological information, and obtains supplementary information of the target occupant based on data input from the radio wave sensor or the other in-vehicle sensor, and sets the target area of ​​the target occupant to the abdomen if the target occupant is an infant.(Technical Idea 12) The biological detection device according to any one of Technical Ideas 1 to 7, wherein the control unit is configured to: measure a respiratory rate as the biological information; set the abdomen and chest as target regions of the target occupant; and select, based on a result of comparing the acquired abdomen attention data, which is data obtained when the abdomen is set as the target region, with the acquired chest attention data, which is data obtained when the chest is set as the target region, from the abdomen attention data and the chest attention data, to select the data to be used to identify the biological information; and identify the biological information based on the selected data. (Technical Idea 13) A biological detection method for acquiring biological information of a target occupant, executed by a processor, comprising: acquiring position information of a reference region of the target occupant based on data received from a radio wave sensor configured with variable directivity or another on-board sensor different from the radio wave sensor; adjusting a direction of the directivity of the radio wave sensor in accordance with the position information of the reference region; and acquiring the biological information based on output data of the radio wave sensor.

Claims

1. A biometric detection device for acquiring biometric information of a target occupant, comprising a control unit (11) that acquires the biometric information using a radio wave sensor configured with variable directivity, wherein the control unit acquires position information of a reference part of the target occupant based on data received from the radio wave sensor or another vehicle-mounted sensor different from the radio wave sensor, and adjusts the direction of the directivity of the radio wave sensor according to the position information of the reference part.

2. The biological detection device described in claim 1, wherein the control unit acquires a target position, which is the position of the target part of the target occupant, based on the position information of the reference part, and adjusts the directivity direction of the radio wave sensor so that it is directed toward the target position.

3. The biological detection device according to claim 1 or 2, wherein the control unit is configured to acquire position information of the reference region based on data received from the other vehicle-mounted sensor.

4. The biological detection device described in claim 3, wherein the other vehicle-mounted sensor is a second radio wave sensor different from the first radio wave sensor, and the second radio wave sensor detects the position of the reference part using a method different from that of the first radio wave sensor.

5. The biological detection device of claim 2, wherein the radio wave sensor transmits radio waves and receives reflected waves that are returned after the radio waves are reflected by an object, and the control unit: acquires distance information from the other vehicle-mounted sensor to be used to calculate the sensor-to-target distance, which is the distance between the radio wave sensor and the target location; calculates the sensor-to-target distance from the distance information; acquires data of the reflected waves from the radio wave sensor; selects a data section from the reflected wave data to be used to identify the biological information; and acquires the biological information based on the selected data section.

6. The biological detection device of claim 2, wherein the radio wave sensor transmits radio waves and receives reflected waves that are returned after the radio waves are reflected by an object, and the control unit: acquires data on the reflected waves from the radio wave sensor; acquires the sensor-target distance, which is the distance between the radio wave sensor and the target location, from a memory unit that stores the sensor-target distance; selects a data section from the reflected wave data to be used to identify the biological information based on the sensor-target distance; and acquires the biological information based on the selected data section.

7. A biological detection device as described in claim 1 or 2, wherein the control unit is configured to obtain position information of the reference part based on data received from the radio wave sensor.

8. The biological detection device according to claim 2, wherein the control unit measures a heart rate as the biological information, and the target site is the chest.

9. The biological detection device according to claim 2, wherein the control unit measures respiratory rate as the biological information, and the target site is the abdomen.

10. The control unit of the biological detection device described in claim 2 measures the respiratory rate as the biological information, obtains gender information of the target occupant based on data input from the radio wave sensor or the other vehicle sensor, and switches the target area between the abdomen and the chest depending on the gender of the target occupant.

11. The control unit measures the respiratory rate as the biological information, obtains supplementary information about the target occupant based on data input from the radio wave sensor or the other vehicle-mounted sensor, and if the target occupant is an infant, the target area is the abdomen. A biological detection device as described in claim 2.

12. The biological detection device of claim 2, wherein the control unit is configured to measure respiratory rate as the biological information, set the target region to the abdomen and chest, and based on a result of comparing the acquired abdomen attention data, which is data obtained when the abdomen is set as the target region, with the chest attention data obtained when the chest is set as the target region, select the data to be used to identify the biological information from the abdomen attention data and the chest attention data, and identify the biological information based on the selected data.

13. A biometric detection method executed by a processor for acquiring biometric information of a target occupant, comprising: acquiring position information of a reference part of the target occupant based on data received from a radio wave sensor configured with variable directivity or from another vehicle-mounted sensor different from the radio wave sensor; adjusting the direction of the directivity of the radio wave sensor according to the position information of the reference part; and acquiring the biometric information based on the output data of the radio wave sensor.

Images