Detection system, radio wave sensor, detection method, and program

Abstract

The present invention addresses the problem of providing a detection system that can easily identify an object when detecting the object by using reflected waves of radio waves transmitted in a prescribed direction. This detection system (100) uses a reception signal to detect an object in a region. The reception signal is a signal based on reflected waves of radio waves transmitted in a prescribed direction. The detection system (100) comprises a processing unit (22) that acquires information pertaining to the position of the object in the region by executing distance measurement processing based on the reception signal. The processing unit (22) acquires a plurality of pieces of position information respectively corresponding to a plurality of portions, which constitutes the object, and identifies the object on the basis of the positional relationship between both ends in the prescribed direction of a distribution of a plurality of points respectively corresponding to the plurality of pieces of position information.

Description

Detection system, radio wave sensor, detection method, and program

[0001] The present disclosure relates to a detection system, a radio wave sensor, a detection method, and a program, and more particularly to a detection system, a radio wave sensor, a detection method, and a program that use a radio wave sensor to detect an object such as a human body.

[0002] Patent Document 1 describes an intrusion detection device that receives reflected waves from objects that are transmitted from different directions and detects an intrusion that has entered a detection area based on the strength of the received reflected waves (received wave strength). This intrusion detection device detects a moving object (moving body), such as an intruder, that is moving in some way, based on changes in the received wave strength over time.

[0003] In the intrusion detection device described in Patent Document 1, depending on the direction of the radio waves, it can be difficult to identify the object, for example, the type of object, such as whether the object is a person, a cleaning robot, or a pet, or to identify the movement of the object, such as whether the object is moving or stationary, and if moving, whether the object is moving as a whole or only partially.

[0004] Japanese Patent Application Laid-Open No. 2002-236171

[0005] The object of the present disclosure is to provide a detection system, a radio wave sensor, a detection method, and a program that can facilitate identification of an object when detecting the object using reflected waves of radio waves transmitted in a predetermined direction.

[0006] A detection system according to one aspect of the present disclosure is a detection system for detecting an object within an area using a received signal. The received signal is a signal based on a reflected wave of a radio wave transmitted in a predetermined direction. The detection system includes a processing unit that acquires information about the position of the object within the area by performing a ranging process based on the received signal. The processing unit acquires a plurality of pieces of position information corresponding to a plurality of parts that make up the object, and identifies the object based on the positional relationship between both ends in the predetermined direction of a distribution of a plurality of points that correspond to the plurality of pieces of position information.

[0007] A radio wave sensor according to one aspect of the present disclosure is a radio wave sensor that transmits radio waves in a predetermined direction, receives reflected waves of the radio waves, and detects an object within an area using a received signal that is a signal based on the reflected waves. The radio wave sensor includes a processing unit that acquires information about the position of the object within the area by performing a distance measurement process based on the received signal. The processing unit acquires multiple pieces of position information corresponding to multiple parts that make up the object, and identifies the object based on the positional relationship between both ends of a distribution of multiple points that correspond to the multiple pieces of position information in the predetermined direction.

[0008] A detection method according to one aspect of the present disclosure is a detection method for detecting an object within an area using a received signal. The received signal is a signal based on a reflected wave of a radio wave transmitted in a predetermined direction. The detection method includes a processing step for acquiring information about the position of the object within the area by performing a ranging process based on the received signal. In the processing step, a plurality of pieces of position information corresponding to a plurality of parts constituting the object are acquired, and the object is identified based on the positional relationship between both ends in the predetermined direction of a distribution of a plurality of points corresponding to the plurality of pieces of position information.

[0009] A program according to one aspect of the present disclosure causes one or more processors to execute the detection method.

[0010] FIG. 1 is a block diagram of a detection system (device control system) according to a first embodiment of the present disclosure. FIG. 2 is a conceptual diagram of a room in which the device control system is used. FIG. 3 is a graph showing changes in frequency of a transmission wave transmitted by a radio wave sensor constituting the device control system. FIG. 4 is a flowchart illustrating a portion of the operation of a control device constituting the device control system. FIG. 5 is a flowchart illustrating another portion of the operation of the device control system. FIG. 6 is a flowchart illustrating a one-to-many frame difference acquisition process included in the operation of the device control system. FIG. 7 is a flowchart illustrating a posture estimation process included in the operation of the device control system. FIG. 8 is a waveform diagram illustrating an example of an inter-frame difference (one-to-one inter-frame difference). FIG. 9A is a frequency spectrum diagram illustrating an FFT result for a current frame Fr0, FIG. 9B is a frequency spectrum diagram illustrating an FFT result for a subsequent frame Fr1, and FIG. 9C is a frequency spectrum diagram illustrating a difference between FFT results. FIG. 10 is a waveform diagram illustrating another example of an inter-frame difference (one-to-many inter-frame difference). FIG. 11 is a conceptual diagram illustrating the characteristics of various human body movements. FIG. 12 is a waveform diagram illustrating an example of one-to-multiple frame difference acquisition in response to various movements in the same. FIG. 13A is a distribution diagram illustrating an example of cluster distribution in a standing position, which is one of human body postures. FIG. 13B is a distribution diagram illustrating an example of cluster distribution in a sitting position. FIG. 13C is a distribution diagram illustrating an example of cluster distribution in a lying position. FIG. 14A is a conceptual diagram illustrating a three-dimensional figure surrounding a group of clusters in a standing position. FIG. 14B is a conceptual diagram illustrating a three-dimensional figure surrounding a group of clusters in a sitting position. FIG. 14C is a conceptual diagram illustrating a three-dimensional figure surrounding a group of clusters in a lying position. FIG. 15 is a data structure diagram of an automatic control information group. FIG. 16 is a data structure diagram of control history information. FIG. 17 is a block diagram of a modified example of the same device control system. FIG. 18 is a flowchart illustrating a portion of the operation of a control device constituting the detection system according to a second embodiment of the present disclosure. FIG. 19 is a flowchart illustrating a type identification process included in the operation of the same. FIG. 20A is a distribution map showing an example of a cluster distribution corresponding to a cleaning robot, and FIG. 20B is a distribution map showing an example of a cluster distribution corresponding to an electric fan.Fig. 21A is a distribution diagram showing an example of cluster distribution before the cat starts moving, and Fig. 21B is a distribution diagram showing an example of cluster distribution after the cat starts moving. Fig. 22 is a flowchart explaining an individual identification process further included in the above operation. Fig. 23A is a distribution diagram showing an example of cluster distribution corresponding to the human body of person AA, and Fig. 23B is a distribution diagram showing an example of cluster distribution corresponding to the human body of person BB. Fig. 24 is a block diagram of a radio wave sensor according to a modified example of the above detection system.

[0011] First Embodiment Hereinafter, a first embodiment of the present disclosure will be described with reference to FIGS.

[0012] (1) Overview of Device Control System In this embodiment, the detection system of the present disclosure is a device control system 100 that has a detection function for detecting an object (for example, a function for detecting an object and identifying the type of the detected object, a function for identifying the movement of the identified object, a human body estimation function for making inferences about an object identified as a human body, and in particular a human body posture estimation function for making inferences about the posture of the human body), as well as a device control function. As shown in FIG. 2 , the device control system 100 of this embodiment uses a radio wave sensor 1 to detect moving objects (hereinafter, sometimes referred to as "moving objects") among objects, and in particular makes inferences about a human body 301.

[0013] An object is an object to be detected by the detection system. The object is, for example, a human body 301, an animal body other than the human body 301 (such as a pet), a moving object other than an animal (such as the electric blinds 200c), and a non-moving object (a stationary object 302). Detecting an object means, for example, detecting an object that matches any of a plurality of predetermined characteristics. Identifying an object means identifying the type of object based on the detected characteristics.

[0014] Identifying the movement of an object is, for example, determining whether the object is moving or not, and if the object is moving, determining which of a plurality of predetermined movements the object's movement corresponds to. Determining whether the object is moving or not may be, for example, detecting a moving object, i.e., a moving body.

[0015] Furthermore, identifying the movement may include, for example, identifying the timing when the object starts to move and the timing when the object stops moving. Furthermore, identifying the movement may include, for example, determining whether the movement is periodic or non-periodic (in other words, whether the movement is regular or irregular).

[0016] The estimation related to the human body 301 includes, for example, estimating whether the target object is the human body 301 or a moving object other than the human body 301, and further estimating the posture of the human body 301.

[0017] The posture of the human body 301 may be, for example, but not limited to, a standing position, a sitting position, a lying position (see FIGS. 13A to 13C), etc. The device control system 100 then detects various movements of the human body 301, and ultimately the actions of the person, based on the changes in the estimated posture, and controls the device 200 in accordance with the detection results.

[0018] The change in posture may be, for example, a change between a standing position, a sitting position, and a lying position, but is not limited to this. The change in posture also includes a continuation of no change in posture (for example, a case where a predetermined period of time has passed since changing to a sitting position, but there is no change to a standing position, etc.). Furthermore, the behavior to be detected is, in particular, a non-operation behavior (described later), but may also be an operation behavior (described later).

[0019] (1-1) Radio wave sensor The radio wave sensor 1 transmits radio waves (transmission waves Tr) from an antenna, receives reflected waves Re that are reflected by an object from the transmission waves Tr, and outputs information that can identify the position of the object (hereinafter referred to as "position-identifying information").

[0020] (1-1-1) Position Identifiable Information Position Identifiable Information is information that can identify the position of an object. The position of the object is preferably a three-dimensional position, but may also be a two-dimensional or one-dimensional position. The position Identifiable Information is, for example, an FFT result group (information that can identify a three-dimensional position) described below, but may also be each of multiple (e.g., three) FFT results that make up the FFT result group (information that can identify a one-dimensional position).

[0021] Also, for example, a signal indicating the time difference Δt from the transmission of the transmission wave Tr to the reception of the reflected wave Re, or a signal indicating the frequency difference Δf corresponding to the time difference Δt (for example, an IF signal indicating the frequency difference Δf between the transmission wave Tr modulated by the FMCW method and the reflected wave Re: see FIG. 3), etc. may also be considered as a type of location identification information. Furthermore, the distance calculated from the time difference Δt or the frequency difference Δf, etc., may itself be location identification information.

[0022] (1-1-2) Antenna The antenna used to transmit the transmission wave Tr and the antenna used to receive the reflected wave Re may be the same antenna (hereinafter referred to as a "shared antenna") or may be separate antennas (hereinafter referred to as a "transmitting antenna" and a "receiving antenna"). In other words, the transmission wave Tr may be transmitted from the shared antenna, and the reflected wave Re of the transmission wave Tr may be received by the shared antenna, or the transmission wave Tr may be transmitted from the transmitting antenna, and the reflected wave Re of the transmission wave Tr may be received by the receiving antenna.

[0023] Generally, to identify the three-dimensional position of an object (for example, to locate a point corresponding to a human body in a virtual three-dimensional space 500), the radio wave sensor 1 needs to have, for example, three or more shared antennas, or one transmitting antenna and three or more receiving antennas. Note that the radio wave sensor 1 having multiple antennas does not necessarily mean that the multiple antennas are housed in a single housing, but also means that the multiple antennas are located in multiple locations separated from each other. In the latter case, each of the multiple antennas is connected to the sensor main body via wire or wirelessly so as to be able to communicate with the sensor main body.

[0024] In this embodiment, the radio wave sensor 1 has one transmitting antenna and three or more (e.g., three) receiving antennas. The positions of the one transmitting antenna and the three or more receiving antennas included in the radio wave sensor 1 are known, and four or more pieces of position information (hereinafter referred to as "antenna position information group") corresponding to the four or more antennas are stored in advance in, for example, the memory of the control device 2.

[0025] However, the number of receiving antennas constituting the radio wave sensor 1 may be two or one (two antennas allow for point placement on a plane, and one antenna allows for point placement along a line).

[0026] (1-1-3) Transmission and Reception Operation The radio wave sensor 1 performs a transmission and reception operation. The transmission and reception operation is an operation of transmitting a transmission wave Tr toward a real space in which a group of objects may exist, receiving a reflected wave Re from the real space, and outputting a signal (e.g., an IF signal) based on the transmission wave Tr and the reflected wave Re. The real space in which a group of objects may exist is, for example, a space (indoor space) surrounded by the floor, ceiling, and side walls of a room 400, and may hereinafter be referred to as the "real space (400)." Note that the real space in which a group of objects may exist is not limited to an indoor space, but may also be an outdoor space such as a corridor or terrace.

[0027] An object group is a collection of one or more objects. In this embodiment, the object group includes one or more of a human body 301, a moving object other than a human body (for example, an electric blind 200c, which may be hereinafter referred to as a "moving object other than a human body (200c)"), and a stationary object 302. In other words, each of the one or more objects constituting the object group is either a human body 301, a moving object other than a human body (200c), or a stationary object 302.

[0028] The radio wave sensor 1 performs the following transmission and reception operation at a predetermined cycle (e.g., once every 20 ms): In this transmission and reception operation, one transmitting antenna transmits a transmission wave Tr to a real space (400) in which a group of objects may exist, and three or more receiving antennas receive reflected waves Re from the real space (400).

[0029] For example, when a human body 301, a moving object (200c) other than a human body, and a stationary object 302 exist in the real space (400), the reflected waves Re from the group of objects (301, 200c, 302) are received by each of three or more receiving antennas. Note that, in the example of Fig. 2, the stationary object 302 is, for example, a desk 302a and a bed 302b placed on the floor of the room 400, but it may also be the floor.

[0030] The transmission wave Tr is a radio wave modulated by a predetermined method. The predetermined method is, for example, but not limited to, the FMCW (Frequency Modulated Continuous Wave) method. In this embodiment, the transmission wave Tr is a radio wave modulated by the FMCW method.

[0031] (1-1-4) FMCW Method The FMCW method is a method in which, as shown in FIG. 3, the frequency f of a transmission wave Tr (transmission signal) having a predetermined time length (chirp length Tc of the chirp signal) is linearly increased (or decreased) from a starting frequency f0 at a predetermined slope S as time t passes.

[0032] It should be noted that the acquisition of the location identifiable information as described above does not have to be performed for all of the transmission and reception operations that are repeatedly performed at a predetermined cycle. For example, as will be described later, if a predetermined time T (e.g., T = 200 ms) is defined as one frame (described later), and N transmission and reception operations (N is a natural number, e.g., 10) are performed in one frame, location identifiable information may be acquired for each of the N transmission and reception operations of the N transmission and reception operations that belong to one frame, or location identifiable information may be acquired for only one of the N transmission and reception operations (e.g., the first transmission and reception operation).

[0033] The predetermined period is, for example, once every 20 ms, and in this embodiment, with one frame being 200 ms, the frequency is 10 times per frame (N=10). However, the predetermined period may also be, for example, 20 times per frame (N=20), 5 times per frame (N=5), etc.

[0034] (1-1-5) IF Signal, FFT Result, and FFT Result Group Each time the radio wave sensor 1 performs a transmission / reception operation, it generates an IF signal for each of three or more receiving antennas, performs an FFT (Fast Fourier Transform) on the IF signal to obtain an FFT result, and outputs an FFT result group. In other words, each time a transmission / reception operation is performed, the radio wave sensor 1 outputs an FFT result group consisting of three or more FFT results corresponding to the three or more receiving antennas. However, a Fourier transform other than FFT may be performed on the IF signal, in which case the radio wave sensor 1 outputs a Fourier transform result group consisting of three or more Fourier transform results corresponding to the three or more receiving antennas.

[0035] The IF signal is a signal that indicates the frequency difference Δf between the transmitted wave Tr and the reflected wave Re, as shown in Fig. 3. The IF signal is a signal that indicates the difference between the frequency of the transmitted wave Tr and the frequency of the reflected wave Re at time t, and is a function Δf(t) of time t, but indicates a constant value when the group of objects (301, 302) is stationary.

[0036] The IF signal is generated by mixing the transmitted wave Tr and the reflected wave Re. The IF signal is generated over the period during which the transmitted wave Tr is being transmitted and the reflected wave Re is being received (i.e., the period from the start of reception of the reflected wave Re to the end of transmission of the transmitted wave Tr).

[0037] The FFT result is the result of performing FFT on the IF signal. The FFT result is information indicating a frequency spectrum (relationship between frequency f and reflection intensity amp) as shown in Figures 9A and 9B, for example.

[0038] The FFT result group is information consisting of three or more FFT results corresponding to three or more receiving antennas, obtained for one transmission / reception operation. The three-dimensional position of an object can be identified using such FFT result groups. Furthermore, by taking the difference between multiple FFT result groups, frequency components corresponding to stationary objects 302 are removed, and only frequency components corresponding to moving objects such as human bodies 301 (frequency f at which amp exceeds a threshold, and the amp value corresponding to that frequency f) are obtained (see Figures 9A to 9C). Based on the frequency components thus obtained, information regarding the three-dimensional position and movement of moving objects such as human bodies 301 can be obtained.

[0039] Specifically, in this embodiment, the radio wave sensor 1 outputs multiple sets of FFT results corresponding to a series of multiple transmission and reception operations, and the output sets of FFT results are stored in chronological order in the memory of the control device 2. Meanwhile, the memory also stores the antenna position information set described above. The control device 2 then calculates the time difference (the time difference between the FFT results for each of the three receiving antennas) between the multiple sets of FFT results (e.g., two adjacent sets of FFT results) stored in chronological order in the memory, and acquires a set of ranging results (two ranging results corresponding to the three receiving antennas) based on the calculated difference. The control device 2 performs three-point positioning using the acquired set of ranging results and a set of pre-stored antenna position information. This allows for the acquisition of information identifying the three-dimensional position of the human body 301, for example, the calculation of three-dimensional coordinates.

[0040] (1-2) Attitude Estimation Function of Device Control System As shown in FIG. 1, the device control system 100 includes a distance measurement unit 221, a point placement unit 222, and an attitude estimation unit 223.

[0041] (1-2-1) Distance Measuring Unit: One-to-One or One-to-Many Difference The distance measuring unit 221 measures the distance from the radio wave sensor 1 to the human body 301 based on the FFT results output by the radio wave sensor 1 .

[0042] More specifically, the distance measuring unit 221 holds the FFT result group output by the radio wave sensor 1 for a period equal to or greater than the predetermined period (preferably, a period equal to or greater than twice the predetermined period).The distance measuring unit 221 then performs distance measurement processing based on the difference between a reference FFT result group (in this embodiment, a current FFT result group), which is one FFT result group among the multiple FFT result groups it holds, and at least one target FFT result group (in this embodiment, at least one preceding FFT result group), to obtain at least one distance measurement result group.

[0043] (1-2-1a) Reference FFT result group and target FFT result group The reference FFT result group is one FFT result group that serves as a starting point for obtaining a difference among the multiple FFT result groups held by the distance measuring unit 221. The target FFT result group is each of one or more FFT result groups that are the targets for obtaining a difference between the reference FFT result group and the multiple FFT result groups held by the distance measuring unit 221. The one or more target FFT result groups are located temporally before or after (usually before) the reference FFT group.

[0044] In this embodiment, one set of FFT results is a current set of FFT results (described below), and one or more sets of target FFT results are one or more sets of previous FFT results (described below).

[0045] By taking the difference between the reference FFT result group and at least one target FFT result group, the reflected components from the stationary object 302 of the reflected wave Re are removed (the reflection intensity, which is the received intensity of the reflected wave Re, becomes below the threshold), and only the reflected components from the human body 301 (moving object) remain (at least one pair, preferably two or more pairs, of a reflection intensity amp exceeding the threshold and a frequency f corresponding to that reflection intensity are detected).

[0046] Note that the number of "at least one" previous FFT result group, i.e., the number of previous FFT result groups from which differences with the current FFT result group are obtained, is preferably two or more (one-to-many inter-frame differences: one-to-five inter-frame differences in the illustrated example) as shown in Figures 10 and 12 in terms of improving resolution, but may be just one. In other words, even when one-to-one differences (one-to-one inter-frame differences) are taken as shown in Figures 8 and 9A to 9C, attitude estimation is possible, and the resolution can also be improved by increasing the number of receiving antennas (hereinafter, "number of antennas").

[0047] (1-2-1b) Current FFT result group and preceding FFT result group The current FFT result group is the most recent FFT result group among the multiple FFT result groups that are stored. The preceding FFT result group is the FFT result group that precedes the current FFT result group among the multiple FFT result groups that are stored. By setting the reference FFT result group as the current FFT result group and the target FFT result group as the preceding FFT result group, posture estimation can be performed in real time.

[0048] (1-2-1c) Distance Measurement Processing Distance measurement processing is processing for measuring (ranging) the distance to an object such as the human body 301 using the radio wave sensor 1. In the distance measurement processing of this embodiment, the distance from each of three or more receiving antennas (hereinafter referred to as each antenna) constituting the radio wave sensor 1 to the human body 301 is measured, and three or more distance measurement results (ranging result groups: described later) corresponding to the three or more receiving antennas are obtained.

[0049] In the distance measurement process, for example, based on the various parameters shown in FIG. 3, namely, the difference (frequency difference Δf) between the transmitted wave Tr and the reflected wave Re, the slope S of the linear change in the frequency f of the transmitted wave Tr, and the speed of light c, the distance d from each antenna to the human body 301 is calculated using the following formula 1: d=(c / 2S)×Δf (Formula 1)

[0050] (1-2-1d) Distance Measurement Result Group The distance measurement result group is information consisting of three or more distance measurement results corresponding to three or more receiving antennas, obtained by the above formula 1 for one transmission / reception operation.

[0051] The three or more distance measurement results corresponding to the three or more receiving antennas are, for example, first to third distance measurement results corresponding to the first to third receiving antennas. The first distance measurement result corresponds to approximately half the propagation distance of the radio waves radiated from the transmitting antenna, reflected by the human body 301, and reaching the first receiving antenna (i.e., the distance from the human body 301 to the first receiving antenna). The second distance measurement result corresponds to approximately half the propagation distance of the radio waves radiated from the transmitting antenna, reflected by the human body 301, and reaching the second receiving antenna (i.e., the distance from the human body 301 to the second receiving antenna). The third distance measurement result corresponds to approximately half the propagation distance of the radio waves radiated from the transmitting antenna, reflected by the human body 301, and reaching the third receiving antenna (i.e., the distance from the human body 301 to the third receiving antenna).

[0052] It should be noted that the number of "at least one" distance measurement result group, that is, the number of distance measurement result groups to be acquired is preferably two or more, but may be one.

[0053] (1-2-2) Point Arrangement Unit: Multi-Point Arrangement The point arrangement unit 222 performs coordinate calculation processing each time the distance measurement unit 221 performs distance measurement processing, and arranges at least one point corresponding to the current FFT result group in three-dimensional space 500, for example, as shown in Fig. 13A. However, the point may be arranged in two-dimensional space (plane) or one-dimensional space (line).

[0054] It should be noted that the number of "at least one" point, i.e., the number of points placed at one time corresponding to the current FFT result group, is preferably two or more (multi-point placement), but may also be one (single-point placement).

[0055] (1-2-2a) Three-dimensional space The three-dimensional space 500 is a virtual space corresponding to the real space (e.g., room 400: see FIG. 2) in which the radio wave sensor 1 and the group of objects (301, 302) exist. Arrangement in the three-dimensional space 500 may be a virtual operation or an operation of simply storing three-dimensional coordinates.

[0056] (1-2-2b) Coordinate Calculation Processing and Point Cloud The coordinate calculation processing is processing for calculating the three-dimensional coordinates of the human body 301 based on at least one acquired group of distance measurement results.

[0057] (1-2-3) Posture Estimation Unit The posture estimation unit 223 estimates the posture of the human body 301 (in this embodiment, whether the posture is standing, sitting, or lying down) based on, for example, a point cloud 501 as shown in Figures 13A to 13C. The point cloud 501 is a collection of one or more points arranged in a three-dimensional space 500 by the point arrangement unit 222.

[0058] As described above, according to this embodiment, it is possible to estimate the posture of the human body 301 using the radio wave sensor 1 based on the FMCW method.

[0059] (2) Details of the Distance Measuring Unit and Point Arrangement Unit (2-1) One-to-Many Difference The distance measuring unit 221 preferably holds the FFT results output by the radio wave sensor 1 for a period of at least twice the predetermined period. The period of at least twice the predetermined period may be, for example, a period of at least twice one frame (2×T).

[0060] Then, the ranging unit 221 calculates the difference between the current FFT result group and each of the two or more previous FFT result groups out of the three or more FFT result groups currently held, and performs ranging processing based on each of the two or more calculated differences, thereby obtaining two or more ranging result groups.

[0061] The point placement unit 222 performs coordinate calculation processing based on each of the two or more distance measurement result groups acquired by the distance measurement unit 221, thereby placing two or more points corresponding to the current FFT result group in three-dimensional space 500.

[0062] In this way, by calculating the difference between the current FFT result group and each of two or more previous FFT result groups, the accuracy of estimating the posture of the human body 301 can be improved without increasing the number of antennas in the radio wave sensor 1.

[0063] (2-2) One-to-Many Difference Suitable for Detecting Various Movements of the Human Body The ranging unit 221 more preferably holds the FFT result groups output by the radio wave sensor 1 for a period of at least three times the predetermined cycle. The ranging unit 221 then calculates the difference between the current FFT result group and each of three or more previous FFT result groups among the four or more FFT result groups currently held, and selects two or more differences from the three or more calculated differences that correspond to various movements of the human body 301 (e.g., body movement, slight breathing movement, hand and foot movement, etc.). The ranging unit 221 performs the ranging process based on each of the two or more differences selected in this manner, thereby acquiring two or more ranging result groups.

[0064] The point placement unit 222 performs the coordinate calculation process based on each of the two or more distance measurement result groups acquired by the distance measurement unit 221, thereby placing two or more points corresponding to the current FFT result group in the three-dimensional space 500.

[0065] In this way, by calculating the difference between the current FFT result and each of three or more previous FFT results and selecting two or more differences corresponding to various movements of the human body 301, it is possible to improve the accuracy of estimating the posture of the human body 301 without increasing the number of antennas and while reducing the number of differences used in the ranging process.

[0066] (2-3) Frame and Inter-Frame Difference A frame is a unit of time for repeatedly performing an operation for detecting a moving object. The above-described transmission and reception operation is performed N times (N is an integer of 2 or more) per frame, with a predetermined time being one frame. In this embodiment, the predetermined time is 200 ms, and N=10.

[0067] The memory stores N sets of FFT results for each frame across multiple frames. Specifically, the radio wave sensor 1 performs, for example, 10 transmission and reception operations per frame, and the memory of the control device 2 stores 10 sets of FFT results for each frame across multiple frames (e.g., six frames). The distance measurement unit 221 obtains inter-frame differences between the multiple frames stored in the memory.

[0068] An inter-frame difference is the difference between a set of FFT results belonging to one frame (e.g., the reference frame Fr0 shown in FIG. 8) and a set of FFT results belonging to one or more other frames (e.g., the subsequent frame Fr1, etc. shown in FIG. 8).

[0069] (2-3-1) One-to-One Interframe Difference The interframe difference is, for example, a one-to-one interframe difference. The one-to-one interframe difference is the difference between a group of FFT results belonging to one frame (e.g., reference frame Fr0) and a group of FFT results belonging to another frame (e.g., subsequent frame Fr1).

[0070] (2-3-2) Representative value of each frame when calculating the difference between frames Each of the two FFT result groups corresponding to the two frames from which the difference is to be obtained (e.g., the FFT result group belonging to the reference frame Fr0 and the FFT result group belonging to the subsequent frame Fr1) is a representative value among the N (e.g., 10) FFT result groups in the frame to which it belongs.

[0071] The representative value is, for example, the average value of N FFT result groups (specifically, information configured of the average value of 10 FFT results corresponding to the first receiving antenna, the average value of 10 FFT results corresponding to the second receiving antenna, the average value of 10 FFT results corresponding to the third receiving antenna, etc.) In this case, the inter-frame difference is the difference between the average value of 10 FFT result groups belonging to the reference frame Fr0 and the average value of 10 FFT result groups belonging to the subsequent frame Fr1.

[0072] Alternatively, the representative value may be one of the N FFT result groups determined according to a predetermined rule (for example, the k-th FFT result group: k is an integer between 1 and N). In this case, the inter-frame difference is the difference between the k-th (for example, the first) FFT result group of the 10 FFT result groups belonging to the reference frame Fr0 and the k-th (for example, the first) FFT result group of the 10 FFT result groups belonging to the subsequent frame Fr1.

[0073] The above-mentioned matters regarding inter-frame differences are not limited to one-to-one inter-frame differences, but also apply to one-to-many inter-frame differences.

[0074] For example, to calculate a one-to-many frame difference (described later), the distance measurement unit 221 holds the group of FFT results output by the radio wave sensor 1 for a period of (K+1) frames (K is an integer equal to or greater than 2). In this embodiment, K=5, and the group of FFT results is held (for example, stored in the memory of the control device 2) for a period of (5+1) frames, that is, 6×200 ms=1200 ms.

[0075] (2-3-3) One-to-Many Frame Difference The distance measurement unit 221 calculates the one-to-many frame difference using, for example, a group of FFT results spanning a (K+1) frame period stored in memory. The one-to-many frame difference is the difference (first time difference T, second time difference 2×T, ...) between the reference frame Fr0 and each of multiple target frames (for example, subsequent frames Fr1, Fr2, ... as shown in FIG. 10 ).

[0076] In Figure 10, the multiple target frames are illustrated as multiple subsequent frames Fr1, Fr2, etc. that follow the reference frame Fr0, but the multiple target frames in this embodiment are, for example, multiple preceding frames Fr-1, Fr-2, etc. that precede the reference frame Fr0, as shown in Figure 12.

[0077] The difference between the FFT result groups is, for example, as shown in Figure 8, the difference between the FFT result group corresponding to the first transmission wave Tr1 of the reference frame Fr0 and the FFT result group corresponding to the first transmission wave Tr1 of the target frame (in the example of Figure 8, the subsequent frame Fr1) that has a time difference T from the reference frame Fr0.

[0078] Alternatively, the difference between the FFT result groups may be, for example, the difference between the FFT result group corresponding to the second transmission wave Tr2 of the reference frame Fr0 and the FFT result group corresponding to the second transmission wave Tr2 of the target frame (subsequent frame Fr1), or the difference between the FFT result group corresponding to the Nth transmission wave TrN of the reference frame Fr0 and the FFT result group corresponding to the Nth transmission wave Tr2 of the target frame (subsequent frame Fr1).

[0079] Alternatively, the difference of the FFT result group may be the sum of the above-mentioned N differences. In this embodiment, the difference of the FFT result group is the sum of the above-mentioned N differences, that is, the sum of the differences for each of the N transmission waves Tr1 to TrN.

[0080] In this embodiment, the one-to-many frame differences are a set of inter-frame differences of the current frame FFT result group for each of K or more (e.g., 5) previous frame FFT result groups out of (N×(K+1)) or more (e.g., 10×(5+1)=60, assuming N=10 and K=5) FFT result groups currently held.

[0081] (2-3-4) Current Frame and Current Frame FFT Result Group The current frame Fr0 is the frame Fr0 that includes the most recent FFT result group. The current frame FFT result group is a set of N FFT result groups that belong to the current frame Fr0.

[0082] (2-3-5) Previous Frame and Previous Frame FFT Result Group Previous frames (Fr-1, Fr-2, ... Fr-K) are frames that precede the current frame Fr0. The previous frame FFT result group is a set of N FFT result groups that belong to K previous frames (Fr-1, Fr-2, ... Fr-K), respectively.

[0083] (2-3-6) Parameter K The larger the value of the parameter K, the easier it is to detect various movements of the human body 301 (for example, body movement, slight breathing movement, and movement of hands and feet).

[0084] (2-3-6a) Body Movement, Respiratory Fluctuations, and Limb Movement Body movement refers to the movement of the entire body (trunk). Body movement is shown by MV1 in FIG. 11 and, as shown in FIG. 12, is irregular, with a long movement time and a large amount of movement. Limb movement refers to the movement of the limbs. Limb movement is shown by MV3 in FIG. 11 and, as shown in FIG. 12, is irregular, with a short movement time and a large amount of movement. Respiratory fluid movement refers to the movement of the trunk associated with breathing. Respiratory fluid movement is shown by MV2 in FIG. 11 and, as shown in FIG. 12, is cyclic, with a small amount of movement and a somewhat short movement time.

[0085] (2-3-6b) Specific examples of parameter K In this embodiment, as shown in FIG. 12, K=5, and of the five consecutive preceding frames (first preceding frame Fr-1, second preceding frame Fr-2, ... fifth preceding frame Fr-5), three frames (second, third and fifth preceding frames Fr-2, Fr-3 and Fr-5) that are suitable for detecting body movement, limb movement and slight respiratory movement are selected, thereby reducing the amount of processing required for calculating the difference and improving detection accuracy, but all five preceding frames Fr-1 to Fr-5 may also be used.

[0086] (2-3-7) Posture Estimation Based on One-to-Many Frame Differences The distance measurement unit 221 acquires K or more distance measurement result groups corresponding to K or more inter-frame differences that constitute the one-to-many frame differences calculated for the current frame Fr0 in this way.

[0087] The point placement unit 222 thus performs coordinate calculation processing based on each of the K or more distance measurement result groups obtained by the distance measurement unit 221 for the current frame Fr0, thereby placing K or more points corresponding to the current FFT result group in three-dimensional space 500.

[0088] In this way, by calculating the one-to-many frame difference of the current FFT result group for each of two or more previous FFT result groups, it is possible to further improve the estimation accuracy of the posture of the human body 301 without increasing the number of antennas.

[0089] (2-3-8) Preferred Value of Parameter K The value of K is preferably an integer equal to or greater than 3. The distance measurement unit 221 selects two or more inter-frame differences corresponding to various movements of the human body 301 (for example, body movement, slight respiratory movement, hand and foot movement, etc.) from among the three or more inter-frame differences that constitute the one-to-many inter-frame difference calculated for the current frame Fr0, and obtains two or more groups of distance measurement results corresponding to the two or more selected inter-frame differences.

[0090] The point placement unit 222 thus performs coordinate calculation processing based on each of the two or more distance measurement result groups obtained by the distance measurement unit 221 for the current frame Fr0, thereby placing two or more points corresponding to the current FFT result group in three-dimensional space 500.

[0091] In this way, by calculating three or more inter-frame differences (one-to-many inter-frame differences) for each of three or more preceding FFT result groups of the current FFT result group, and calculating two or more inter-frame differences among the three or more inter-frame differences according to various movements of the human body 301, it is possible to further improve the estimation accuracy of the posture of the human body 301 without increasing the number of antennas and while suppressing the number of inter-frame differences used in the ranging process.

[0092] A more preferable value of K is 5. The distance measurement unit 221 selects three inter-frame differences corresponding to the body movement, respiratory micromovement, and limb movement of the human body 301 from the five inter-frame differences that make up the one-to-many inter-frame difference calculated for the current frame Fr0, and obtains three groups of distance measurement results corresponding to the selected three inter-frame differences.

[0093] The three inter-frame differences corresponding to the body movement, respiratory micromovement, and limb movement of the human body 301 are, for example, the second, third, and fifth inter-frame differences ΔA2, ΔA3, and ΔA5 out of the first to fifth inter-frame differences ΔA1 to ΔA5, as shown in Figure 12.

[0094] The point placement unit 222 performs coordinate calculation processing based on each of the three groups of distance measurement results obtained by the distance measurement unit 221 for the current frame, thereby placing three points corresponding to the current group of FFT results in three-dimensional space 500.

[0095] In this way, by calculating five inter-frame differences (one-to-many inter-frame differences) for each of the five preceding FFT result groups of the current FFT result group, and calculating three of the five inter-frame differences corresponding to the body movement, respiratory micromovement, and limb movement of the human body 301, it is possible to further improve the estimation accuracy of the posture of the human body 301 without increasing the number of antennas and while suppressing the number of inter-frame differences used in the ranging process.

[0096] Note that the above is merely an example, and for example, the number of inter-frame differences to be calculated and the time difference can be changed as appropriate to increase the estimation accuracy.

[0097] (2-3-9) Details of the Posture Estimation Unit (2-3-9a) Clustering and Distribution of Cluster Groups The posture estimation unit 223, for example, performs clustering on the point cloud 501 to obtain a cluster group (CL, CL1, CL2) which is a collection of one or more clusters (CL, CL1, CL2).

[0098] The cluster group (CL, CL1, CL2) may be, for example, a first cluster CL1 corresponding to the upper body (head, torso, and arms) and a second cluster CL2 corresponding to the lower body (legs) of a human body 301, as shown in Figures 13A and 13B. Alternatively, the cluster group (CL, CL1, CL2) may be, for example, a single cluster CL corresponding to the entire body (head, torso, arms, and legs), as shown in Figure 13C.

[0099] Furthermore, as the number of points constituting the point cloud 501 increases, it is expected that the cluster group (CL, CL1, CL2) will have a resolution sufficient to distinguish the general shape (silhouette) of the human body 301 and, ultimately, each part constituting the human body 301.

[0100] The posture estimation unit 223 estimates the posture of the human body 301 based on the distribution of the thus acquired cluster group (CL, CL1, CL2) in the three-dimensional space 500 (hereinafter simply referred to as "distribution").

[0101] The distribution may be, for example, the number of acquired clusters (CL, CL1, CL2), the direction of spread of one cluster (CL, CL1, CL2), the distance between multiple clusters, and the like.

[0102] (2-3-9b) Posture Estimation Based on Distribution Conditions The posture estimation unit 223 estimates whether the posture of the human body 301 is lying down, sitting, or standing, based on distribution conditions, which are conditions related to distribution, for example.

[0103] The distribution conditions include, for example, a lying-down condition. A lying-down condition (first lying-down condition) constituting the distribution conditions is, for example, "only a single cluster CL is acquired, and the height of the single cluster CL from the floor surface is low enough to be below a threshold" (see FIG. 13C).

[0104] The distribution condition further includes, for example, a locus condition. The locus condition (first locus condition) constituting the distribution condition is, for example, "two clusters, ie, first and second clusters CL1 and CL2, are obtained, and the distance between the two clusters CL1 and CL2 is so close that it is below a threshold" (see FIG. 13B).

[0105] The posture estimation unit 223 may, for example, determine that the posture of the human body 301 is lying down if the distribution satisfies the above-mentioned lying down condition (first lying down condition), determine that the posture is sitting down if the distribution satisfies the above-mentioned sitting down condition (first sitting down condition), and determine that the posture is standing up if the distribution satisfies neither the above-mentioned lying down condition nor the above-mentioned sitting down condition.

[0106] In this way, the posture of the human body 301 can be estimated with high accuracy based on the distribution of the cluster group (CL, CL1, CL2).

[0107] (2-3-9c) Posture estimation based on size ratio conditions Alternatively, the posture estimation unit 223 may estimate the posture of the human body 301 based on size ratio conditions regarding the ratios (hereinafter referred to as "size ratios") of the length D, width W, and height H of the three-dimensional figure 502 surrounding the cluster group (CL, CL1, CL2).

[0108] The three-dimensional figure 502 in this embodiment is, for example, a rectangular parallelepiped with length D, width W, and height H as shown in Figures 14A to 14C, and the dimensional ratio is, for example, the ratio between length D or width W and height H.

[0109] However, the three-dimensional figure 502 may be, for example, an elliptical cylinder (not shown), etc. In an elliptical cylinder, the minor axis corresponds to the vertical direction D and the major axis corresponds to the horizontal direction W.

[0110] The size ratio condition includes a lying-down condition. A lying-down condition (second lying-down condition) constituting the size ratio condition may be, for example, "the length D or width W of the three-dimensional figure 502 must be larger than a predetermined ratio (e.g., three times) of the height H."

[0111] The size ratio condition further includes, for example, a standing condition. The standing condition constituting the size ratio condition may be, for example, "the height H of the three-dimensional figure 502 must be greater than a predetermined ratio (e.g., three times) of the length D or width W."

[0112] The posture estimation unit 223 may, for example, determine that the posture of the human body 301 is lying down if the dimension ratio satisfies the above-mentioned lying down condition (second lying down condition), determine that the posture is standing if the dimension ratio satisfies the above-mentioned standing up condition, and determine that the posture is sitting if the dimension ratio satisfies neither the lying down condition nor the standing up condition.

[0113] In this way, the posture of the human body 301 can be estimated with high accuracy based on the ratios of the length D, width W, and height H of the three-dimensional figure 502 surrounding the cluster group (CL, CL1, CL2).

[0114] (2-3-9d) Posture estimation based on distribution size ratio conditions The posture estimation unit 223 may estimate whether the posture of the human body 301 is lying down, sitting, or standing, based on distribution size ratio conditions, which are conditions related to distribution and size ratios, for example.

[0115] The distribution size ratio condition includes a lying-down condition. A lying-down condition (third lying-down condition) constituting the distribution size ratio condition may be, for example, "only one cluster CL is acquired, and the length D or width W of the three-dimensional figure 502 is so large as to exceed a predetermined ratio (e.g., three times) of the height H."

[0116] The distribution size ratio condition further includes, for example, a locus condition. The locus condition (second locus condition) constituting the distribution size ratio condition may be, for example, "first and second clusters CL1 and CL2 are obtained, and the distance between the two clusters CL1 and CL2 is so close that it is below a threshold value."

[0117] The posture estimation unit 223 may, for example, determine that the posture of the human body 301 is lying down if the distribution and size ratio satisfy the above-mentioned lying down condition (third lying down condition), determine that the posture is sitting down if the distribution and size ratio satisfy the above-mentioned sitting down condition (second sitting down condition), and determine that the posture is standing up if the distribution and size ratio satisfy neither the above-mentioned lying down condition nor the above-mentioned sitting down condition.

[0118] (2-4) Device Control Function of Device Control System As shown in FIG. 1, the device control system 100 further includes a behavior detection unit 224 and a device control unit 225.

[0119] (2-4-1) Behavior Detection Unit The behavior detection unit 224 detects the behavior of the person corresponding to the human body 301 based on the estimation result of the posture estimation unit 223 for the human body 301 .

[0120] The behavior detection unit 224 in this embodiment detects non-operation behavior based at least on a change in the estimation result of the posture estimation unit 223 in particular.

[0121] In this embodiment, the change in the estimation result is a change between a standing position, a sitting position, and a lying position, and specifically, for example, a change from a standing position to a sitting position, a change from a sitting position to a standing position, a change from a standing position or a sitting position to a lying position, a change from a lying position to a standing position or a sitting position, etc.

[0122] Furthermore, the behavior detection unit 224 may detect a non-operation behavior when an unchanged state in which no change in the estimation result of the posture estimation unit 223 is detected continues for a predetermined time or more. The behavior detection unit 224 detects a non-operation behavior, for example, in response to an unchanged state continuing for a predetermined time or more after a change in the estimation result of the posture estimation unit 223 is detected.

[0123] Specifically, the behavior detection unit 224 detects a non-operation behavior of a continuation of a lying state (for example, falling asleep) in response to, for example, a change from a standing or sitting position to a lying position in which no change from the lying position to another position is detected continuing for a predetermined time or more after the estimation result of the posture estimation unit 223 has changed from a standing or sitting position to a lying position. Also, the behavior detection unit 224 detects a non-operation behavior of a continuation of a lying state (for example, falling asleep) in response to, for example, a change from a standing or lying position to a sitting position in which no change from the sitting position to another position is detected continuing for a predetermined time or more after the estimation result of the posture estimation unit 223 has changed from a standing or lying position to a sitting position.

[0124] (2-4-1a) Non-operational behavior and operational behavior Non-operational behavior is a behavior that is different from operational behavior.

[0125] An operation behavior is an action intended to operate device 200. Device 200 is, for example, a lighting fixture 200a attached to the ceiling of room 400, a television (TV) 200b placed on the floor, and an electric blind 200c attached to window 402, as shown in Fig. 2. An operation behavior is, for example, an on / off operation of lighting fixture 200a via a wall switch, an on / off operation or channel change operation of TV 200b via a remote control, and an open / close operation of electric blind 200c via a remote control.

[0126] The non-operational behavior is, for example, a daily behavior that does not intend to operate the device 200. The daily behavior is, for example, a behavior of entering the room 400 (entering the room 400 through the entrance / exit 401), a behavior of lying down on the bed 302b (going to sleep), a behavior of getting up from the bed 302b (getting up), a behavior of leaving the room 400 (leaving the room through the entrance / exit 401), etc. Furthermore, the daily behavior may also be, for example, a behavior of sitting at the desk 302a (sitting) (desk work or relaxation), a behavior of leaving the desk 302a (leaving one's seat), etc.

[0127] Note that sitting may be distinguished, for example, into sitting to work at the desk 302a (desk work) and sitting to relax and watch the TV 200b (relaxation). Whether it is desk work or relaxation may be determined based on, for example, the time period (daytime or nighttime) in which sitting is detected.

[0128] The behavior detection unit 224 may detect non-operation behavior by taking into consideration environmental information, history information, etc. (described later). For example, waking up may be distinguished into waking up in the morning and waking up in the middle of the night. Whether it is waking up in the morning or waking up in the middle of the night may be determined based on the time period when a change from a lying position to a sitting or standing position is detected.

[0129] (2-4-2) Device Control Unit The device control unit 225 controls the device 200 based at least on the non-operation behavior detected by the behavior detection unit 224 .

[0130] Note that the device control unit 225 performs control (manual control) in response to an operation on the device 200, in addition to control (automatic control) in response to a non-operational behavior. The operation may be, for example, an operation of an operation device (touch panel, operation button, etc.) possessed by the control device 2, but may also be an operation of an operation device (wall switch, remote control, etc.) possessed by the device 200, or a gesture for operating the device 200. The operation of the operation device possessed by the device 200 is detected, for example, by intercepting a control signal sent from an operation device such as a wall switch to the device 200. The gesture is detected, for example, by analyzing an image of the human body 301 captured by a camera (not shown), but may also be detected by the radio wave sensor 1, distinguished from a non-operational behavior.

[0131] In this way, by detecting the behavior (particularly non-operational behavior) of the human body 301 based on the estimated posture change and performing equipment control according to the detection results, it is possible to control the equipment 200 without operating it (operation-less equipment control).

[0132] (2-4-3) Information Acquisition Unit As shown in Fig. 1, the device control system 100 further includes an information acquisition unit 226. The information acquisition unit 226 acquires various types of information, such as environmental information and a person identifier (both of which will be described later).

[0133] The information acquisition unit 226 acquires, for example, environmental information. The environmental information is information about the environment in which the human body 301 is present. The environment may be, for example, the time of day (daytime / nighttime, early morning / late night, etc.), weather (sunny / rainy, etc.), temperature, humidity, etc., but may also be the type of room 400 in which the human body 301 is present (bedroom / living room) or the direction (south-facing / north-facing), etc.

[0134] In addition, when multiple people (two or more human bodies 301) may exist in the same environment, the information acquisition unit 226 may acquire a person identifier. In detail, for example, a set of group information (group of group information) is stored in advance in a memory, which is a pair of a person identifier that identifies a person and feature information that indicates the characteristics of the person. The person identifier is, for example, an ID such as "1" or "2," but may be any information that can identify a person, such as a name or an email address. The feature information paired with the person identifier is, for example, height information that indicates the height of the person. However, the feature information may be speed information that indicates the walking speed of the person, or any information that indicates a feature that can distinguish the person from others. The feature information itself may be used as the person identifier.

[0135] When the posture estimation unit 223 detects the human body 301, the information acquisition unit 226 may acquire feature information of the human body 301 from the posture estimation unit 223, and read out from memory a person identifier that pairs with the acquired feature information.

[0136] The automatic control information (see FIG. 15) described later may be customized for each of a plurality of person identifiers.

[0137] The behavior detection unit 224 detects non-operation behavior further based on the environmental information acquired by the information acquisition unit 226 .

[0138] In this way, by using environmental information in addition to posture changes, it is possible to improve the accuracy of detecting non-operation behavior.

[0139] (2-4-3a) Automatic Control and Automatic Control Information The device control unit 225 controls the device 200 based on the non-operation behavior, for example, by using one or more (six in this case) pieces of automatic control information (automatic control information group) as shown in FIG. 15. The automatic control information group is stored in advance in the memory of the control device 2, for example. In this embodiment, control based on the automatic control information group is referred to as "automatic control."

[0140] Each of the one or more pieces of automatic control information constituting the automatic control information group includes information regarding a status change, a posture change, an environment, a non-operational action, a control content, and a control ID, as shown in Fig. 15. Specifically, of the six pieces of automatic control information shown in Fig. 15, the first piece of automatic control information includes a status change of "absent → present", a posture change of "-", an environment of "-", a non-operational action of "enter", a control content of "lights normally on (brightness 5)", and a control ID of "1". Note that "-" indicates that the information is not included (the same applies below).

[0141] Similarly, the second automatic control information includes a status change "-", a posture change "standing → sitting", an environment "daytime", a non-operation action "desk work", a control content "brighten the lights (+2)", and a control ID "2". The third automatic control information includes a status change "-", a posture change "standing → sitting", an environment "nighttime", a non-operation action "relaxing", a control content "TV on", and a control ID "3". The fourth automatic control information includes a status change "-", a posture change "standing or sitting → lying down", an environment "nighttime", a non-operation action "sleeping", a control content "lights off", and a control ID "4". The fifth automatic control information includes a status change "-", a posture change "lying down → standing or sitting", an environment "late at night", a non-operation action "awakening mid-evening", a control content "lights dim (brightness 2)", and a control ID "5". The sixth automatic control information includes a state change "-", a posture change "lying down to standing or sitting", an environment "morning", a non-operational action "wake up", a control content "open blinds", and a control ID "6".

[0142] (2-4-3b) Manual Control The device control unit 225 can also perform control (automatic control) using the automatic control information in response to an operation (hereinafter simply referred to as an “operation”) on the device 200. Control based on an operation is referred to as “manual control” in this embodiment.

[0143] (2-4-4) History Storage and Learning The device control system 100 further includes a history storage unit 227 and a learning unit 228.

[0144] (2-4-4a) History Accumulation Unit The history accumulation unit 227 accumulates control history information. The control history information is information relating to the control history of the device 200 performed by the device control unit 225. The control history information is, for example, automatic control history information, but may also be manual control history information. Alternatively, the control history information may include automatic control history information and manual control history information.

[0145] The automatic control history information is information about the control history of the device 200 in response to an action (for example, a non-operational action, but it may also be an operational action). The automatic control history information associates the control content of the action-based control with the time when the control was performed, environmental information, and a person identifier.

[0146] The manual control history information is information relating to the control history of the device 200 in response to an operation. The manual control history information associates the control content of the device 200 based on an operation with one or more pieces of information selected from the time the operation was performed, environmental information, and a person identifier.

[0147] When the device control unit 225 controls the device 200, the history accumulation unit 227 accumulates, for example, control history information as shown in Fig. 16 in the memory of the control device 2. Note that the control history information may be accumulated in an external memory.

[0148] For example, if, as of 9:02, device control unit 225 has executed three control operations on lighting device 200a, three pieces of control history information are stored in the memory of control device 2, as shown in Fig. 16. Each of the three pieces of control history information includes information regarding a history ID, a time, a control ID, and an operation.

[0149] Of the three pieces of control history information shown in FIG. 16, the first piece of control history information includes a history ID of "1", a time of "9:00", a control ID of "1", and an operation of "-". The second piece of control history information includes a history ID of "2", a time of "9:01", a control ID of "2", and an operation of "-". The third piece of control history information includes a history ID of "3", a time of "9:02", a control ID of "-", and an operation of "operation to dim the lights slightly (-1)".

[0150] The aforementioned automatic control history information has operation information "-", which corresponds to the first and second control history information of the three control history information. The aforementioned manual control history information has control ID "-", which corresponds to the third control history information of the three control history information.

[0151] (2-4-5) Learning Unit The learning unit 228 updates the automatic control information when the time and control content indicated by the automatic control history information and the time and operation indicated by the manual control history information satisfy predetermined update conditions.

[0152] (2-4-5a) Update conditions and updating of information for automatic control based thereon An update condition is, for example, "after automatic control based on non-operational behavior is performed on a device, an operation to cancel the control content of the automatic control is accepted within a predetermined time (for example, 2 minutes)."

[0153] In the example of Figure 16, the time "9:01" and control content (i.e., the control content corresponding to control ID "2" in Figure 15) "brighten the lights (+2)" indicated by the automatic control information, which is the second control history information, and the time "9:03" and operation "slightly dim the lights (-1)" indicated by the manual control information, which is the third control history information, satisfy the above update condition.

[0154] In other words, after the automatic control "brighten the lights (+2)" based on the non-operational behavior "desk work" was performed on one device "lighting device 200a," an operation "dimming the lights slightly (-1)" that cancels the automatic control was received within two minutes, and therefore the learning unit 228 determines that the update condition is satisfied.Then, of the six pieces of automatic update information in FIG. 15, the learning unit 228 updates the second piece of automatic control information corresponding to control ID "2" to, for example, "dimming the lights slightly (+1)."

[0155] Alternatively, the update condition may be, for example, "an operation different from the content of automatic control is received a predetermined number of times or more within a predetermined period (e.g., "two or more times a day," "three or more times a week," etc.)."

[0156] In this way, by accumulating historical information regarding automatic control in response to non-operational behavior (automatic control historical information) and historical information regarding manual control in response to operation, and updating the information for automatic control based on these two pieces of historical information when the update conditions are met, it is possible to realize a learning function.

[0157] (3) Specific Example of Device Control System As shown in Fig. 1, the device control system 100 of this example includes a radio wave sensor 1 and a control device 2. The radio wave sensor 1 and the control device 2 each have a communication module and are connected to each other so that they can communicate with each other via wired or wireless communication. The control device 2 also has a processor and memory. Programs and various information are stored in the memory, and the processor operates based on the programs and the like in the memory (and cooperates with the communication module), thereby realizing the operation of the control device 2 as described below. Note that the radio wave sensor 1 may also have a processor and memory, and transmission and reception operations, communication with the control device 2, and the like may be performed under the control of a program.

[0158] 2, the radio wave sensor 1 is provided on the ceiling of a room 400. The control device 2 is, for example, a multi-remote control device, and is provided on a side wall of the room 400 as shown in FIG.

[0159] The control device 2 estimates the posture (standing, sitting, or lying down) of the human body 301 present in the room 400 based on the output of the radio wave sensor 1. The control device 2 then detects non-operational behavior of the human body 301 based on changes in the estimated posture, and automatically controls the multiple devices 200 (lighting fixture 200a, TV 200b, and electric blinds 200c) according to the detection result. The control device 2 also has an operation device such as a touch panel, and can manually control the multiple devices 200 according to operation.

[0160] 2 , the control device 2 includes a reception unit 21, a processing unit 22, and an output unit 23. The processing unit 22 includes a distance measurement unit 221, a point placement unit 222, a posture estimation unit 223, a behavior detection unit 224, a device control unit 225, an information acquisition unit 226, a history accumulation unit 227, and a learning unit 228.

[0161] The receiving unit 21 receives various types of information. The various types of information include, for example, operation information for the device 200. The receiving unit 21 may also receive information other than operation information, such as personal information such as the height of the human body 301.

[0162] The processing unit 22 performs various types of processing, such as processing by a distance measurement unit 221, processing by a point placement unit 222, processing by a posture estimation unit 223, processing by a behavior detection unit 224, processing by a device control unit 225, processing by an information acquisition unit 226, processing by a history accumulation unit 227, and processing by a learning unit 228. The processing unit 22 also performs various types of determinations that will be explained in the flowcharts.

[0163] Specifically, the processing unit 22 may execute, for example, a holding process, an acquisition process, a position identification process, a human body estimation process, and a placement process. The processing unit 22 executes this series of processes, for example, every time position identification information is output from the radio wave sensor 1, but may also execute these processes every time a number of position identification information corresponding to one frame is output.

[0164] The retention process is a process of retaining the location identifiable information output by the radio wave sensor 1. The acquisition process is a process of acquiring the difference between multiple pieces of retained location identifiable information. The location determination process is a process of determining the position of a moving object (301, 200c) among one or more objects (301, 302, 200c) constituting a group of objects (301, 302, 200c) based on the difference, and acquiring location information indicating the determined position. The human body detection process is a process of detecting a human body 301 using a signal received by the radio wave sensor. The human body estimation process is a process of estimating, for example, whether the acquired location information corresponds to a human body 301 or a moving object other than the human body 301 (e.g., electric blinds 200c) based at least on a change in the difference. Note that, in this embodiment, the change typically refers to a change in a value, such as a difference, over time, and may be referred to as a "temporal change." However, the change may also be, for example, a change in a value according to a position in space (spatial change).

[0165] The placement process is a process of virtually placing a point corresponding to the acquired position information in space (three-dimensional space 500) when it is estimated that the acquired position information corresponds to the human body 301. The placement process is executed by a point placement unit 222 constituting the processing unit 22.

[0166] Particularly in this embodiment, the storage process stores three or more pieces of location identifiable information. The acquisition process acquires two or more differences with different time differences for the three or more pieces of location identifiable information stored. The location identification process acquires two or more pieces of location information corresponding to the two or more acquired differences. The human body detection process, for example, estimates whether each of the two or more pieces of acquired location information corresponds to a human body 301 or a moving object (200c) other than the human body 301. The placement process virtually places in space (500) a point corresponding to the position information estimated to correspond to the human body 301 among the two or more pieces of acquired location information.

[0167] In addition, the placement process may receive the estimation results of the human body detection process and display points corresponding to objects estimated to be human bodies 301 and points corresponding to objects estimated to be moving objects (200c) other than human bodies 301 in a manner that allows them to be distinguished from each other via the output unit 23 (for example, by displaying them in different colors, different sizes, etc.).

[0168] The processing unit 22 in this embodiment further executes device control processing. The device control processing is processing for controlling the device 200 based at least on a point cloud 501, which is a collection of multiple points arranged in a space (500). The human body detection processing estimates the posture or posture change of the human body 301 based on the distribution of the point cloud 501.

[0169] The processing unit 22 may further execute a behavior detection process. The behavior detection process is a process for detecting a behavior of a person corresponding to the human body 301 based on the estimated posture or a change in posture. The device control process controls the device 200 based on the detected behavior, for example.

[0170] The processing unit 22 may further execute information acquisition processing. The information acquisition processing is processing for acquiring environmental information related to the environment in which the human body 301 exists. The human body detection processing detects behavior based on the environmental information acquired by the information acquisition processing.

[0171] The device control process controls the device 200 based on the behavior, for example, by using pre-stored automatic control information.

[0172] The processing unit 22 may further execute a history accumulation process. The history accumulation process is a process of accumulating history information related to the control history of the device 200. The history information is, for example, automatic control history information. Alternatively, the history information may be manual control history information. The history accumulation process preferably accumulates both automatic control history information and manual control history information, for example, but may accumulate only one of them.

[0173] The processing unit 22 may further execute a learning process. The learning process is a process of updating the automatic control information when the time and operation indicated by the manual control history information satisfy predetermined update conditions. Alternatively, the learning process may be a process of updating the automatic control information when the time and control content indicated by the automatic control history information and the time and operation indicated by the manual control history information satisfy predetermined update conditions.

[0174] The radio wave sensor 1 may execute the conversion process described in "(10) Second Modification of Device Control System," and the processing unit 22 may execute the storage process, difference acquisition process, and moving object detection process described in "(10) Second Modification of Device Control System." In other words, the radio wave sensor 1 may include the conversion unit 11 shown in Fig. 17, and the processing unit 22 may include the conversion unit 11, storage unit 12, difference acquisition unit 13, and moving object detection unit 14 shown in Fig. 17.

[0175] The output unit 23 outputs various types of information. The various types of information are, for example, control information (e.g., a remote control signal) to the device 200. The output here is usually transmission to the device 200, but may also include display on a display.

[0176] (4) Example of Operation of Device Control System The control device 2 constituting the device control system 100 operates, for example, according to the flowcharts of FIGS.

[0177] (4-1) Overall Processing The processing in FIG. 4 is started when the device control system 100 is started, and is ended when the operation is stopped.

[0178] First, the processing unit 22 constituting the control device 2 determines (step S1) whether or not an FFT result group has been output from the radio wave sensor 1. If it is determined that an FFT result group has not been output, the process proceeds to step S13.

[0179] If it is determined in step S1 that the FFT result group has been output, the distance measurement unit 221 constituting the processing unit 22 holds the FFT result group (step S2).

[0180] Next, the distance measurement unit 221 determines whether or not a group of FFT results for (K+1) frame periods or more has been stored (step S3). If it is determined that a group of FFT results for (K+1) frame periods or more has not yet been stored, the process returns to step S1.

[0181] If it is determined in step S3 that a group of FFT results for at least (K+1) frame periods has been stored, the distance measurement unit 221 performs a one-to-multiple frame difference acquisition process (step S4). The one-to-multiple frame difference acquisition process will be described with reference to the flowchart in FIG. 5.

[0182] Next, the distance measurement unit 221 acquires L (or K) distance measurement results (distance measurement result group) corresponding to the L (or K) differences (difference group) (step S5).

[0183] Next, the point arrangement unit 222 calculates L (or K) three-dimensional coordinates (a group of three-dimensional coordinates) based on the group of distance measurement results acquired in step S5 (step S6).

[0184] Next, the point arrangement unit 222 arranges L (or K) points (point group 501) in the three-dimensional space 500 based on the three-dimensional coordinate group acquired in step S6 (step S7).

[0185] It is determined whether or not a determination start condition is satisfied (step S8). The determination start condition is, for example, a condition that a predetermined number or more points are arranged in the three-dimensional space 500. If it is determined that the determination start condition is not yet satisfied, the process returns to step S1.

[0186] If it is determined in step S8 that the determination start condition is satisfied, the posture estimation unit 223 performs posture estimation processing (step S9). The posture estimation processing will be described with reference to the flowchart in FIG.

[0187] Next, the behavior detection unit 224 uses the automatic control information (see FIG. 15 ) to detect a non-operation behavior based on changes in the estimation results in step S9 (for example, a change in state, a change in posture, or an environment) (step S10). For example, in response to a change in state from "absent to present," a non-operation behavior of "entering" is detected. In addition, in response to a change in posture from "standing to sitting," it is determined whether the environment (current time zone) is daytime or nighttime, and in the case of daytime, a non-operation behavior of "desk work" is detected, and in the case of nighttime, a non-operation behavior of "relaxing" is detected.

[0188] Next, device control unit 225 performs device control (automatic control) in response to the detection results of the non-operational behavior, etc., in step S10 (step S11). For example, in response to the detection of the non-operational behavior "entering a room," automatic control (control with control ID "1") is performed to turn on lighting device 200a at normal brightness (brightness 5). In addition, in response to the detection of the non-operational behavior "desk work," automatic control (control with control ID "2") is performed to brighten lighting device 200a (brightness +2). In addition, in response to the detection of the non-operational behavior "relaxing," automatic control (control with control ID "3") is performed to turn on TV 200b.

[0189] Next, the history accumulation unit 227 accumulates automatic control history information relating to the automatic control performed in step S11 (step S12), after which the process returns to step S1.

[0190] If it is determined in step S1 that the FFT results have not yet been output, the processing unit 22 determines whether the receiving unit 21 has received an operation for the device 200 (step S13). If it is determined that the receiving unit 21 has not yet received an operation for the device 200, the processing returns to step S1.

[0191] If it is determined in step S13 that accepting unit 21 has accepted an operation for device 200, device control unit 225 performs device control (manual control) in response to the operation (step S14). For example, if an operation to slightly dim lighting device 200a (brightness -1) is accepted, manual control to slightly dim lighting device 200a is performed.

[0192] Next, the history accumulation unit 227 accumulates manual control history information relating to the manual control performed in step S14 (step S15).

[0193] Next, the learning unit 228 determines whether the update condition is satisfied (step S16). If it is determined that the update condition is not satisfied, the process returns to step S1.

[0194] If it is determined that the update condition is met, history accumulation unit 227 updates the automatic control information (step S17). For example, as shown in FIG. 16, one minute after the automatic control with control ID "2," manual control is performed to slightly dim lighting fixture 200a. This determines that the update condition is met, and the control content of the automatic control with control ID "2" is updated to "brighten the lights (+1)." Then, processing returns to step S1.

[0195] 4, the radio wave sensor 1 may output three or more IF signals (received signals) corresponding to three or more receiving antennas instead of the FFT result group (location identifiable information), and the processing unit 22 may acquire an FFT transformation result based on the three or more output IF signals. In this case, in step S1, the processing unit 22 may perform an FFT transformation on each of the three or more IF signals and determine whether or not the FFT transformation result has been acquired.

[0196] (4-2) One-to-Many Frame Difference Acquisition Processing The one-to-many frame difference acquisition processing in step S4 is executed, for example, according to the flowchart of FIG.

[0197] The processing unit 22 sets the initial value "1" to a variable i (step S41).

[0198] Next, the processing unit 22 determines whether or not the variable i is greater than K (step S42). If it is determined that the variable i is greater than K, the processing proceeds to step S45.

[0199] If it is determined in step S42 that the variable i is not greater than K (i.e., is less than or equal to K), the distance measurement unit 221 obtains the difference between the current FFT result group and the i-th preceding FFT result group, which is the i-th time difference (= i x T seconds) before the current FFT result group, and sets the obtained difference to the variable "i-th difference" (step S43).

[0200] Next, the processing unit 22 increments the variable i (step S44), after which the processing returns to step S42.

[0201] If it is determined in step S42 that the variable i is greater than K, the distance measurement unit 221 selects L differences (for example, three differences ΔA1, ΔA2, and ΔA3: see FIG. 12) corresponding to various movements of the human body 301 from the first to Kth K differences (for example, five differences ΔA1 to ΔA5: only some of which are shown in FIG. 12) (step S45). Thereafter, the process returns to the upper flowchart (see FIG. 4).

[0202] (4-3) Attitude Estimation Processing The attitude estimation processing in step S9 is executed, for example, according to the flowchart of FIG.

[0203] Posture estimation section 223 performs clustering on point group 501 arranged in three-dimensional space 500 to obtain cluster groups (CL, CL1, CL2: see FIGS. 13A to 13C) (step S91).

[0204] Next, the posture estimation unit 223 places a solid figure (rectangular parallelepiped) 502 (see FIGS. 14A to 14C) surrounding the group of clusters (CL, CL1, CL2) acquired in step S91 in three-dimensional space 500 (step S92).

[0205] Next, the posture estimation unit 223 acquires the ratio (dimension ratio) of the length D, width W, and height K of the three-dimensional figure 502 arranged in step S92 (step S93).

[0206] Next, the posture estimation unit 223 determines whether the distribution (cluster distribution) of the cluster group (CL, CL1, CL2) acquired in step S91 and the dimensional ratio acquired in step S93 satisfy the supine position condition (step S94).

[0207] If it is determined that the cluster distribution and the size ratio satisfy the lying-down condition, the posture estimation unit 223 sets the value "lying-down" to the variable "posture" (step S95). After that, the process returns to the upper-level flowchart (see FIG. 4).

[0208] If it is determined that the cluster distribution and the size ratio do not satisfy the lying position condition, the posture estimation unit 223 further determines whether or not the cluster distribution and the size ratio satisfy the sitting position condition (step S96).

[0209] If it is determined that the cluster distribution and the size ratio satisfy the sitting position condition, the posture estimation unit 223 sets the variable "posture" to the value "sitting position" (step S97). After that, the process returns to the upper flowchart (see FIG. 4).

[0210] If it is determined that the cluster distribution and the size ratio do not satisfy the sitting condition either, the posture estimation unit 223 sets the variable "posture" to the value "standing" (step S98). After that, the process returns to the upper flowchart (see FIG. 4).

[0211] (5) Human Body Pose Estimation Method, Device Control Method, and Program The human body posture estimation function of the present disclosure may be realized by a human body posture estimation method or a program. The human body posture estimation method includes, among the various steps described above, at least steps S2 to S5 (ranging steps), steps S6 and S7 (placement steps), and step S9 (posture estimation step). The program is a program for causing one or more processors to execute the human body posture estimation method.

[0212] The device control function of the present disclosure may be realized by a device control method or a program. The device control method further includes step S10 (behavior detection step) and step S11 (device control step) in addition to the ranging step, the positioning step, and the attitude estimation step. The program is a program for causing one or more processors to execute the device control method.

[0213] (6) Modified Examples of Non-Operational Behavior In addition to going to bed and waking up, non-operational behavior may further include, for example, falling asleep after going to bed and waking up before waking up. Falling asleep and waking up can be detected, for example, based on changes (stopping and starting) in body movement. Stopping body movement may be, for example, a state in which no body movement exceeding a threshold is detected continuing for a predetermined period of time or more. Starting body movement may be, for example, a state in which body movement exceeding a threshold is detected continuing for a predetermined period of time or more.

[0214] (7) Modified Examples of Arrangement of Each Unit In this embodiment, the control device 2 includes the distance measurement unit 221, point placement unit 222, posture estimation unit 223, behavior detection unit 224, device control unit 225, information acquisition unit 226, history storage unit 227, and learning unit 228, but all of these elements may be included in the radio wave sensor 1. Alternatively, the radio wave sensor 1 may include only the distance measurement unit 221 out of these elements, and the other elements may be included in the control device 2. Alternatively, the control device 2 may be built into the radio wave sensor 1.

[0215] (8) First Modification of Device Control System: Human Body Pose Estimation System In this modification, the elements related to the device control function, i.e., the behavior detection unit 224, the device control unit 225, the information acquisition unit 226, the history accumulation unit 227, and the learning unit 228, are omitted from the device control system 100 shown in Fig. 1. This device control system 100 includes at least a distance measurement unit 221, a point placement unit 222, and a posture estimation unit 223, and may be referred to as a "human body posture estimation system 100."

[0216] In addition, the human body posture estimation system 100 may further include, in addition to the distance measurement unit 221, point placement unit 222, and posture estimation unit 223, a behavior detection unit 224, an equipment control unit 225, an information acquisition unit 226, a history storage unit 227, and a learning unit 228.

[0217] (9) Variation of Difference Acquisition: Intra-Frame Difference The radio wave sensor 1 may perform transmission and reception operations N times (where N is an integer greater than or equal to 4) per frame, with one frame being a predetermined time, and output N pieces of location identifiable information per frame. The processing unit 22 holds at least N pieces of location identifiable information corresponding to one frame. The processing unit 22 then selects three or more pieces of location identifiable information (representative location identifiable information) representing one frame from the N or more pieces of location identifiable information held corresponding to one or more frames, and acquires two or more differences for the selected three or more representative location identifiable information. The processing unit 22 may then acquire two or more pieces of location information based on two or more differences acquired for the three or more representative location identifiable information representing one frame.

[0218] In this example, the processing unit 22 further stores N pieces of location information corresponding to the frame following the first frame. When two or more pieces of location information acquired for three or more pieces of representative location information all correspond to a moving object (200c) other than the human body 301, the processing unit 22 designates the first frame and the next frame as a new frame. Then, the processing unit 22 selects three or more pieces of representative location information representing the new frame from the 2×N pieces of location information corresponding to the new frame, and obtains two or more differences between the selected three or more pieces of representative location information. The processing unit 22 may obtain two or more pieces of location information based on two or more differences between the three or more pieces of representative location information representing the new frame.

[0219] (10) Second Modification of Device Control System: Device Control System Including Moving Object Detection System In this modification, the explanation of the matters already mentioned in the embodiment may be omitted or simplified.

[0220] 17 , the device control system 100 in this modification includes a moving object detection system 100A and a device control device 2B. The moving object detection system 100A includes a radio wave sensor 1 and a moving object detection device 2A. The moving object detection device 2A includes a conversion unit 11, a storage unit 12, a difference acquisition unit 13, a moving object detection unit 14, and a point placement unit 222. The moving object detection unit 14 includes a distance measurement unit 221, a posture estimation unit 223, and a behavior detection unit 224. The device control device 2B includes a device control unit 225, an information acquisition unit 226, a history accumulation unit 227, and a learning unit 228.

[0221] 17, the moving object detection device 2A includes the conversion unit 11, but the conversion unit 11 may be built into the radio wave sensor 1 or may be interposed between the radio wave sensor 1 and the moving object detection device 2A. In the above-described embodiment, the conversion unit 11 is built into the radio wave sensor 1, although not shown.

[0222] 17, the moving object detection device 2A and the equipment control device 2B are separate entities, but they may be configured as an integrated unit. Furthermore, although not shown in FIG. 17, the moving object detection system 100A typically further includes a reception unit 21, a processing unit 22 (for example, a first processing unit on the moving object detection device 2A side and a second processing unit on the equipment control device 2B side), and an output unit 23 (see FIG. 1).

[0223] (10-1) Radio wave sensor The radio wave sensor 1 that constitutes the moving object detection system 100A transmits a transmission wave Tr, which is a radio wave modulated in a predetermined manner, toward a real space (for example, the inside of the room 400 shown in FIG. 2 ; hereinafter, referred to as “real space (400)”), receives a reflected wave Re from the real space (400), and performs a transmission / reception operation to output a reception signal based on the transmission wave Tr and the reflected wave Re.

[0224] The transmission and reception operations are repeated at predetermined intervals (i.e., periodically) as in the embodiment, but may also be performed irregularly.

[0225] While the radio wave sensor 1 in the embodiment outputs position-identifiable information such as an FFT result group, the radio wave sensor 1 in this modified example outputs a received signal such as an IF signal. The received signal is converted into position-identifiable information outside the radio wave sensor 1 (for example, by a conversion unit 11 constituting the moving object detection device 2A).

[0226] (10-1-1) Received Signal In the case of radio waves modulated by the FMCW method, for example, the received signal is a signal (IF signal: see Figure 3) indicating the frequency difference Δf between the transmitted wave Tr and the reflected wave Re at the same time. However, the received signal may also be a signal indicating the time difference Δt between the transmitted wave Tr and the reflected wave Re, which is calculated from the frequency difference Δf. Note that in the case of radio waves modulated by the pulse modulation method, the received signal may also be a signal indicating the time difference Δt between the transmitted wave Tr and the reflected wave Re.

[0227] (10-1-2) Modulation Method The predetermined method, i.e., the radio wave modulation method, is the FMCW method in this example, but it may also be, for example, a pulse modulation method, or any method that results in the acquisition of location-identifiable information.

[0228] (10-2) Moving Object Detection Device The conversion unit 11 constituting the moving object detection device 2A performs conversion processing. The conversion processing is processing for converting the received signal output by the radio wave sensor 1 into location-identifying information. The conversion processing is, for example, processing for performing a Fourier transform on an IF signal, which is a type of received signal. The Fourier transform is, for example, an FFT, but may also be an STFT (short-time Fourier transform) or the like.

[0229] The position-identifiable information is information that can identify the position of each of one or more objects (human body 301, electric blinds 200c, stationary object 302) that exist in the real space (400), and is, for example, the FFT result group described in the embodiment, but is not limited to this.

[0230] The storage unit 12 performs a storage process. In this example, the storage process is a process of storing the position identifiable information converted by the conversion process. The position identifiable information is written, for example, to a memory included in the moving object detection device 2A and stored in the memory for a predetermined period (for example, one frame period, three frame periods, etc.).

[0231] The difference acquisition unit 13 performs a difference acquisition process, which is a process for acquiring differences between a plurality of pieces of location identifiable information held by the holding process.

[0232] The moving object detection unit 14 performs a moving object detection process. The moving object detection process is a process for detecting a moving object using a received signal received by the radio wave sensor 1. The moving object detection process is a process for making an estimation regarding a moving object based on, for example, a difference acquired using the received signal. Specifically, the moving object detection process is a process for estimating whether each of one or more objects is a moving object or a non-moving object (a stationary object 302) based on the difference acquired by the difference acquisition process, for example, to identify the position of the object estimated to be a moving object (a human body 301, an electric blind 200c), and to acquire moving object position information indicating the identified position.

[0233] The point placement unit 222 performs point placement processing. The point placement processing is processing for virtually placing points corresponding to the moving object position information acquired by the moving object detection processing in a virtual space (e.g., three-dimensional space 500 in the embodiment) corresponding to the real space (400). The point placement processing may be, for example, processing such as multi-point placement described as the operation of the point placement unit 222 in the embodiment.

[0234] The difference acquisition process in this example acquires two or more differences with different time differences for the three or more pieces of location identifiable information stored in the storage process. The moving object detection process acquires two or more pieces of moving object position information corresponding to the two or more differences acquired by the difference acquisition process. The point placement process virtually places two or more points corresponding to the two or more pieces of moving object position information acquired by the moving object detection process in a virtual space (500).

[0235] Here, "virtually placing two or more points in a virtual space (500)" includes the case where "all points are placed" and the case where "only some points are placed" (in other words, at least one point is placed) of the two or more points corresponding to two or more pieces of moving object position information acquired by the moving object detection process, and may also include the case where "not a single point is placed."

[0236] However, cases in which "not even one point is placed" may be excluded, and such point placement processing virtually places "all or part of" two or more points corresponding to two or more pieces of moving object position information acquired by the moving object detection processing in the virtual space (500). In other words, even if all of the two or more points are determined to be moving objects other than the human body 301, for example, one representative point of the two or more points (any one point, or one point corresponding to the average value of the two or more pieces of moving object position information corresponding to the two or more points) may be placed.

[0237] In addition, if all of the two or more points corresponding to the two or more pieces of moving body position information acquired by the moving body detection process are always positioned, there is no need to determine whether the moving body is a human body 301 or a moving object other than a human body 301 (whether the moving body position information is information corresponding to a human body 301 or information corresponding to a moving object other than a human body 301), as is done in this example.

[0238] In this example, the above determination is made for each of the two or more pieces of acquired moving object position information, and if it is determined that the information corresponds to the human body 301, a point corresponding to that moving object position information is placed. Therefore, depending on the individual determination results, there may be cases where all of the two or more points corresponding to the two or more pieces of moving object position information are placed, where only some of the points are placed (at least one point is placed), or where no points are placed.

[0239] In addition, for example, if the moving object position information to be judged is information from which an effective judgment result is not expected, it may be excluded from the judgment target, and as a result, only some points or no points may be placed. In this example, for example, based on information such as the reception strength of the reflected wave Re corresponding to the moving object position information, it may be determined whether or not each of two or more moving object position information acquired by the moving object detection process should be subject to the above judgment, and the above judgment may be made only for the moving object position information that has been judged to be subject.

[0240] (10-2-1) Details of Point Placement In the point placement process, for example, only the point corresponding to the human body 301 among the moving objects (i.e., some of the two or more points) may be placed. In other words, points corresponding to moving objects other than the human body 301 may not be placed. However, points corresponding to moving objects other than the human body 301 may also be placed (i.e., all of the two or more points).

[0241] Note that the point placement process may place points corresponding to, for example, the human body 301 and living bodies (such as pets: not shown) other than the human body 301 among the moving bodies. In other words, points corresponding to moving bodies other than living bodies may be excluded from placement targets.

[0242] In this way, by obtaining two or more differences with different time differences for three or more pieces of location identifiable information, it is expected that the number of points to be placed in the virtual space (500) will increase (the number of points in the point cloud 501 will increase). As a result, it will be possible to more accurately determine whether the point cloud 501 corresponds to a moving object such as a human body 301 or a stationary object 302, thereby improving the accuracy of moving object detection using reflected waves of transmitted radio waves, for example, improving the accuracy of estimation regarding a moving object using the radio wave sensor 1.

[0243] Furthermore, as a result of improving the accuracy of estimation regarding moving objects in this way, the feasibility of controlling the device 200 using the radio wave sensor 1 (for example, the control accuracy of the device control device 2B described below) is improved.

[0244] (10-2-2) Estimation of Moving Object Position Information: Is the Moving Object a Human Body or a Moving Object Other Than a Human Body? The moving object is either the human body 301 or a moving object other than the human body 301. The moving object other than the human body 301 is, for example, the electric blinds 200c that open and close electrically, but it may be any moving object, such as a cleaning robot that cleans while moving. The moving object other than the human body 301 may also include the body of a living organism other than a human (for example, a pet kept by a person).

[0245] The moving object detection process further estimates whether each of the two or more pieces of moving object position information acquired corresponds to the human body 301 or a moving object (200c) other than the human body 301, based on at least a change in the difference. The point placement process virtually places, in the virtual space (500), points corresponding to the moving object position information that the moving object detection process estimates to correspond to the human body 301, out of the two or more pieces of moving object position information acquired by the moving object detection process. In other words, points corresponding to the moving object position information that the human body detection process estimates to correspond to a moving object other than the human body 301 are excluded from the placement targets.

[0246] In this way, in the moving object detection system 100A, when detecting a human body 301, for example, when making an estimation regarding the human body 301, by obtaining two or more differences with different time differences for three or more pieces of position-identifiable information, it becomes possible to detect various movements of the human body 301 (for example, body movement, slight breathing movement, movement of the hands and feet, etc.: see Figure 12), thereby improving the detection accuracy of the human body 301, for example, improving the accuracy of estimation regarding the human body 301.

[0247] (10-2-3) Variant of estimation regarding moving object position information: whether the moving object other than a human body is a living organism or a moving object other than a living organism. The moving object other than the human body 301 may be, for example, either a living organism other than the human body 301 (for example, the body of a living creature such as a pet kept by a person: not shown) or a moving object other than a living organism (such as electric blinds 200c).

[0248] In this case, the point placement process places points corresponding to the living body in the virtual space 500. In other words, points corresponding to moving objects other than the living body are excluded from the placement targets.

[0249] In this example, the device control process (described later) may perform control such as turning on lighting fixtures or turning on an air conditioner when a point cloud 501 corresponding to a living organism is detected (for example, when a person or pet enters a room).

[0250] Furthermore, the moving object detection process may distinguish multiple points in the virtual space (500) into a point group 501 corresponding to the human body 301 and a point group 501 corresponding to a non-human living body based on the distribution of multiple points in the virtual space (500) (such as the shape of the cluster CL).

[0251] The device control process may perform different device control depending on whether the point cloud 501 corresponds to the human body 301 or a living organism other than the human body 301 (for example, a pet). In detail, the device control process may perform control such that, for example, the television is turned on when the point cloud 501 corresponding to the human body 301 is detected, but the television is not turned on even when the point cloud 501 corresponding to a living organism other than the human body 301 (a pet) is detected. In this way, by performing different device control depending on whether the target object is the human body 301 or a living organism other than the human body 301, it is possible to make the coexistence environment between people and pets more comfortable, and further to achieve both comfort and energy conservation.

[0252] (10-2-4) Estimation taking into account the reception strength of reflected waves The moving object detection process estimates whether each of the two or more acquired moving object position information corresponds to a human body 301 or a moving object (200c) other than a human body 301 based on at least one of the reception strength of the reflected wave Re corresponding to the moving object position information and the change in the reception strength, in addition to the change in the difference.

[0253] In this way, by utilizing at least one of the received strength and the change in received strength of the reflected wave Re in addition to the change in the difference, the accuracy of estimation as to whether the moving body (301, 200c) is a human body 301 or a moving object (200c) other than a human body 301 is improved.

[0254] (10-2-5) Estimation of Point Cloud The moving object detection process estimates whether the point cloud 501 as a whole corresponds to the human body 301 or a moving object (200c) other than the human body 301, further based on at least one of the distribution and a change in the distribution of the point cloud 501. In this way, by also using at least one of the distribution and a change in the distribution of the point cloud 501 arranged in the virtual space (500), it is possible to improve the accuracy of estimation as to whether the moving object (301, 200c) is the human body 301 or a moving object (200c) other than the human body 301.

[0255] (10-2-6) Inter-Frame Difference The radio wave sensor 1 performs transmission and reception operations N times (N is an integer equal to or greater than 1) per frame, with a predetermined time being one frame. As a result, N pieces of location identifiable information are output from the radio wave sensor 1 per frame. The storage process stores 3×N or more pieces of location identifiable information corresponding to three or more frames. The difference acquisition process selects one piece of representative location identifiable information representing each of the three or more frames from the 3×N or more pieces of location identifiable information corresponding to three or more frames stored in the storage process. The difference acquisition process then acquires two or more differences for the selected three or more pieces of representative location identifiable information. The moving object detection process acquires two or more pieces of moving object location information based on two or more differences acquired for three or more pieces of representative location identifiable information representing each of the three or more frames.

[0256] This makes it possible to select three or more pieces of representative location identifiable information among three or more frames, and to obtain two or more differences for the selected three or more pieces of representative location identifiable information.

[0257] (10-2-7) Intra-Frame Difference The radio wave sensor 1 performs transmission and reception operations N times (N is an integer equal to or greater than 3) per frame, with a predetermined time being one frame, and outputs N or more received signals per frame. The conversion process converts each of the N or more received signals output in one frame into location-identifiable information. The storage process stores at least the N pieces of location-identifiable information corresponding to one frame.

[0258] The difference acquisition process selects three or more representative pieces of location identifiable information representing one frame from N or more pieces of location identifiable information corresponding to one or more frames held in the holding process.The difference acquisition process then acquires two or more differences for the selected three or more pieces of representative location identifiable information.The moving object detection process acquires two or more pieces of moving object location information based on the two or more differences acquired for the three or more pieces of representative location identifiable information representing one frame.

[0259] This makes it possible to select three or more pieces of representative location identifiable information within one frame, and to obtain two or more differences for the selected three or more pieces of representative location identifiable information.

[0260] (10-2-7a) Frame Combination The holding process further holds N pieces of location identifiable information corresponding to the frame following the first frame. When two or more pieces of moving object location information acquired by the moving object detection process for three or more pieces of representative location identifiable information all correspond to moving objects (200c) other than a human body 301, the difference acquisition process treats the first frame and the next frame as a new frame. Then, the difference acquisition process selects three or more pieces of representative location identifiable information representing the new frame from the 2×N pieces of location identifiable information corresponding to the new frame, and acquires two or more differences for the selected three or more pieces of representative location identifiable information. The moving object detection process acquires two or more pieces of moving object location information based on the two or more differences acquired by the difference acquisition process for the three or more pieces of representative location identifiable information representing the new frame.

[0261] As a result, if two or more differences cannot be obtained within one frame, a new frame is formed with the next frame, and two or more differences can be obtained within the new frame.

[0262] (10-2-8) Specific Examples of Moving Object Detection (10-2-8a) Identifying One-Dimensional Position Using One Antenna The radio wave sensor 1 has at least one antenna that receives reflected waves Re. The received signal is an IF signal generated based on the transmitted waves Tr and the reflected waves Re received by one antenna for one transmission / reception operation. The IF signal is a signal that indicates the frequency difference between the transmitted waves Tr and the reflected waves Re, and is generated by mixing the transmitted waves Tr and the reflected waves Re.

[0263] The transform process is a Fourier transform process. The Fourier transform process is a process of performing a Fourier transform on an IF signal. The location identifiable information is a Fourier transform result of the Fourier transform process. The storage process stores multiple Fourier transform results corresponding to one antenna.

[0264] The moving object detection process includes a ranging process. The ranging process in this example is a process of measuring the distance from one antenna to an object estimated to be a moving object (301, 200c) based on the difference between multiple Fourier transform results corresponding to one antenna. The moving object position information is one-dimensional position information based on the ranging result of the ranging process.

[0265] This improves the accuracy of detecting a moving object, for example, when processing (moving object detection processing) for detecting a moving object (301, 200c) is performed by the radio wave sensor 1 that uses radio waves modulated by the FMCW method. Furthermore, even if there is only one antenna that receives the reflected wave Re, it is possible to at least improve the accuracy of measuring the distance to the moving object (301, 200c) (identifying the one-dimensional position).

[0266] (10-2-8b) Identifying Two-Dimensional or Three-Dimensional Position Using Multiple Antennas The radio wave sensor 1 has multiple antennas that receive reflected waves Re. Multiple IF signals corresponding to the multiple antennas are output from the radio wave sensor 1 for each transmission / reception operation. For each of these multiple antennas, the converter 11 performs a conversion process, the holder 12 performs a storage process, the difference acquirer 13 performs a difference acquisition process, and the moving object detector 14 performs a moving object detection process including a ranging process.

[0267] The location-identifiable information is a group of Fourier transform results consisting of multiple Fourier transform results corresponding to multiple antennas. The storage process stores the multiple Fourier transform result groups. The moving object detection process includes multiple ranging processes that measure the distance from each of the multiple antennas to an object estimated to be a moving object (301, 200c) based on the difference between the multiple Fourier transform result groups for each of the multiple antennas. The moving object position information is two-dimensional or three-dimensional position information based on multiple ranging results corresponding to the multiple ranging processes.

[0268] This improves the accuracy of the moving object detection process that identifies the two-dimensional position (e.g., two-dimensional coordinates in the two-dimensional virtual space 500, or a set of direction and distance, etc.) or three-dimensional position (e.g., three-dimensional coordinates in the three-dimensional virtual space 500, or a set of direction and distance, etc.) of the moving object (301, 200c).

[0269] (10-2-9) Moving object detection method and program The moving object detection method in this example includes at least step S1 (conversion step), step S2 (storage step), step S4 (difference acquisition step), steps S5 and S6 (moving object detection step), and step S7 (point placement step) among the various steps described in the embodiment. The program causes one or more processors to execute this moving object detection method.

[0270] (10-3) Device Control Device The device control unit 225 constituting the device control device 2B performs device control processing to control the device 200 based at least on a point cloud 501, which is a collection of multiple points arranged in a virtual space (500). In this way, the radio wave sensor 1 acquires the point cloud 501 corresponding to the moving object (301, 200c), and the device 200 is controlled based on the acquired point cloud 501, thereby realizing control of the device 200 using the radio wave sensor 1.

[0271] In particular, by acquiring a point cloud 501 corresponding to the human body 301 and controlling the device 200 based on the acquired point cloud 501, it is possible to realize control of the device 200 based on the position and movement of the human body 301.

[0272] (10-3-1) Posture Estimation and Behavior Detection The moving object detection process by the moving object detection device 2A includes a posture estimation process. The posture estimation process is a process for estimating the posture of the human body 301. The posture estimation process estimates the posture of the human body 301 based on, for example, the distribution of the point cloud 501 (FIGS. 13A to 14C). Specifically, the posture estimation process may be, for example, the process described in the embodiment using FIG. 7.

[0273] The behavior detection unit 224 executes behavior detection processing. The behavior detection processing in this example is processing for detecting the behavior of a person corresponding to the human body 301 based on the posture or posture change estimated by the posture estimation processing. The device control processing in this example controls the device 200 based on the behavior detected by the behavior detection processing.

[0274] In this way, the posture or posture change of the human body 301 is estimated based on the distribution of the point cloud 501, the human behavior corresponding to the human body 301 is detected based on the estimation result, and the device 200 is controlled based on the detection result, thereby realizing control of the device 200 based on the posture of the human body 301 and ultimately the human behavior.

[0275] (10-3-2) Acquisition of Environmental Information The information acquisition unit 226 executes information acquisition processing. As described above, the information acquisition processing is processing for acquiring environmental information related to the environment in which the human body 301 exists. The moving object detection processing detects behavior based on the environmental information acquired by the information acquisition processing.

[0276] In this way, by using environmental information in addition to posture or posture changes, the accuracy of detecting behavior and therefore the accuracy of controlling the device 200 can be improved.

[0277] (10-3-3) Device Control Using Automatic Control Information The device control process controls the device 200 based on behavior using pre-stored automatic control information. This allows the device 200 to be automatically controlled based on behavior using the automatic control information.

[0278] (10-3-4) Accumulation of Automatic Control History Information The history accumulation unit 227 in this example executes a history accumulation process. The history accumulation process in this example accumulates automatic control history information. Accumulating automatic control history information in this manner enables learning processing based on the automatic control history information, and ultimately makes it possible to improve control accuracy using the automatic control history information. Furthermore, as described above, the automatic control history information associates the behavior-based control content of the device 200 with one or more pieces of information from the time, environmental information, and person identifier. This allows control suited to each individual person.

[0279] (10-3-5) Further Accumulation of Manual Control History Information The history accumulation process further accumulates manual control history information. The manual control history information is information about the control history of the device 200 based on operations on the device 200. The operations are, for example, operations to adjust the brightness of lighting, as described above, and are accepted via an operating device such as a remote control. In this way, by further accumulating manual control history information, it becomes possible to improve control accuracy by utilizing two types of control history information, automatic and manual.

[0280] For example, based on the accumulated automatic control history information and manual control history information, if it is detected that an operation to cancel the control (e.g., an operation to dim the lights) is not performed within a certain time (e.g., within one minute) after automatic control to brighten the lights in response to the detection of a person sitting down (hereinafter referred to as ``tacit inaction''), the system will learn that the control is appropriate a certain number of times in a certain period of time (e.g., three times or more in one week), thereby improving control accuracy.

[0281] However, the above-described learning process can be realized by accumulating only automatic control history information, without accumulating manual control history information. For example, while accumulating automatic control history information, a learning process may be repeated in which a determination is made based on real-time operation information as to whether tacit inaction has been detected a predetermined number of times or more in a predetermined period (e.g., three or more times in a week), and if tacit inaction has been detected a predetermined number of times or more in a predetermined period, the control content is learned to be appropriate.

[0282] Alternatively, only manual control history information may be stored without storing automatic control history information. For example, if, based on the stored manual control history information, it is detected that an operation to brighten the lights has been received within a certain time period (e.g., within one minute) after the detection of seating, and this is detected a predetermined number of times in a predetermined period (e.g., three or more times in one week), the control content "brighten the lights in response to the detection of seating" is registered in the automatic control information. By repeating this process, it is possible to improve the control accuracy of automatic control of lighting brightness and diversify the content of automatic control.

[0283] As described above, the manual control history information associates the control content of the device 200 based on the operation with one or more pieces of information from the time, environmental information, and person identifier. The learning unit 228 executes a learning process to update the automatic control information when, for example, the time and operation indicated by the control history information satisfy predetermined update conditions. This learning process using the automatic control history information makes it possible to achieve control suited to the behavior of each individual person.

[0284] (10-3-6) Updating of Automatic Control Information Alternatively, the learning unit 228 may execute a learning process to update the automatic control information when a predetermined update condition is satisfied between the time and control content indicated by the information and the time and operation indicated by manual control history information regarding the control history of the device 200 based on operations on the device 200. Such a learning process using two types of control history information, automatic and manual, makes it possible to achieve control suited to the behavior and operation of each individual person.

[0285] (10-3-7) Clustering The virtual space (500) in this example is a three-dimensional space 500, similar to that in the embodiment. The moving object detection process performs clustering on the point cloud 501 to obtain a cluster group (CL, CL1, CL2), which is a set of one or more clusters (CL, CL1, CL2), and estimates the posture of the human body 301 based on the distribution of the cluster group (CL, CL1, CL2) in the three-dimensional space 500. This makes it possible to accurately estimate the posture of the human body 301 based on the distribution of the cluster group (CL, CL1, CL2).

[0286] (10-3-8) Posture Estimation The moving object detection process estimates the posture of the human body 301 based on the ratios of the length, width, and height of the three-dimensional figure 502 that surrounds the cluster group (CL, CL1, CL2). This makes it possible to accurately and easily estimate the posture of the human body 301 based on the ratios of the length D, width W, and height H of the three-dimensional figure 502 that surrounds the cluster group (CL, CL1, CL2).

[0287] (10-3-9) Detection of Behavior The behavior detection process detects a non-operation behavior, which is a behavior different from an operation of the human body 301 on the device 200, based at least on the posture or a change in posture estimated by the posture estimation process. The device control process controls the device 200 based at least on the detected non-operation behavior. In this way, by controlling the device 200 based on the non-operation behavior (for example, a behavior that does not intend to operate the device 200, such as a daily activity), it is possible to control the device 200 without intentionally operating the device 200 (operation-less device control).

[0288] (10-3-10) Device Control Method and Program The device control method is a device control method that detects a moving object and controls the device 200 based on the detection results. For example, the device control method uses a radio wave sensor 1 to make an estimation regarding the moving object (a human body 301, an electric blind 200c), which is a moving object, and controls the device 200 based on the estimation results. The radio wave sensor 1 performs a transmission / reception operation of transmitting a transmission wave Tr, which is a radio wave modulated using a predetermined method, toward a real space (the interior of a room 400), receiving a reflected wave Re from the real space (400), and outputting a reception signal based on the transmission wave Tr and the reflected wave Re. The device control method includes a conversion step (S1), a holding step (S2), a difference acquisition step (S4), a moving object detection step (S5, S6), a point arrangement step (S7), and a device control step (S11). The conversion step (S1) performs a conversion process to convert the received signal output by the radio wave sensor 1 into location-identifiable information capable of identifying the respective positions of one or more objects (301, 200c, 302) present in the real space (400). The storage step (S2) performs a storage process to store the location-identifiable information converted by the conversion process. The difference acquisition step (S4) performs a difference acquisition process to acquire differences between the multiple pieces of location-identifiable information stored in the storage process. The moving object detection steps (S5, S6) detect a moving object (301, 200c) based on the differences acquired by the difference acquisition process. The moving object detection steps (S5, S6) perform, for example, a moving object detection process to estimate whether each of the one or more objects is a moving object or a stationary object (302) based on the acquired differences, identify the positions of the objects estimated to be moving objects (301, 200c), and acquire moving object position information indicating the identified positions. The point arrangement step (S7) performs a point arrangement process in which points corresponding to the moving object position information acquired by the moving object detection process are virtually arranged in a virtual space (500) corresponding to the real space (400). The device control step (S11) controls the device 200 based at least on a point cloud 501, which is a collection of multiple points arranged in the virtual space (500). The program causes one or more processors to execute this device control method.

[0289] Second Embodiment Next, a second embodiment and a modified example of the present disclosure will be described with reference to FIGS. 1 to 22 (particularly FIGS. 4, 5, and 18 to 22).

[0290] The second embodiment is similar to the first embodiment except for the process of identifying an object using a distribution of multiple points (CL, CL1, CL2). In the following, the explanation of the already mentioned matters will be omitted or simplified, and commonalities with the first embodiment and differences from the first embodiment will be explained in detail.

[0291] (1) Commonalities with the First Embodiment The radio wave sensor 1 transmits radio waves (transmission waves Tr: hereinafter, sometimes referred to as "radio waves Tr") in a predetermined direction, receives reflected waves Re, and outputs a received signal.

[0292] Note that transmitting in a predetermined direction means radiating the radio waves Tr within a certain range centered on the predetermined direction. In other words, pointing the radio wave sensor 1 in a predetermined direction results in the radio waves Tr propagating within a certain range centered on the predetermined direction.

[0293] The radio wave sensor 1 in the first or second embodiment is located above the region R1 and transmits radio waves Tr in the height direction of the region R1 (specifically, vertically downward, i.e., in the vertical linear downward direction). In other words, the predetermined direction is the height direction of the region R1 (see FIG. 2), which is the Z direction in the three-dimensional space 500 shown in FIG. 13A etc. Therefore, the radio wave sensor 1 transmits radio waves Tr in the height direction, receives reflected waves Re of the transmitted radio waves Tr, and outputs a received signal.

[0294] The processing unit 22 detects objects (301, 200c) using the received signal output by the radio wave sensor 1. The objects are, for example, a human body 301, an electric blind 200c which is a moving object other than the human body 301, and a stationary object 302 such as a desk 302 (see FIG. 2). The human body 301 is a moving object that performs an action such as walking, in other words, a non-periodic (or irregular) movement. The electric blind 200c is a moving object that performs a periodic (or regular) movement such as opening and closing.

[0295] (2) Differences from the First Embodiment (2-1) Object Identification in the First Embodiment: Identification Based on the Dimensional Ratio of Length, Width, and Height The processing unit 22 in the first embodiment identified objects such as a human body 301 and a stationary object 302 in the three-dimensional space 500 (see Figures 13A to 14C) corresponding to the region R1 based on the dimensional ratio of length D, width W, and height H of a rectangular parallelepiped 502 that surrounds the distribution of one or more points (i.e., a point cloud 501 that is a collection of one or more points, or one or more clusters CL, CL1, CL2 formed by the point cloud 501).

[0296] (2-2) Object identification in the second embodiment 1: Identification based on the positional relationship between the ends in the height direction In contrast, the processing unit 22 in the second embodiment identifies the object based on the positional relationship between the two ends in a distribution of one or more points (one or more clusters CL, CL1, CL2 or point cloud 501).

[0297] In this embodiment, the objects include not only the moving objects (301, 200c) such as the human body 301 and the electric blinds 200c shown in FIG. 2, but also, for example, a cleaning robot 303 shown in FIG. 20A, an electric fan 304 shown in FIG. 20B, and a cat 305 shown in FIGS. 20A and 20B (hereinafter, these may be referred to as "objects 301 to 305"). The cleaning robot 303 is an example of a moving object that moves automatically (e.g., independently). The electric fan 304 is an example of a moving object that moves partially (e.g., the fins rotate, or the upper part including the fins oscillates). The cat 305 is an example of an animal (e.g., a pet) other than the human body 301.

[0298] (2-2-1) Both ends of the distribution in the height direction The both ends of the distribution (CL, CL1, CL2) in the height direction are, for example, the upper cluster CL1 and the lower cluster CL2 in the distribution shown in Figure 13A (point cloud 501 including two clusters CL1 and CL2).

[0299] The positional relationship between both ends in the height direction is, for example, the distance between the upper cluster CL1 and the lower cluster CL2. The distance between the upper cluster CL1 and the lower cluster CL2 is, for example, the difference between the height of the upper cluster CL1 based on the XY plane (corresponding to the floor surface of the region R1 shown in FIG. 2) shown in FIG. 13A and the height of the lower cluster CL2 based on the XY plane (i.e., the height difference DH between the upper cluster CL1 and the lower cluster CL2: see FIGS. 20A and 20B ).

[0300] The height of cluster CL1 is the Z component of the position information corresponding to cluster CL1, and the height of cluster CL2 is the Z component of the position information corresponding to cluster CL2.

[0301] Alternatively, the ends of the distribution (CL, CL1, CL2) in the height direction may be, for example, the distance between two horizontal planes passing through the highest and lowest points of a cluster CL, in other words, the difference between the Z component of the coordinate of the highest point and the Z component of the coordinate of the lowest point.

[0302] Furthermore, the positional relationship between both ends may further include a change in the distance over time, i.e., a movement, in addition to the distance such as the height difference DH. The movement includes various types of movement, such as a periodic or regular movement of the object (such as the movement of the cleaning robot 303 or the swinging of the electric fan 304), a non-periodic or irregular movement of the object (such as the walking of the human body 301 or the cat 305), a general movement of the object (such as the movement of the cleaning robot 303 or the walking of the human body 301), and a partial movement of the object (such as the swinging of the electric fan 304 or the movement of the hand of the human body 301).

[0303] Specifically, the processing unit 22 acquires a plurality of pieces of position information corresponding to a plurality of parts constituting the object (301 to 305), and acquires a distribution (CL, CL1, CL2) of a plurality of points (point cloud 501) corresponding to the plurality of pieces of position information, and then identifies the object (301 to 305) based on the positional relationship between both ends in a predetermined direction of the acquired distribution of point cloud 501.

[0304] In this embodiment, as described above, the predetermined direction is the height direction of region R1, i.e., the Z direction of the three-dimensional space 500 corresponding to region R1 (see FIGS. 20A to 21B). In this embodiment, the positional relationship is the distance between both ends in the height direction (Z direction), and more specifically, the height difference DH between the upper cluster CL1 and the lower cluster CL2, or the height H of the rectangular parallelepiped 502.

[0305] The distance between the two ends in the height direction is the difference in the Z component between the two pieces of position information corresponding to the two ends. For example, the processing unit 22 may calculate the difference in the Z component between the position information corresponding to the upper cluster CL1 and the position information corresponding to the lower cluster CL2 as the height difference DH.

[0306] The position information corresponding to the upper cluster CL1 may be, for example, the coordinates of a representative point of cluster CL1. The representative point of cluster CL1 is, for example, the highest point of cluster CL1, but it may also be the center of gravity or the lowest point. The position information corresponding to the lower cluster CL2 may be, for example, the coordinates of a representative point of cluster CL2. The representative point of cluster CL2 is, for example, the lowest point of cluster CL2, but it may also be the center of gravity or the highest point.

[0307] In this way, the processing unit 22 acquires a plurality of pieces of position information corresponding to a plurality of parts constituting the object (301-305) using a received signal based on the reflected wave Re of the radio wave Tr transmitted in the height direction (Z direction) from the radio wave sensor 1 located above the region R1, and acquires the distribution (CL, CL1, CL2) of the plurality of points (501).Then, the object (301-305) is identified based on the positional relationship (DH, H) between both ends in the height direction of the distribution (CL, CL1, CL2) of the plurality of points (501), thereby facilitating the identification of the object (301-305).

[0308] Furthermore, by including the positional relationship in the distance between both ends of the distributions (CL, CL1, CL2) in the height direction (height difference DH), it is possible to facilitate and improve the accuracy of identifying the target objects (301 to 305).

[0309] (2-3) Target Identification 2 in the Second Embodiment: Identification Based on the Spread of Distribution in a Plane Perpendicular to the Height Direction The processing unit 22 identifies the target objects (301 to 305) based further on the spread of the distribution (CL, CL1, CL2) of the point cloud 501 in a plane whose normal direction is the height direction (Z direction).

[0310] The plane having the height direction (Z direction) as its normal direction is a horizontal plane, for example, the XY plane along the X and Y directions shown in FIG. 20A, or a plane parallel to the XY plane.

[0311] The extent in the horizontal plane is, for example, the distance DW between both ends in the horizontal direction (X direction) as shown in Figures 20A to 21B. The distance DW corresponds to the width W of the rectangular parallelepiped 502 in the first embodiment.

[0312] The extent in the horizontal plane (XY plane) may include, for example, at least one of the extent in the horizontal direction (X direction), i.e., the positional relationship between both ends in the horizontal direction, and the extent in the vertical direction (Y direction), i.e., the positional relationship between both ends in the vertical direction.

[0313] In this way, the processing unit 22 identifies the object based on not only the positional relationship between the ends of the distribution (CL, CL1, CL2) of the point cloud 501 in a predetermined direction, but also the spread (DW, W) of the distribution (CL, CL1, CL2) of the point cloud 501 in a plane (horizontal plane) perpendicular to the height direction, thereby making it even easier to identify the object (301 to 305).

[0314] However, it is possible to facilitate identification of the target objects (301 to 305) by not considering the positional relationship between the ends of the distribution of the point cloud 501 (CL, CL1, CL2) in the height direction, but by considering the spread (DW, W) of the distribution of the point cloud 501 (CL, CL1, CL2) in a plane (horizontal plane) perpendicular to the height direction.

[0315] (2-4) Identifying an Object Further Based on Positional Relationships, Changes in Positional Relationships over Time, and Extent and Changes in Extent over Time Each of the one or more points described above is, in detail, a moving point, i.e., a moving point. That is, in the first embodiment or the second embodiment, a "point" is a moving point, and may be referred to as a "moving point" hereinafter as appropriate.

[0316] The positional relationship further includes a change in distance (height difference DH) over time, for example, the movement of the object in the vertical direction. The aforementioned spread further includes a change in spread over time, for example, the expansion and contraction of the object in the horizontal plane. By including a change in distance and spread over time in the positional relationship, it is possible to further facilitate the identification of the objects (301-305).

[0317] (2-4-1) Obtaining temporal changes in positional relationships, etc. based on multiple time differences Radio waves Tr are transmitted in each of multiple frames, and the processing unit 22 calculates multiple time differences by repeatedly performing a process of calculating the time difference, which is the difference between two received signals received in two different frames among the multiple frames.

[0318] The processing unit 22 acquires, from the plurality of time differences, a distribution (CL, CL1, CL2) of one or more moving points whose time differences are equal to or greater than a threshold value. Then, the processing unit 22 identifies the target objects (301 to 305) based on at least one of the positional relationship (DH, H) and the change in the positional relationship over time in a predetermined direction (height direction) of the distribution (CL, CL1, CL2) of the one or more moving points, and the extent (DW, W) and the change in the extent (DW, W) over time in a plane (XY plane) normal to the predetermined direction (height direction).

[0319] In this way, the processing unit 22 identifies the objects (301 to 305) based on at least one of the positional relationship (DH, H) between both ends in the height direction (Z direction) and the change in the positional relationship over time of the distribution of one or more moving points (CL, CL1, CL2) obtained from multiple time differences, and the spread (DW, W) in the horizontal plane (XY plane) and the change in the spread (DW, W) over time, thereby making it even easier to identify the objects (301 to 305).

[0320] (2-4-2) Identifying Motion Based on Spread of Distribution in the Horizontal Plane The processing unit 22 identifies the motion of the object (301 to 305) based on, for example, the change over time in the spread (DW, W) in the horizontal plane of the distribution (CL, CL1, CL2) of one or more moving points.

[0321] In this way, the processing unit 22 identifies the objects (301 to 305) based on the time change in the spread (DW, W) of the distribution of multiple moving points (CL, CL1, CL2) in the horizontal plane (XY plane), thereby making it even easier to identify the objects (301 to 305).

[0322] (2-4-3) Identifying movement based on the positional relationship between both ends of the distribution in the height direction Alternatively, the processing unit 22 may identify the movement of the object (301 to 305) based on the change over time in the positional relationship (DH, H) in the height direction (Z direction) of one or more moving point distributions (CL, CL1, CL2).

[0323] In this way, the processing unit 22 identifies the objects (301 to 305) based on the change over time in the positional relationship (DH, H) in a predetermined direction (height direction) of the distribution of multiple moving points (CL, CL1, CL2), thereby making it even easier to identify the objects (301 to 305).

[0324] (2-4-4) Identifying Motion Based on Positional Relationship Between Ends of Distribution in the Height Direction The processing unit 22 may identify the objects (301 to 305) and identify the motion of the identified objects (301 to 305) based on the spread (DW, W) and the change in spread (DW, W) over time in a plane having the height direction as the normal direction, i.e., a horizontal plane (in the three-dimensional space 500, the XY plane or a plane parallel to the XY plane), of one or more distributions of moving points (CL, CL1, CL2).

[0325] However, the processing unit 22 may identify the objects (301 to 305) and their movements based on either the horizontal spread (DW, W) of the distribution (CL, CL1, CL2) or the change in the spread (DW, W) over time.

[0326] In addition, the processing unit 22 may identify either the target object (301-305) or the movement based on the horizontal spread (DW, W) of the distribution (CL, CL1, CL2) and the change in the spread (DW, W) over time.

[0327] Furthermore, the processing unit 22 may identify either the target object (301-305) or the movement thereof based on either the horizontal spread (DW, W) of the distribution (CL, CL1, CL2) or the change in the spread (DW, W) over time.

[0328] In this way, the processing unit 22 identifies the objects (301 to 305) based on at least one of the spread (DW, W) in a plane perpendicular to the height direction of the distribution of multiple moving points (CL, CL1, CL2) and the change in the spread (DW, W) over time, thereby making it even easier to identify the objects (301 to 305).

[0329] (2-4-5) Identifying a movement based on the positional relationship between both ends of the distribution in the height direction and the change in the positional relationship over time The processing unit 22 may identify the objects (301 to 305) and identify the movement of the identified objects (301 to 305) based on the positional relationship (DH, H) in the height direction of one or more distributions of moving points (CL, CL1, CL2) and the change in the positional relationship (DH, H) over time.

[0330] However, the processing unit 22 may identify the objects (301 to 305) and their movements based on either the positional relationship (DH, H) of the distribution (CL, CL1, CL2) in a predetermined direction (height direction) or the change in the positional relationship (DH, H) over time.

[0331] In addition, the processing unit 22 may identify either the target objects (301 to 305) or their movements based on the positional relationship (DH, H) of the distributions (CL, CL1, CL2) in a predetermined direction (height direction) and the change in the positional relationship (DH, H) over time.

[0332] Furthermore, the processing unit 22 may identify either the target objects (301 to 305) or their movements based on either the positional relationship (DH, H) of the distributions (CL, CL1, CL2) in a predetermined direction (height direction) or the change in the positional relationship (DH, H) over time.

[0333] In this way, the processing unit 22 identifies the objects (301 to 305) based on at least one of the positional relationship (DH, H) between the ends of the distribution of multiple moving points (CL, CL1, CL2) in a predetermined direction (height direction) and the change in the positional relationship over time, thereby making it even easier to identify the objects (301 to 305).

[0334] (2-4-6) Calculation of time differences at multiple different time intervals The processing unit 22 performs the process of calculating the time differences at multiple different time intervals (T, 2×T, ... 5×T: see Figure 10) to obtain multiple different time differences (ΔA1, ΔA2, ΔA3, etc.).

[0335] This makes it possible to improve the accuracy of identifying the objects (301 to 305) and further facilitate the identification of the objects (301 to 305).

[0336] (2-5) Operation Example The control device 2 of this embodiment, like the control device 2 of the first embodiment, basically operates according to the flowcharts of Figures 4 to 7. However, in this embodiment, in the processing shown in Figures 4 and 5, the processing from step S8 to step S10, i.e., "processing of estimating the posture of the human body and detecting non-operation behavior based on the estimation results," is replaced with processing including six steps S8a, S8b, S9a to S9c, and S10a as shown in Figure 18, i.e., "processing of identifying the type of object based on the height of the object."

[0337] (2-5-1) Processing for Determining the Height of an Object After step S7, the processing unit 22 determines whether four or more points have been placed (step S8a). If it is determined that four or more points have not yet been placed (No in step S8a), the processing returns to step S1 (see FIG. 4).

[0338] If it is determined in step S8a that four or more points have been arranged (Yes), processing unit 22 determines whether two or more cloud points have been formed (step S8b). If it is determined that two or more cloud points have not yet been formed (No in step S8ba), the process returns to step S1 (see FIG. 4).

[0339] If it is determined in step S8b that two or more clouds of points have been formed (Yes), the processing unit 22 determines the point cloud with the highest height among the two or more formed point clouds as the upper end of the object (step S9a). Next, the processing unit 22 determines the point cloud with the lowest height among the two or more formed point clouds as the lower end of the object (step S9b). Next, the processing unit 22 determines the difference in elevation between the upper end and the lower end as the height of the object (step S9c).

[0340] Next, the processing unit 22 executes a type identification process to identify the type of the object based on the height of the object (step S10a), after which the process proceeds to step S11 (see FIG. 5).

[0341] (2-5-2) Type Identification Processing The type identification processing in step S10a is executed according to the flowchart of FIG. 19, for example.

[0342] In this example, the memory of the control device 2 pre-stores a first threshold value, a second threshold value (where the first threshold value is smaller than the second threshold value), and a plurality of pieces of feature information corresponding to a plurality of types of objects. The plurality of types of objects are, for example, a cleaning robot 303, a fan 304, a cat 305, and a human body 301. The human body 301 includes three types of bodies: an adult, a child, and a baby. The plurality of pieces of feature information include first feature information corresponding to the cleaning robot 303 as shown in FIG. 20A , second feature information corresponding to the fan 304 as shown in FIG. 20B , third feature information corresponding to the cat 305 as shown in FIGS. 21A and 21B , fourth feature information corresponding to an adult, fifth feature information corresponding to a child, and sixth feature information corresponding to a baby as shown in FIG. 13A .

[0343] It should be noted that a child is shorter in height (height in a standing position) than the adult shown in Fig. 13A. Furthermore, a baby is in a lying position, so is even shorter than a child.

[0344] The first characteristic information corresponding to the cleaning robot 303 is information indicating that "the height (DH, H) is equal to or less than a first threshold, and the robot moves periodically." The second characteristic information corresponding to the electric fan 304 is information indicating that "the height is greater than the first threshold and equal to or less than a second threshold, and the robot moves periodically."

[0345] The third feature information corresponding to the cat is information indicating that "the height is equal to or less than the first threshold, and the point cloud extends horizontally when it starts to move." The state of the point cloud before it starts to move is shown in FIG. 21A, and the state of the point cloud after it starts to move is shown in FIG. 21B. The horizontal (X direction) extent (DW, W) of the point cloud (CL1, CL2) in the state of FIG. 21B is relatively larger than the horizontal (X direction) extent (DW, W) of the point cloud (CL1, CL2) in the state of FIG. 21B.

[0346] The fourth feature information corresponding to an adult is information indicating that "the height is greater than the second threshold and the body moves non-periodically." The fifth feature information corresponding to a child's body is information indicating that "the height is greater than the first threshold and is equal to or less than the second threshold and the body moves non-periodically." The sixth feature information corresponding to a baby is information indicating that "the height is less than the second threshold and the body moves non-periodically."

[0347] In the type identification process, first, the processing unit 22 determines whether the height of the object is equal to or less than a first threshold value (step S101). If it is determined that the height of the object is not equal to or less than the first threshold value (No in step S101: that is, less than the first threshold value), the process returns to the upper flowchart (see FIG. 18).

[0348] If it is determined in step S101 that the height of the object is equal to or less than the first threshold (Yes), the processing unit 22 determines whether the movement of at least one of the formed two or more point clouds is periodic (step S102). If it is determined that the movement of none of the point clouds is periodic (No in step S102), the processing proceeds to step S104.

[0349] If it is determined in step S102 that the movement of at least one point cloud is periodic (Yes), the processing unit 22 sets the type information indicating the type of the object to "cleaning robot" based on the first feature information (step S103). After that, the processing returns to the upper-level flowchart (see FIG. 18).

[0350] In step S104, the processing unit 22 determines whether the point cloud extends laterally (in the X direction) when it starts to move. If it is determined that the point cloud does not extend laterally (in the X direction) when it starts to move (No in step S104), the processing proceeds to step S106.

[0351] If it is determined in step S104 that the point cloud has extended horizontally (in the X direction) when it starts to move (Yes), the processing unit 22 sets the type information to "cat" based on the third feature information (step S105). After that, the processing returns to the upper-level flowchart (see FIG. 18).

[0352] In step S106, the processing unit 22 sets the type information to "baby" based on the sixth characteristic information. Thereafter, the processing returns to the upper-level flowchart (see FIG. 18).

[0353] In step S107, it is determined whether the height of the object is greater than the first threshold and equal to or less than the second threshold. If it is determined that the height of the object is greater than the first threshold and not equal to or less than the second threshold (No in step S107: greater than the second threshold), the process proceeds to step S111.

[0354] If it is determined in step S107 that the height of the object is greater than the first threshold and less than or equal to the second threshold (Yes), the processing unit 22 determines whether the movement of at least one of the two or more formed point clouds is periodic (step S108).If it is determined that the movement of none of the point clouds is periodic (No in step S108), the processing proceeds to step S110.

[0355] If it is determined in step S108 that the movement of at least one point cloud is periodic (Yes), the processing unit 22 sets the type information to "electric fan" based on the second feature information (step S109). After that, the processing returns to the upper-level flowchart (see FIG. 18).

[0356] In step S110, the processing unit 22 sets the type information to "child" based on the fifth characteristic information, and then the process returns to the upper-level flowchart (see FIG. 18).

[0357] In step S111, the processing unit 22 sets the type information to "adult" based on the fourth characteristic information, after which the processing returns to the upper-level flowchart (see FIG. 18).

[0358] (2-6) Detection Method and Program The detection method is a detection method for detecting objects (301-305) within a region R1 using a received signal, which is a signal based on a reflected wave Re of a radio wave Tr transmitted in a predetermined direction (height direction). The detection method includes a processing step for acquiring information about the positions of the objects (301-305) within the region R1 by performing a distance measurement process based on the received signal. The processing steps are steps S8 to S10 in the first embodiment, and steps S8a to S10a and steps S101 to S111 in the second embodiment. In the processing steps (S8 to S10; S8a to 10a and S101 to S111), multiple pieces of position information corresponding to multiple parts that make up the object (301 to 305) are obtained, and the object (301 to 305) is identified based on the positional relationship (height difference DH, height H of the rectangular solid 502) between both ends in a predetermined direction (height direction) of the distribution of multiple points (CL, CL1, CL2) that correspond to the multiple pieces of position information.

[0359] The program is a program for causing one or more processors to execute the detection method.

[0360] (3) Modifications Next, modifications of the present embodiment will be described. Note that the description of the previously mentioned items will be omitted or simplified, and only the differences will be described in detail.

[0361] (3-1) First Modification of Detection System In the process of FIG. 18, following the type identification process of step S10a, an individual identification process shown in FIG. 22 may be further executed.

[0362] In this example, two pieces of personal information about two adults and two conditions corresponding to the two pieces of personal information are pre-stored in the memory of the control device 2. One of the two pieces of personal information is about AA, and includes identification information "AA" that identifies AA, and AA's personal information "H=180 cm, W=40 cm." The other of the two pieces of personal information is about BB, and includes identification information "BB" that identifies BB, and BB's personal information "H=160 cm, W=50 cm."

[0363] AA's personal information "H=180 cm, W=40 cm" corresponds to the point distribution shown in FIG. 23A, and BB's personal information "H=160 cm, W=50 cm" corresponds to the point distribution shown in FIG. 23B.

[0364] Of the two conditions, the first condition corresponding to the personal information "H=180 cm, W=40 cm" is, for example, "175≦H<185 and 35≦W<45", and the second condition corresponding to the personal information "H=160 cm, W=50 cm" is, for example, "155≦H<165 and 45≦W<55".

[0365] 22, the processing unit 22 first determines whether the type information is "adult" (step S201). If it is determined that the type information is not "adult" (No in step S201), the processing returns to the upper flowchart (see FIG. 18).

[0366] If it is determined in step S201 that the type information is "adult" (Yes), the processing unit 22 determines whether the height H and width W of the point cloud satisfy the first condition (step S202). If it is determined that the height H and width W of the point cloud do not satisfy the first condition (No in step S202), the processing proceeds to step S204.

[0367] If it is determined in step S202 that the height H and width W of the point cloud satisfy the first condition (Yes), the processing unit 22 sets "AA" as the identification information (step S203). After that, the processing returns to the upper-level flowchart (see FIG. 18).

[0368] In step S204, the processing unit 22 determines whether the height H and width W of the point cloud satisfy the second condition. If it is determined that the height H and width W of the point cloud do not satisfy the second condition (No in step S204), the processing proceeds to step S206.

[0369] If it is determined in step S204 that the height H and width W of the point cloud satisfy the second condition (Yes), the processing unit 22 sets "BB" as the identification information (step S205). After that, the processing returns to the upper-level flowchart (see FIG. 18).

[0370] In step S206, the processing unit 22 sets the identification information to "unregistered person." Thereafter, the process returns to the upper-level flowchart (see FIG. 18).

[0371] 18, the process proceeds to step S11 (see FIG. 5). In the automatic control of step S11, if the type information about the detected object is "adult," control may be performed taking into account identification information that identifies an individual. For example, control may be performed such that the television 200b is powered on if the identification information is "AA," but the television 200b is not powered on if the identification information is "BB."

[0372] In this modification, the height H may be the difference in elevation DH, and the width W may be the distance DW. In addition, in this modification, even if the target is a child, the individual may be identified in the same manner as in the case of an adult. Furthermore, the specific numerical values ​​of the first and second conditions are merely examples and may be changed as appropriate.

[0373] According to this modification, when the target object (301 to 305) is identified as a human body 301, it is possible to further identify the individual corresponding to the human body 301 based on pre-stored personal information, the height H (DH), and the width W (DW). As a result, it is possible to, for example, enable different device control for each individual.

[0374] (3-2) Second Modification of Detection System The detection system 100 does not have to include the radio wave sensor 1. In this modification, the radio wave sensor 1 is disposed outside the detection system 100. Like the radio wave sensor 1 of the first or second embodiment, the radio wave sensor 1 transmits radio waves Tr in a predetermined direction (height direction), receives reflected waves Re, and outputs a received signal. However, the output in this modification is transmission of the received signal to the detection system 100. In the detection system 100, the reception unit 21 receives the received signal from the radio wave sensor 1, and the processing unit 22 performs processing based on the received signal received by the reception unit 21.

[0375] In this modification, as in the first or second embodiment, it is possible to facilitate identification of the objects (301 to 305).

[0376] (3-3) Third Modification of Detection System The detection system 100 may be a radio wave sensor 1. Similar to the radio wave sensor 1 in the first or second embodiment, the radio wave sensor 1 in this modification transmits radio waves Tr in a predetermined direction (height direction), receives reflected waves Re of the radio waves Tr, and detects objects (301 to 305) within the region R1 using a received signal that is based on the reflected waves Re. More specifically, the radio wave sensor 1 in this modification includes a control device 2, as shown in FIG. 24 .

[0377] In this modification, as in the first or second embodiment, it is possible to facilitate identification of the objects (301 to 305).

[0378] (3-4) Other Modifications of the Detection System Detection system 100 may be a wiring device including control device 2. This wiring device may further include radio wave sensor 1. The wiring device may be, for example, an outlet to which lighting device 200a shown in FIG. 2 is connected.

[0379] The detection system 100 may be a lighting fixture including a control device 2 and a lighting device main body. The lighting device main body includes, for example, a light-emitting element such as an LED, a drive circuit for driving the light-emitting element, and a power supply for supplying power to the drive circuit. This lighting fixture may further include a radio wave sensor 1. For example, the lighting fixture 200a shown in FIG. 2 may include the radio wave sensor 1 and the control device 2 built in.

[0380] (3-5) Modified Example of Radio Wave Sensor Arrangement and Radio Wave Transmission Direction The radio wave sensor 1 does not necessarily have to be located above the region R1. In this modified example, the radio wave sensor 1 is located to the side of the region R1. In this case, the transmission direction of the radio waves Tr, that is, the "predetermined direction," is the horizontal direction (sideways).

[0381] According to this modified example, the detection accuracy of the object may be reduced compared to when the predetermined direction is the height direction (vertical downwards). However, the detection accuracy can be improved by having the processing unit 22 identify the object (301 to 305) based on the positional relationship between the ends of the distribution (CL, CL1, CL2) in the horizontal direction (for example, the X direction, or the Y direction, or each of the X direction and the Y direction).

[0382] Furthermore, the radio wave sensor 1 may be located on either side of the region R1, as long as it transmits radio waves Tr in a predetermined direction relative to the region R1. In this way, the processing unit 22 identifies the target objects (301 to 305) based on the positional relationship between the ends of the distributions (CL, CL1, CL2) in the predetermined direction and the change in that positional relationship over time, as well as the spread in a plane perpendicular to the predetermined direction and the change in that spread over time, thereby improving detection accuracy.

[0383] (3-6) Modified Examples of Detection Method The detection process of the detection system 100 may be a process that can detect not only moving objects but also stationary objects. For example, such a process may be a process in which the calculation of the time difference between received signals is omitted from the detection process of the second embodiment. While omitting the calculation of the time difference may make it difficult to identify the movement of an object, it is possible to identify the object based on the positional relationship between the ends of the point cloud distribution in a predetermined direction, the spread in a plane perpendicular to the predetermined direction, and the like.

[0384] (4) Summary The detection system (100) according to the first aspect is a detection system (100) that detects objects (301-305) within a region (R1) using received signals. The received signals are signals based on reflected waves (Re) of radio waves (transmitted waves Tr) transmitted in a predetermined direction (height (Z) direction). The detection system (100) includes a processing unit (22) that acquires information about the positions of the objects (301-305) within the region (R1) by performing a ranging process based on the received signals. The processing unit (22) acquires multiple pieces of position information corresponding to multiple parts constituting the objects (301-305), and identifies the objects (301-305) based on the positional relationship (height difference DH, height H of the rectangular parallelepiped 502) between both ends in the predetermined direction (height (Z) direction) of a distribution of multiple points (CL, CL1, CL2) corresponding to the multiple pieces of position information.

[0385] According to this aspect, when detecting the objects (301-305) using the reflected waves (Re) of the radio waves (Tr) transmitted in a predetermined direction (height direction), the objects (301-305) can be easily identified by taking into consideration the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction). Note that the predetermined direction (height direction) is preferably the height direction, but the objects (301-305) can also be easily identified in directions other than the height direction.

[0386] In the detection system (100) according to the second aspect, in the first aspect, the positional relationship (DH, H) includes the distance (DH, H) between both ends.

[0387] According to this aspect, by taking into consideration at least the distance (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in a predetermined direction (height direction), it is possible to facilitate and improve the accuracy of identifying the target objects (301 to 305).

[0388] In the detection system (100) according to the third aspect, in the first or second aspect, the processing unit (22) identifies the objects (301 to 305) further based on the spread (the distance DW between the ends in the lateral direction, the width W of the rectangular parallelepiped 502) of the distributions (CL, CL1, CL2) in a plane (a horizontal plane (an XY plane or a plane parallel to the XY plane)) having a normal direction in a predetermined direction (height direction).

[0389] According to this aspect, by (further) considering the spread (DW, W) of the distribution of multiple points (CL, CL1, CL2) in a plane (horizontal plane) perpendicular to a predetermined direction (height direction), it is possible to facilitate identification of the target objects (301 to 305).

[0390] In the detection system (100) according to the fourth aspect, in the third aspect, the positional relationship includes a distance (DH, H) between both ends and a spread (DW, W) in a plane (horizontal plane) of the distribution (CL, CL1, CL2). The processing unit (22) identifies the type of the target (301-305) based on the distance (DH, H) and the spread (DW, W).

[0391] According to this aspect, the type of the target object (301-305) can be identified by considering the distance (DH, H) between the ends of the distribution (CL, CL1, CL2) in a predetermined direction (height direction) and the spread (DW, W) in a plane (horizontal plane) perpendicular to the predetermined direction (height direction).

[0392] In the detection system (100) according to the fifth aspect, in the fourth aspect, personal information including information on the dimensions of the individual's body is stored in advance for each of the multiple individuals. When the processing unit (22) has determined that the type is a human body (301), it further identifies the individual among the multiple individuals corresponding to the human body (301) based on each of the multiple personal information corresponding to the multiple individuals, the distance (DH, H), and the spread (DW, W).

[0393] According to this aspect, when the type of object (301-305) is identified as a human body (301), the individual corresponding to the human body (301) can be further identified by taking into account the distance (DH, H) and the spread (DW, W).

[0394] In a detection system (100) according to a sixth aspect, in the first aspect, radio waves (Tr) are transmitted in each of a plurality of frames. A processing unit (22) calculates a plurality of time differences by repeatedly performing a process of calculating a time difference, which is the difference between two received signals received in two different frames among the plurality of frames. The processing unit (22) acquires, from the plurality of time differences, a distribution (CL, CL1, CL2) of one or more moving points, which are points whose time difference is equal to or greater than a threshold value. The processing unit (22) identifies the target (301-305) based on at least one of the positional relationship (DH, H) and time change of the positional relationship in a predetermined direction (height direction) of the distribution (CL, CL1, CL2) of the one or more moving points, and the spread (DW, W) and time change of the spread (DW, W) in a plane (XY plane or a plane parallel to the XY plane) having the predetermined direction (height direction) as its normal direction.

[0395] According to this aspect, by taking into consideration at least one of the positional relationship (DH, H) and the change in the positional relationship over time of the distribution of multiple moving points (CL, CL1, CL2) obtained from multiple time differences, and the spread (DW, W) within a plane (horizontal plane) and the change in the spread (DW, W) over time, it is possible to further facilitate the identification of the target objects (301 to 305).

[0396] In the detection system (100) according to the seventh aspect, in the sixth aspect, the processing unit (22) identifies the movement of the object (301-305) based on the change over time in the spread (DW, W) in a plane (horizontal plane) of the distribution (CL, CL1, CL2) of one or more moving points.

[0397] According to this aspect, by taking into consideration the change over time in the spread (DW, W) of the distribution of multiple moving points (CL, CL1, CL2) within a plane (horizontal plane), it is possible to further facilitate the identification of the target objects (301 to 305).

[0398] In the detection system (100) according to the eighth aspect, in the sixth or seventh aspect, the processing unit (22) identifies the movement of the object (301-305) based on the change over time in the positional relationship (DH, H) in a predetermined direction (height direction) of the distribution of one or more moving points (CL, CL1, CL2).

[0399] According to this aspect, by taking into consideration the change over time in the positional relationship (DH, H) in a predetermined direction (height direction) of the distribution of multiple moving points (CL, CL1, CL2), it is possible to further facilitate the identification of the target object (301 to 305).

[0400] In the detection system (100) according to the ninth aspect, in the sixth aspect, the processing unit (22) identifies the object (301-305) or the motion of the object (301-305) based on at least one of the spread (DW, W) of one or more moving point distributions (CL, CL1, CL2) in a plane (XY plane or a plane parallel to the XY plane) having a predetermined direction (height direction) as its normal direction, and the change in the spread (DW, W) over time.

[0401] According to this aspect, by taking into consideration at least one of the spread (DW, W) of the distribution of multiple moving points (CL, CL1, CL2) in a plane perpendicular to a predetermined direction (height direction) and the change in the spread (DW, W) over time, it is possible to further facilitate the identification of the target objects (301 to 305).

[0402] In the detection system (100) according to the tenth aspect, in the sixth or ninth aspect, the processing unit (22) identifies the object (301-305) or the motion of the object (301-305) based on at least one of the positional relationship (DH, H) in a predetermined direction (height direction) of the distribution of one or more moving points (CL, CL1, CL2) and the change over time in the positional relationship (DH, H).

[0403] According to this aspect, by taking into consideration at least one of the positional relationship (DH, H) between the ends of the distribution of multiple moving points (CL, CL1, CL2) in a predetermined direction (height direction) and the change in the positional relationship over time, it is possible to further facilitate the identification of the target objects (301 to 305).

[0404] In the detection system (100) according to an eleventh aspect, in any one of the sixth to tenth aspects, the processing unit (22) performs the process of calculating the time difference at a plurality of time intervals that are different from each other.

[0405] According to this aspect, it is possible to improve the accuracy of identifying the objects (301 to 305) and further facilitate the identification of the objects (301 to 305).

[0406] A detection system (100) according to a twelfth aspect is any one of the first to eleventh aspects, further comprising a radio wave sensor (1). The radio wave sensor (1) transmits radio waves (Tr) in a predetermined direction (height direction), receives reflected waves (Re), and outputs a received signal.

[0407] According to this aspect, when radio waves (Tr) are transmitted in a predetermined direction (height direction) and the target objects (301 to 305) are detected using the reflected waves (Re) of the transmitted radio waves (Tr), the target objects (301 to 305) can be easily identified by taking into consideration the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction).

[0408] In the detection system (100) according to the thirteenth aspect, in the twelfth aspect, the radio wave sensor (1) is located above the region (R1), and the predetermined direction (height direction) is the height direction of the region (R1).

[0409] According to this aspect, when the radio wave sensor (1) transmits radio waves (Tr) from above the region (R1) and detects the target objects (301-305) using the reflected waves (Re) of the transmitted radio waves (Tr), the target objects (301-305) can be easily identified by taking into consideration the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the height direction of the region (R1).

[0410] A radio wave sensor (1) according to a fourteenth aspect transmits radio waves (transmission waves Tr) in a predetermined direction (height direction), receives reflected waves (Re) of the radio waves (Tr), and detects objects (301-305) within a region (R1) using a received signal based on the reflected waves (Re). The radio wave sensor (1) includes a processing unit (22) that acquires information about the positions of the objects (301-305) within the region (R1) by performing a distance measurement process based on the received signal. The processing unit (22) acquires multiple pieces of position information corresponding to multiple parts that make up the objects (301-305), and identifies the objects (301-305) based on the positional relationship (height difference DH, height H of the rectangular parallelepiped 502) between both ends in the predetermined direction (height direction) of a distribution of multiple points (CL, CL1, CL2) that correspond to the multiple pieces of position information.

[0411] According to this aspect, when detecting an object (301 to 305) using a reflected wave (Re) of a radio wave (Tr) transmitted in a predetermined direction (height direction), the object (301 to 305) can be easily identified by taking into consideration the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction).

[0412] The radio wave sensor (1) according to the fifteenth aspect is the fourteenth aspect, and is positioned above the region (R1). The predetermined direction (height direction) is the height direction of the region (R1).

[0413] According to this aspect, when radio waves (Tr) are transmitted from above the region (R1) and the target objects (301-305) are detected using the reflected waves (Re) of the transmitted radio waves (Tr), the target objects (301-305) can be easily identified by taking into consideration the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the height direction of the region (R1).

[0414] A detection method according to a sixteenth aspect is a detection method for detecting objects (301-305) within a region (R1) using a received signal, which is a signal based on a reflected wave (Re) of a radio wave (transmitted wave Tr) transmitted in a predetermined direction (height direction). The detection method includes processing steps (steps S8 to S10 in the first embodiment; steps S8a to S10a and steps S101 to S111 in the second embodiment) for acquiring information about the positions of the objects (301-305) within the region (R1) by performing a distance measurement process based on the received signal. In the processing steps (S8 to S10; S8a to 10a and S101 to S111), multiple pieces of position information corresponding to multiple parts that make up the object (301 to 305) are obtained, and the object (301 to 305) is identified based on the positional relationship (height difference DH, height H of the rectangular solid 502) between both ends in a predetermined direction (height direction) of the distribution of multiple points (CL, CL1, CL2) that correspond to the multiple pieces of position information.

[0415] According to this aspect, when detecting an object (301 to 305) using a reflected wave (Re) of a radio wave (Tr) transmitted in a predetermined direction (height direction), the object (301 to 305) can be easily identified by taking into consideration the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction).

[0416] A program according to a seventeenth aspect is for causing one or more processors to execute the detection method of the sixteenth aspect.

[0417] According to this aspect, when detecting an object (301 to 305) using a reflected wave (Re) of a radio wave (Tr) transmitted in a predetermined direction (height direction), the object (301 to 305) can be easily identified by taking into consideration the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction).

[0418] 100, 100A Detection system (device control system) 1 Radio wave sensor 22 Processing unit 200c Moving object (electric blinds) 301 Moving object (human body) 302 Stationary object 303 Moving object (cleaning robot) 304 Moving object (electric fan) 305 Moving object (cat) 502 Point cloud (distribution) CL, CL1, CL2 Cluster group (distribution) Tr Transmitted wave (radio wave) Re Reflected wave R1 Region R1a Wall surface R1b Ceiling ΔA1 to ΔA5 Inter-frame difference (time difference) Fr1 to Fr5 Frames

Claims

1. A detection system that detects an object within an area using a received signal, which is a signal based on a reflected wave of a radio wave transmitted in a specified direction, comprising a processing unit that acquires information regarding the position of the object within the area by performing a ranging process based on the received signal, wherein the processing unit acquires a plurality of pieces of position information corresponding to a plurality of parts that make up the object, and identifies the object based on the positional relationship between both ends in the specified direction of a distribution of a plurality of points that respectively correspond to the plurality of pieces of position information.

2. The detection system according to claim 1, wherein the positional relationship includes a distance between the two ends.

3. The detection system according to claim 1 or 2, wherein the processing unit identifies the object based further on the spread of the distribution in a plane having the predetermined direction as a normal direction.

4. The detection system of claim 3, wherein the positional relationship includes the distance between the two ends and the spread of the distribution on the plane, and the processing unit identifies the type of the object based on the distance and the spread.

5. The detection system of claim 4, wherein personal information including information regarding the dimensions of the individual's body is pre-stored for each of a plurality of individuals, and when the processing unit determines that the type is the human body, it further identifies the individual among the plurality of individuals that corresponds to the human body based on each of the plurality of personal information corresponding to the plurality of individuals, the distance and the spread.

6. The detection system of claim 1, wherein the radio waves are transmitted in each of a plurality of frames, and the processing unit calculates a plurality of time differences by repeatedly performing a process of calculating a time difference, which is the difference between two received signals received in two different frames among the plurality of frames, obtains from the plurality of time differences the distribution of one or a plurality of moving points, which are the points whose time difference is equal to or greater than a threshold value, and identifies the object based on at least one of the positional relationship and the change in the positional relationship over time in the specified direction of the distribution of the one or a plurality of moving points, and the spread in a plane having the specified direction as a normal direction and the change in the spread over time.

7. The detection system according to claim 6, wherein the processing unit determines the motion of the object based on the time change in the extent of the distribution of the one or more moving points in the plane.

8. The detection system according to claim 6 or 7, wherein the processing unit identifies the motion of the object based on the time change in the positional relationship of the distribution of the one or more moving points in the specified direction.

9. The detection system of claim 6, wherein the processing unit identifies the object or identifies the motion of the object based on at least one of the extent and the change in the extent over time of the distribution of the one or more moving points in a plane having the specified direction as a normal direction.

10. The detection system described in claim 6 or 9, wherein the processing unit identifies the object or identifies the motion of the object based on at least one of the positional relationship and the change over time of the positional relationship in the specified direction of the distribution of the one or more moving points.

11. The detection system according to any one of claims 6 to 10, wherein the processing unit performs the process of calculating the time difference at a plurality of time intervals that are different from one another.

12. The detection system according to any one of claims 1 to 11, further comprising a radio wave sensor that transmits the radio waves in the predetermined direction, receives the reflected waves, and outputs the received signal.

13. The detection system according to claim 12, wherein the radio wave sensor is positioned above the area, and the predetermined direction is a height direction of the area.

14. A radio wave sensor that transmits radio waves in a predetermined direction, receives reflected waves of the radio waves, and detects an object within an area using a received signal that is based on the reflected waves, and has a processing unit that acquires information regarding the position of the object within the area by performing a ranging process based on the received signal, wherein the processing unit acquires a plurality of pieces of position information respectively corresponding to a plurality of parts that make up the object, and identifies the object based on the positional relationship between both ends in the predetermined direction of a distribution of a plurality of points that respectively correspond to the plurality of pieces of position information.

15. The radio wave sensor according to claim 14, which is located above the area, and the predetermined direction is a height direction of the area.

16. A detection method for detecting an object within an area using a received signal, which is a signal based on a reflected wave of a radio wave transmitted in a specified direction, comprising a processing step for acquiring information regarding the position of the object within the area by executing a distance measurement process based on the received signal, wherein the processing step acquires a plurality of pieces of position information respectively corresponding to a plurality of parts constituting the object, and identifies the object based on the positional relationship between both ends in the specified direction of a distribution of a plurality of points respectively corresponding to the plurality of pieces of position information.

17. A program for causing one or more processors to execute the detection method according to claim 16.

Images