Detection system, radio wave sensor, detection method, and program

Abstract

The present invention aims to provide a detection system that makes it easier to identify an object when using the reflected wave of an electromagnetic wave transmitted in a predetermined direction. The detection system (100) detects objects within a region using a received signal. The received signal is based on the reflected wave of an electromagnetic wave transmitted in a predetermined direction. The detection system (100) includes a processing unit (22) that acquires information related to the position of an object within the region by performing ranging processing based on the received signal. The processing unit (22) acquires multiple position information corresponding to multiple parts constituting the object, and determines the object based on the positional relationship between the two ends of multiple points corresponding to the multiple position information distributed in a predetermined direction.

Description

Technical Field

[0001] The present invention relates to a detection system, a radio wave sensor, a detection method and a program. More specifically, the present invention relates to a detection system, a radio wave sensor, a detection method and a program for performing detection related to an object such as a human body by using a radio wave sensor. Prior Art

[0002] Japanese Patent Application Publication No. 2002-236171 (hereinafter referred to as "Document 1") describes an intruder detection device that receives reflected waves from objects when radio waves are transmitted from different directions. Based on the intensity of the received reflected waves (received wave intensity), the device detects intruders within its detection range. This intruder detection device detects moving objects (moving bodies), such as intruders, based on real-time changes in received wave intensity.

[0003] In the intrusion detection device described in Document 1, it is sometimes difficult to determine the object by the direction of the radio waves. For example, it is difficult to determine the type of the object, such as whether the object is a person, a sweeping robot, or a pet. Or it is difficult to determine the movement of the object, such as whether the object is moving or stationary, or whether the object is moving as a whole or partially when moving. Summary of the Invention

[0004] The object of the present invention is to provide a detection system, radio wave sensor, detection method and program, which can make it easier to determine the object when using the reflected wave of the radio wave sent in a predetermined direction to detect the object.

[0005] According to one aspect of the present invention, a detection system detects objects within an area using a received signal. The received signal is based on a reflected wave of radio waves transmitted in a predetermined direction. The detection system includes a processing unit that performs ranging processing based on the received signal to obtain information related to the position of the object within the area. The processing unit obtains multiple pieces of position information corresponding to multiple parts of the object and identifies the object based on the positional relationship between the two ends of a distribution of multiple points corresponding to the multiple pieces of position information in the predetermined direction.

[0006] According to one aspect of the present invention, a radio wave sensor transmits radio waves in a predetermined direction, receives reflected waves from the radio waves, and uses a signal based on the reflected waves, i.e., a received signal, to detect objects within an area. The radio wave sensor includes a processing unit that performs ranging processing based on the received signal to obtain information related to the position of the object within the area. The processing unit obtains multiple pieces of position information corresponding to multiple parts of the object and identifies the object based on the positional relationship between the two ends of a distribution of multiple points corresponding to the multiple pieces of position information in the predetermined direction.

[0007] According to one aspect of the present invention, a detection method uses a received signal to detect an object within an area. The received signal is based on a reflected wave of a radio wave transmitted in a predetermined direction. The detection method includes a processing step of obtaining information related to the position of the object within the area by performing ranging processing based on the received signal. In this processing step, multiple pieces of position information corresponding to multiple parts of the object are obtained, and the object is identified based on the positional relationship between the two ends of a distribution of multiple points corresponding to the multiple pieces of position information in the predetermined direction.

[0008] According to one aspect of the present invention, a program is used to enable more than one processor to execute the detection method. Simple diagram description

[0009] FIG1 is a block diagram of a detection system (equipment control system) according to a first embodiment of the present invention. FIG2 is a conceptual diagram showing a room using the same device control system as above. FIG3 is a graph showing changes in the frequency of the transmission wave transmitted by the radio wave sensor constituting the above device control system. FIG4 is a flow chart illustrating a portion of the operation of the control device constituting the same equipment control system. FIG5 is a flow chart illustrating another part of the above operation. FIG. 6 is a flow chart illustrating a process of obtaining a difference between multiple frames included in the above operation. FIG. 7 is a flow chart illustrating the posture estimation process included in the above operation. FIG8 is a waveform diagram for explaining an example of inter-frame difference (one-to-one inter-frame difference). FIG. 9A is a frequency spectrum diagram showing the FFT result in the current frame Fr0. FIG9B is a frequency spectrum diagram showing the FFT result in the subsequent frame Fr1. FIG. 9C is a frequency spectrum diagram showing the difference between FFT results. FIG. 10 is a waveform diagram for explaining another example of inter-frame difference (one-to-many inter-frame difference). FIG11 is a conceptual diagram for explaining the characteristics of various changes in the human body. FIG. 12 is a waveform diagram for explaining an example of obtaining a one-to-multiple frame difference corresponding to various variations described above. FIG. 13A is a distribution diagram showing an example of cluster distribution in a standing posture, which is one of human body postures. FIG. 13B is a distribution diagram showing an example of cluster distribution in a sitting position. FIG13C is a distribution diagram showing an example of cluster distribution in a prone position. FIG. 14A is a conceptual diagram showing a three-dimensional figure surrounding a cluster in a standing position. FIG14B is a conceptual diagram showing a three-dimensional figure surrounding a cluster in a sitting position. FIG14C is a conceptual diagram showing a three-dimensional figure surrounding a cluster in a lying position. FIG15 is a diagram showing the data structure of the automatic control information group. FIG16 is a diagram showing the data structure of the control history record information. FIG17 is a block diagram of a modified example of the above equipment control system. FIG. 18 is a flow chart illustrating a portion of the operation of the control device constituting the detection system according to the second embodiment of the present invention. FIG. 19 is a flow chart illustrating the type determination processing included in the above operation. FIG. 20A is a distribution diagram showing an example of cluster distribution corresponding to a cleaning robot. FIG. 20B is a distribution diagram showing an example of cluster distribution corresponding to electric fans. FIG. 21A is a distribution diagram showing an example of cluster distribution before the cat starts moving. FIG. 21B is a distribution diagram showing an example of cluster distribution after the cat starts moving. FIG. 22 is a flow chart illustrating the individual determination processing further included in the above operations. FIG. 23A is a distribution diagram showing an example of cluster distribution corresponding to the human body of Mr. AA. FIG. 23B is a distribution diagram showing an example of cluster distribution corresponding to Mr. BB's body. FIG24 is a block diagram of a radio wave sensor according to a modified example of the above detection system. Implementation Method

[0010] (First embodiment) 1 to 17, a first embodiment of the present invention will be described below.

[0011] (1) Overview of equipment control system In this embodiment, the detection system of the present invention includes, in addition to object detection functions (e.g., functions for detecting objects and identifying the type of the detected objects, functions for identifying the movement of the identified objects, and functions for estimating human body postures related to objects identified as human bodies), a device control system 100 that also includes device control functions. As shown in FIG2 , device control system 100 of this embodiment uses radio wave sensor 1 to detect moving objects (hereinafter sometimes referred to as "moving objects") among the objects, and in particular, to estimate human body postures.

[0012] An object is an object that the detection system detects. Examples of an object include a human body 301, the body of an animal other than human 301 (such as a pet), a moving object other than an animal (such as a motorized blind 200c), and an immobile object (such as a stationary object 302). Object detection involves, for example, detecting an object that meets any of a plurality of pre-defined characteristics. Object identification involves determining the type of object based on the detected characteristics.

[0013] Determining the object's motion, for example, involves determining whether the object is moving, and when the object is moving, determining which of a plurality of pre-defined motions the object's motion corresponds to. Determining whether the object is moving can also involve detecting a moving object, i.e., a moving body.

[0014] Furthermore, determining the motion may include, for example, determining the time when the object starts moving and determining the time when the object stops moving. Furthermore, determining the motion may include, for example, determining whether the motion is periodic or non-periodic (in other words, whether the motion is regular or irregular).

[0015] The so-called inference related to the human body 301 is, for example, inferring that the object is the human body 301 or a moving object other than the human body 301, and further inferring the posture of the human body 301.

[0016] The postures of human body 301 include, but are not limited to, standing, sitting, and lying down (see Figures 13A to 13C). Furthermore, device control system 100 detects various movements of human body 301 based on changes in estimated posture, thereby detecting the person's actions and controlling device 200 based on the detection results.

[0017] Furthermore, changes in posture include, but are not limited to, changes between standing, sitting, and lying positions. Changes in posture also include continued unchanged posture (e.g., after changing to a sitting position, the posture does not change back to a standing position even after a predetermined period of time). Furthermore, the actions of the detection target are particularly, for example, non-operational actions (described below), but may also be operational actions (described below).

[0018] (1-1) Radio wave sensor The radio wave sensor 1 transmits radio waves (transmission waves Tr) from an antenna, receives reflected waves Re when the transmission waves Tr are reflected by an object, and outputs information that can identify the position of the object (hereinafter referred to as "position-determinable information").

[0019] (1-1-1) Location information can be determined Positionable information refers to information that can determine the position of an object. The object's position is preferably three-dimensional, but may also be two-dimensional or one-dimensional. For example, the positionable information may be the FFT result group described below (information that can determine a three-dimensional position), but it may also be each of the multiple (e.g., three) FFT results that comprise the FFT result group (information that can determine a one-dimensional position).

[0020] For example, a signal representing the time difference Δt between the transmission of the transmitted wave Tr and the reception of the reflected wave Re, or a signal representing the frequency difference Δf corresponding to the time difference Δt (for example, an IF signal representing the frequency difference Δf between the FMCW-modulated transmitted wave Tr and the reflected wave Re; see Figure 3) can also be considered a type of position determination information. Furthermore, the distance calculated from the time difference Δt or frequency difference Δf can itself be information that can be used to determine the position.

[0021] (1-1-2) Antenna Furthermore, the antenna used to transmit the transmission wave Tr and the antenna used to receive the reflected wave Re may be the same antenna (hereinafter referred to as the "shared antenna") or separate antennas (hereinafter referred to as the "transmitting antenna" and the "receiving antenna"). That is, the transmission wave Tr may be transmitted from the shared antenna, and the reflected wave Re of the transmission wave Tr may be received by the shared antenna. Alternatively, the transmission wave Tr may be transmitted from the transmitting antenna, and the reflected wave Re of the transmission wave Tr may be received by the receiving antenna.

[0022] Generally speaking, to determine the three-dimensional position of an object (for example, to locate a point corresponding to a human body in virtual three-dimensional space 500), the radio wave sensor 1 must have, for example, three or more shared antennas, or one transmitting antenna and three or more receiving antennas. Furthermore, when the radio wave sensor 1 has multiple antennas, these antennas are not limited to being housed in a single housing; they can also be located in separate locations. In the latter case, each of the multiple antennas is connected to the sensor body in a manner that enables communication via wired or wireless communication.

[0023] The radio wave sensor 1 in this embodiment includes a transmitting antenna and three or more (e.g., three) receiving antennas. The positions of the transmitting antenna and the three or more receiving antennas included in the radio wave sensor 1 are known, and the four or more pieces of position information corresponding to the four or more antennas (hereinafter referred to as "antenna position information group") are pre-stored in the memory of the control device 2, for example.

[0024] However, the number of receiving antennas constituting the radio wave sensor 1 may be two or one (two means point arrangement on a plane, and one means point arrangement along a line).

[0025] (1-1-3) Send and receive actions The radio wave sensor 1 performs a transmission and reception operation. The transmission and reception operation is an operation of transmitting a transmission wave Tr toward a real space where a group of objects may exist, receiving a reflection wave Re from the real space, and outputting a signal (e.g., an IF signal) based on the transmission wave Tr and the reflection wave Re. The real space where a group of objects may exist is, for example, a space surrounded by the bottom, ceiling, and side walls of a room 400 (indoor space), and may be referred to as "real space (400)" hereinafter. Furthermore, the real space where a group of objects may exist is not limited to an indoor space, and may also be an outdoor space such as a passage or a platform.

[0026] The object group is a collection of one or more objects. In this embodiment, the object group includes one or more of the following: a human body 301, a moving object other than a human body (e.g., a motorized blind 200c, etc.; hereinafter, sometimes referred to as "a moving object other than a human body (200c)"), and a stationary object 302. In other words, the one or more objects constituting the object group are each one of the human body 301, the moving object other than a human body (200c), and the stationary object 302.

[0027] The radio wave sensor 1 performs the following transmission and reception operation at a predetermined period (e.g., once every 20 ms). The transmission and reception operation is an operation in which a transmission wave Tr is transmitted to a real space (400) where a group of objects may exist using a transmission antenna, and a reflected wave Re is received from the real space (400) using three or more reception antennas.

[0028] For example, in a real space (400), a human body 301, a moving object (200c) other than a human body, and a stationary object 302 are present. Reflected waves Re from the object groups (301, 200c, 302) are received by three or more receiving antennas. Furthermore, in the example of FIG2 , the stationary object 302 is, for example, a table 302a and a bed 302b placed on the bottom surface of the room 400. However, the bottom surface may also be used.

[0029] The transmission wave Tr is a radio wave modulated using a predetermined method. The predetermined method is, for example, but not limited to, the FMCW (Frequency Modulated Continuous Wave) method. In this embodiment, the transmission wave Tr is a radio wave modulated using the FMCW method.

[0030] (1-1-4) FMCW method The so-called FMCW method, as shown in Figure 3, is a method in which the frequency f of a transmission wave Tr (transmission signal) having a predetermined time length (the chirp length Tc of the chirp signal) is linearly increased (or decreased) from a starting frequency f0 at a predetermined slope S as time t passes.

[0031] Furthermore, as described above, the acquisition of determinable location information need not be performed for all transmission and reception operations that are repeated at a predetermined period. For example, as described later, if a predetermined time T (e.g., T = 200 ms) is defined as one frame (described later), and N transmission and reception operations are performed in one frame (N is a natural number, e.g., 10), determinable location information may be acquired for each of the N transmission and reception operations within the frame, or determinable location information may be acquired only for a predetermined one (e.g., the first transmission and reception operation).

[0032] The predetermined period is, for example, a period of 20 ms. In this embodiment, one frame is set to 200 ms, and the frequency is ten times per frame (N=10). However, the predetermined period may also be, for example, twenty times per frame (N=20), five times per frame (N=5), etc.

[0033] (1-1-5) IF signal, FFT result, and FFT result group Each time the radio wave sensor 1 performs a transmission / reception operation, it generates an IF signal for each of the three or more receiving antennas, obtains an FFT (Fast Fourier Transform) result from the IF signal, and outputs a group of FFT results. In other words, each time a transmission / reception operation is performed, the radio wave sensor 1 outputs a group of FFT results consisting of three or more FFT results corresponding to the three or more receiving antennas. However, a Fourier transform other than the FFT can be performed on the IF signal. In this case, the radio wave sensor 1 outputs a group of Fourier transform results consisting of three or more Fourier transform results corresponding to the three or more receiving antennas.

[0034] As shown in FIG3 , the so-called IF signal represents the frequency difference Δf between the transmitted wave Tr and the reflected wave Re. The IF signal represents the difference between the frequencies of the transmitted wave Tr and the reflected wave Re at time t, and is a function of time t, Δf(t). However, when the object groups (301, 302) are stationary, the IF signal represents a fixed value.

[0035] The IF signal is generated by mixing the transmission wave Tr and the reflected wave Re. The IF signal is generated during the period when the transmission wave Tr is being transmitted and the reflected wave Re is being received (i.e., the period from the start of receiving the reflected wave Re to the end of transmitting the transmission wave Tr).

[0036] The so-called FFT result is the result obtained by performing FFT on the IF signal. The FFT result is, for example, information representing the frequency spectrum (the relationship between frequency f and reflection intensity amp) as shown in FIG9A and FIG9B.

[0037] The so-called FFT result group is information consisting of three or more FFT results corresponding to three or more receiving antennas, obtained for a single transmission / reception operation. This group of FFT results can be used to determine the three-dimensional position of an object. Furthermore, by taking the difference between the complex FFT result groups, the frequency components corresponding to stationary objects 302 are removed, leaving only the frequency components corresponding to moving objects such as human body 301 (the frequency f at which amp exceeds the threshold value, and the amp value corresponding to this frequency f) (see Figures 9A to 9C). Based on these obtained frequency components, information related to the three-dimensional position and movement of moving objects such as human body 301 can be obtained.

[0038] Specifically, in this embodiment, a complex FFT result group corresponding to a series of multiple transmission and reception operations is output from the radio wave sensor 1. This output complex FFT result group is stored in a time-sequential manner in the memory of the control device 2. Meanwhile, the memory also stores the aforementioned antenna position information group. Furthermore, the control device 2 calculates the time difference between the complex FFT result groups (e.g., two adjacent FFT result groups) stored in the memory in a time-sequential manner (the time difference between the FFT results of each of the three receiving antennas), and based on the calculated difference, obtains a distance measurement result group (the two distance measurement results corresponding to the three receiving antennas). The control device 2 performs three-point positioning using the thus-obtained distance measurement result group and the pre-stored antenna position information group. This allows the acquisition of information determining the three-dimensional position of the human body 301, for example, calculation of three-dimensional coordinates.

[0039] (1-2) Attitude estimation function of equipment control system As shown in FIG. 1 , the device control system 100 includes a distance measuring unit 221 , a point arrangement unit 222 , and a posture estimation unit 223 .

[0040] (1-2-1) Distance measurement: one-to-one or one-to-many difference The distance measuring unit 221 measures the distance from the radio wave sensor 1 to the human body 301 based on the FFT result group output by the radio wave sensor 1.

[0041] Specifically, the ranging unit 221 stores the FFT result groups output by the radio wave sensor 1 over a period exceeding the predetermined period (preferably a period exceeding twice the predetermined period). Furthermore, the ranging unit 221 performs ranging processing based on the difference between one FFT result group (in this embodiment, the current FFT result group) serving as a reference FFT result group within the stored plurality of FFT result groups and at least one target FFT result group (in this embodiment, at least one previous FFT result group), thereby obtaining at least one ranging result group.

[0042] (1-2-1a) Reference FFT result group and target FFT result group The reference FFT result group is a single FFT result group within the complex FFT result group stored in the distance measuring unit 221 that serves as the starting point for differential calculations. The target FFT result group is one or more FFT result groups within the complex FFT result group stored in the distance measuring unit 221 that serve as the targets for differential calculations with the reference FFT result group. The one or more target FFT groups are temporally located before or after (usually before) the reference FFT group.

[0043] In this embodiment, one FFT result group is a current FFT result group (described later), and one or more target FFT result groups are one or more previous FFT result groups (described later).

[0044] By obtaining the difference between the baseline FFT result group and at least one target FFT result group, the reflection component from the stationary object 302 in the reflection wave Re can be removed (the receiving intensity of the reflection wave Re, that is, the reflection intensity is below the critical value), leaving only the reflection component from the human body 301 (the moving object) (detecting at least one group, preferably two or more groups, of reflection intensities amp exceeding the critical value, and a combination of frequencies f corresponding to the reflection intensities).

[0045] Furthermore, the "at least one" previous FFT result group, i.e., the number of previous FFT result groups from which differences are obtained with the current FFT result group, is preferably two or more (one-to-many inter-frame differences; in the illustrated example, one-to-five inter-frame differences) to improve resolution, as shown in Figures 10 and 12, but may also be just one. In other words, even when obtaining one-to-one differences (one-to-one inter-frame differences) as shown in Figures 8 and 9A-9C, attitude estimation is possible, and resolution can be improved by increasing the number of receiving antennas (hereinafter referred to as the "number of antennas").

[0046] (1-2-1b) Current FFT result group and previous FFT result group The current FFT result group is the most recent FFT result group among the stored complex FFT result groups. The previous FFT result group is the FFT result group that precedes the current FFT result group among the stored complex FFT result groups. By setting the current FFT result group as the reference FFT result group and the previous FFT result group as the target FFT result group, real-time posture estimation can be achieved.

[0047] (1-2-1c) Distance Measurement Processing Distance measurement processing refers to the process of measuring the distance to an object, such as a human body 301, using the radio wave sensor 1. In this embodiment, the distance measurement processing measures the distance from each of the three or more receiving antennas (hereinafter, "each antenna") constituting the radio wave sensor 1 to the human body 301, thereby obtaining three or more distance measurement results (distance measurement result groups; described later) corresponding to the three or more receiving antennas.

[0048] In the ranging process, for example, based on the various parameters shown in Figure 3, namely, the above-mentioned difference (frequency difference Δf) between the transmitted wave Tr and the reflected wave Re, the slope S of the linear change in the frequency f of the transmitted wave Tr, and the speed of light c, the distance d from each antenna to the human body 301 is calculated using the following formula 1. d = (c / 2S) × Δf (Formula 1)

[0049] (1-2-1d) Distance measurement result group The so-called ranging result group is information consisting of three or more ranging results corresponding to three or more receiving antennas obtained by the above formula 1 for one transmission and reception operation.

[0050] The three or more ranging results corresponding to three or more receiving antennas are, for example, the first, second, and third ranging results corresponding to the first, third, and fourth receiving antennas. The first ranging result corresponds to approximately half the propagation distance of the radio wave transmitted from the transmitting antenna, reflected by the human body 301, and reaching the first receiving antenna (i.e., the distance from the human body 301 to the first receiving antenna). The second ranging result corresponds to approximately half the propagation distance of the radio wave transmitted from the transmitting antenna, reflected by the human body 301, and reaching the second receiving antenna (i.e., the distance from the human body 301 to the second receiving antenna). The third ranging result corresponds to approximately half the propagation distance of the radio wave transmitted from the transmitting antenna, reflected by the human body 301, and reaching the third receiving antenna (i.e., the distance from the human body 301 to the third receiving antenna).

[0051] Furthermore, “at least one” distance measurement result group, that is, the number of the obtained distance measurement result groups is preferably two or more, but may also be one.

[0052] (1-2-2) Point configuration: Multi-point configuration Each time the distance measurement unit 221 performs distance measurement, the point placement unit 222 performs coordinate calculation processing and places at least one point corresponding to the current FFT result group in the three-dimensional space 500, as shown in FIG13A . However, the point placement destination may also be a two-dimensional space (plane) or a one-dimensional space (line).

[0053] Furthermore, “at least one” point, that is, the number of points configured at one time corresponding to the current FFT result group, is preferably two or more (multi-point configuration), but may also be one (single-point configuration).

[0054] (1-2-2a) Three-dimensional space The three-dimensional space 500 is a virtual space corresponding to the real space (e.g., room 400; see FIG2 ) in which the radio wave sensor 1 and the group of objects (301, 302) exist. The configuration of the three-dimensional space 500 is a virtual operation, and may simply be an operation of storing three-dimensional coordinates.

[0055] (1-2-2b) Coordinate Calculation and Point Group The so-called coordinate calculation processing is a process of calculating the three-dimensional coordinates of the human body 301 based on at least one distance measurement result group obtained.

[0056] (1-2-3) Posture Estimation Unit The posture estimation unit 223 estimates the posture of the human body 301 (in this embodiment, the posture is any of standing, sitting, and lying down) based on the point group 501 shown in Figures 13A to 13C. The point group 501 is a collection of one or more points arranged in the three-dimensional space 500 by the point arrangement unit 222.

[0057] Thus, according to this embodiment, the posture of the human body 301 can be estimated using the radio wave sensor 1 using the FMCW method.

[0058] (2) Details of the distance measurement unit and point configuration unit (2-1) One-to-many difference The distance measuring unit 221 preferably stores the FFT result group output by the radio wave sensor 1 over a period of at least twice the predetermined period. The period of at least twice the predetermined period may be, for example, a period of at least twice one frame (2×T).

[0059] Furthermore, the ranging unit 221 calculates the difference between the current FFT result group and each of the two or more previous FFT result groups among the three or more FFT result groups currently stored, and performs ranging processing based on each of the two or more calculated differences, thereby obtaining two or more ranging result groups.

[0060] The point arrangement unit 222 performs coordinate calculation processing based on each of the two or more distance measurement result groups obtained by the distance measurement unit 221, thereby arranging two or more points corresponding to the current FFT result group in the three-dimensional space 500.

[0061] In this way, by calculating the difference between the current FFT result group and each of two or more previous FFT result groups, the estimation accuracy of the posture of the human body 301 can be improved without increasing the number of antennas of the radio wave sensor 1.

[0062] (2-2) One-to-many difference detection for various human body movements The distance measuring unit 221 preferably stores the FFT result group output by the radio wave sensor 1 over a period that is at least three times the predetermined period. Furthermore, the distance measuring unit 221 calculates the difference between the current FFT result group and each of the three or more previous FFT result groups, among the four or more currently stored FFT result groups, and selects two or more differences from the three or more calculated differences that correspond to various movements of the human body 301 (e.g., body movement, respiratory fluctuations, hand and foot movements, etc.). The distance measuring unit 221 performs the distance measurement process based on each of the two or more selected differences, thereby obtaining two or more distance measurement result groups.

[0063] The point arrangement unit 222 performs the coordinate calculation process based on each of the two or more distance measurement result groups obtained by the distance measurement unit 221, thereby arranging two or more points corresponding to the current FFT result group in the three-dimensional space 500.

[0064] In this way, by calculating the difference between the current FFT result and each of three or more previous FFT results and selecting two or more differences corresponding to various movements of the human body 301, the accuracy of estimating the posture of the human body 301 can be improved without increasing the number of antennas and suppressing the number of differences used for ranging processing.

[0065] (2-3) Frames and inter-frame differences A frame is a unit of time used to repeatedly perform motion detection. The aforementioned sending and receiving operations are performed N times (N is an integer greater than or equal to 2) within a predetermined time frame. In this embodiment, the predetermined time frame is 200ms, and N = 10.

[0066] The memory stores N FFT result groups for each frame, covering multiple frames. Specifically, the radio wave sensor 1 performs ten transmission and reception operations per frame, for example. The memory of the control device 2 stores ten FFT result groups for each frame, covering multiple frames (e.g., six frames). The ranging unit 221 obtains inter-frame differences between the multiple frames stored in the memory.

[0067] The so-called inter-frame difference is the difference between the FFT result group belonging to one frame (for example, the reference frame Fr0 shown in FIG8 ) and the FFT result group belonging to one or more other frames (for example, the subsequent frames Fr1 . . . shown in FIG8 ).

[0068] (2-3-1) One-to-one inter-frame difference The inter-frame difference is, for example, a one-to-one inter-frame difference, which is the difference between the FFT result group of one frame (e.g., the reference frame Fr0) and the FFT result group of another frame (e.g., the subsequent frame Fr1).

[0069] (2-3-2) Calculating the representative value of each frame when performing inter-frame difference calculations The two FFT result groups corresponding to the two frames for which the difference is obtained (for example, the FFT result group belonging to the reference frame Fr0 and the FFT result group belonging to the subsequent frame Fr1) are respectively representative values of the N (for example, ten) FFT result groups in the frames to which they belong.

[0070] The representative value is, for example, the average value of N FFT result groups (specifically, information consisting of the average value of ten FFT results corresponding to the first receiving antenna, the average value of ten FFT results corresponding to the second receiving antenna, and the average value of ten FFT results corresponding to the third receiving antenna). In this case, the inter-frame difference is the difference between the average value of the ten FFT results belonging to the reference frame Fr0 and the average value of the ten FFT results belonging to the subsequent frame Fr1.

[0071] Alternatively, the representative value may be one FFT result group (e.g., the kth FFT result group, where k is an integer greater than 1 and less than N) among the N FFT result groups, determined according to a predetermined rule. In this case, the inter-frame difference is the difference between the kth (e.g., first) FFT result group among the ten FFT result groups belonging to the reference frame Fr0 and the kth (e.g., first) FFT result group among the ten FFT result groups belonging to the subsequent frame Fr1.

[0072] Furthermore, the above-mentioned inter-frame difference is not limited to one-to-one inter-frame difference, but is also applicable to one-to-many inter-frame difference.

[0073] The ranging unit 221 stores the FFT results output by the radio wave sensor 1 for a period of (K+1) frames (K is an integer greater than or equal to 2), for example, to calculate a one-to-many inter-frame difference (described later). In this embodiment, K=5, and the FFT results are stored (e.g., in the memory of the control device 2) for a period of (5+1) frames, or 6×200ms=1200ms.

[0074] (2-3-3) One-to-many inter-frame difference The distance measurement unit 221 calculates a one-to-many inter-frame difference using, for example, a group of FFT results covering a (K+1) frame period stored in memory. The one-to-many inter-frame difference is the difference (first time difference T, second time difference 2×T, etc.) between the FFT results of the reference frame Fr0 and a plurality of target frames (e.g., subsequent frames Fr1, Fr2, etc. as shown in FIG10 ).

[0075] In addition, in Figure 10, a plurality of subsequent frames Fr1, Fr2, ... following the reference frame Fr0 are illustrated as a plurality of object frames, but the plurality of object frames in this embodiment are, for example, a plurality of previous frames Fr-1, Fr-2, ... that are earlier than the reference frame Fr0 as shown in Figure 12.

[0076] As shown in FIG8 , the difference between the FFT result groups, for example, is the difference between the FFT result group corresponding to the first transmitted wave Tr1 of the reference frame Fr0 and the FFT result group corresponding to the first transmitted wave Tr1 of the target frame (the subsequent frame Fr1 in the example of FIG8 ) with a time difference T relative to the reference frame Fr0.

[0077] Alternatively, the difference of the FFT result group can be, for example, the difference between the FFT result group corresponding to the second transmission wave Tr2 of the reference frame Fr0 and the FFT result group corresponding to the second transmission wave Tr2 of the object frame (subsequent frame Fr1), or the difference between the FFT result group corresponding to the Nth transmission wave TrN of the reference frame Fr0 and the FFT result group corresponding to the Nth transmission wave Tr2 of the object frame (subsequent frame Fr1).

[0078] Alternatively, the difference of the FFT result group can also be the sum of the N differences as described above. In this embodiment, the difference of the FFT result group is the sum of the N differences as described above, that is, the sum of the differences of the N transmission waves Tr1 to TrN.

[0079] The one-to-many inter-frame differences in this embodiment are a collection of inter-frame differences between the current frame FFT result group and each of K or more (for example, five) previous frame FFT result groups, among the currently stored (N×(K+1)) or more (for example, 10×(5+1)=60) FFT result groups.

[0080] (2-3-4) Current frame and current frame FFT result group The current frame Fr0 is the frame Fr0 that includes the latest FFT result group. The current frame FFT result group is a collection of N FFT result groups belonging to the current frame Fr0.

[0081] (2-3-5) Previous frame and previous frame FFT result group The so-called previous frame (Fr-1, Fr-2, ..., Fr-K) is a frame that is earlier than the current frame Fr0. The previous frame FFT result group is a set of N FFT result groups belonging to the K previous frames (Fr-1, Fr-2, ..., Fr-K).

[0082] (2-3-6) Parameter K Furthermore, the larger the value of parameter K is, the easier it is to detect various movements of the human body 301 (for example, body movement, breathing movements, and movements of hands and feet).

[0083] (2-3-6a) Body movement, breathing movements, and hand and foot movements Body movements are changes in the entire body (trunk). Body movements are represented by MV1 in Figure 11 and, as further shown in Figure 12, are irregular movements with long durations and large fluctuations. Hand and foot movements are changes in the hands and feet. Hand and foot movements are represented by MV3 in Figure 11 and, as further shown in Figure 12, are irregular movements with short durations and large fluctuations. Respiratory micro-movements are changes in the body that occur with breathing. Respiratory micro-movements are represented by MV2 in Figure 11 and, as further shown in Figure 12, are periodic changes with small fluctuations and short durations.

[0084] (2-3-6b) Specific example of parameter K In this embodiment, as shown in Figure 12, K=5, and three (the second, third and fifth previous frames Fr-2, Fr-3 and Fr-5) that are suitable for the detection of body movement, hand and foot movement and respiratory micro-movement are selected to suppress the processing volume of differential calculation and improve the detection accuracy, but all five previous frames Fr-1~Fr-5 can also be used.

[0085] (2-3-7) Posture Estimation Based on One-to-Many Frame Difference In this manner, the distance measuring unit 221 obtains a group of K or more distance measurement results corresponding to K or more inter-frame differences constituting a pair of multi-frame differences calculated for the current frame Fr0.

[0086] The point arrangement unit 222 performs coordinate calculation processing based on each of the K or more distance measurement result groups obtained by the distance measurement unit 221 for the current frame Fr0, thereby arranging the K or more points corresponding to the current FFT result group in the three-dimensional space 500.

[0087] In this way, by calculating the one-to-many inter-frame difference of the current FFT result group relative to each of two or more previous FFT result groups, the estimation accuracy of the posture of the human body 301 can be further improved without increasing the number of antennas.

[0088] (2-3-8) Optimal value of parameter K The value of K is preferably an integer greater than or equal to 3. The distance measuring unit 221 selects two or more inter-frame differences corresponding to various movements of the human body 301 (e.g., body movement, respiratory micro-movement, hand and foot movement, etc.) from among the three or more inter-frame differences constituting the one-to-multi-frame difference calculated for the current frame Fr0, and obtains two or more groups of distance measurement results corresponding to the selected two or more inter-frame differences.

[0089] The point arrangement unit 222 performs coordinate calculation processing based on each of the two or more distance measurement result groups obtained by the distance measurement unit 221 for the current frame Fr0, thereby arranging two or more points corresponding to the current FFT result group in the three-dimensional space 500.

[0090] In this way, by calculating three or more inter-frame differences (one-to-many inter-frame differences) of the current FFT result group relative to each of three or more previous FFT result groups, and calculating two or more inter-frame differences corresponding to various movements of the human body 301 among the three or more inter-frame differences, the estimation accuracy of the posture of the human body 301 can be further improved without increasing the number of antennas and suppressing the number of inter-frame differences used for ranging processing.

[0091] The value of K is more preferably 5. The distance measuring unit 221 selects three inter-frame differences corresponding to the body movement, respiratory micro-movement, and limb movement of the human body 301 from the five inter-frame differences constituting the one-to-multi-frame difference calculated for the current frame Fr0, and obtains three groups of distance measurement results corresponding to the selected three inter-frame differences.

[0092] Specifically, as shown in FIG12 , the three inter-frame differences corresponding to the body movement, respiratory micro-movement, and limb movement of the human body 301 are, for example, the second, third, and fifth inter-frame differences ΔA2, ΔA3, and ΔA5 among the first to fifth inter-frame differences ΔA1 to ΔA5.

[0093] The point placement unit 222 performs coordinate calculation processing based on the three distance measurement result groups obtained by the distance measurement unit 221 for the current frame, thereby placing three points corresponding to the current FFT result group in the three-dimensional space 500.

[0094] In this way, by calculating the five inter-frame differences (one-to-many inter-frame differences) of the current FFT result group relative to each of the five previous FFT result groups, and calculating the three inter-frame differences corresponding to the body movement, respiratory micro-movement, and hand and foot movement of the human body 301 among the five inter-frame differences, the estimation accuracy of the posture of the human body 301 can be further improved without increasing the number of antennas and suppressing the number of inter-frame differences used for ranging processing.

[0095] Furthermore, the above is merely an example, and for example, the number of inter-frame differences and the time difference to be calculated may be appropriately changed to further improve the estimation accuracy.

[0096] (2-3-9) Details of the Posture Estimation Unit (2-3-9a) Distribution of Clusters and Cluster Groups For example, the posture estimating unit 223 clusters the point cloud 501 to obtain a set of one or more clusters (CL, CL1, CL2), that is, a cluster group (CL, CL1, CL2).

[0097] The clusters (CL, CL1, CL2) may be, for example, a first cluster CL1 corresponding to the upper torso (head, body, and wrists) of the human body 301, and a second cluster CL2 corresponding to the lower torso (feet), as shown in Figures 13A and 13B . Alternatively, the clusters (CL, CL1, CL2) may be a single cluster CL corresponding to the entire body (head, body, wrists, and feet), as shown in Figure 13C .

[0098] Furthermore, it is expected that by increasing the number of points constituting the point group 501 , the resolution of the clusters (CL, CL1, CL2) can be increased to a level that allows the human body 301 to be identified, and further, the contours (silhouettes) of the various parts constituting the human body 301 to be identified.

[0099] The posture estimating unit 223 estimates the posture of the human body 301 based on the distribution (hereinafter simply referred to as “distribution”) of the acquired clusters (CL, CL1, CL2) in the three-dimensional space 500 in this manner.

[0100] The distribution includes, for example, the number of clusters (CL, CL1, CL2) obtained, the direction of dispersion of a cluster (CL, CL1, CL2), the distance between multiple clusters, etc.

[0101] (2-3-9b) Posture estimation based on distribution conditions The posture estimating unit 223 estimates whether the posture of the human body 301 is a lying posture, a sitting posture, or a standing posture based on the distribution condition, which is a condition related to the distribution.

[0102] The distribution conditions include, for example, a lying position condition. The lying position condition (first lying position condition) constituting the distribution conditions may be, for example, "only a single cluster CL is obtained, and the height of the single cluster CL from the bottom surface is lower than a critical value" (see FIG. 13C ).

[0103] The distribution conditions may further include a sitting posture condition. The sitting posture condition (first sitting posture condition) constituting the distribution conditions may be, for example, "acquire first and second clusters CL1 and CL2, and the distance between the two clusters CL1 and CL2 is close to a level below a threshold" (see FIG13B ).

[0104] For example, when the distribution satisfies the above-mentioned lying position condition (first lying position condition), the posture estimation unit 223 judges that the posture of the human body 301 is a lying position; when the distribution satisfies the above-mentioned sitting position condition (first sitting position condition), the posture estimation unit 223 judges that the posture is a sitting position; when the distribution does not satisfy the above-mentioned lying position condition or the above-mentioned sitting position condition, the posture is judged to be a standing position.

[0105] In this way, based on the distribution of clusters (CL, CL1, CL2), the posture of the human body 301 can be estimated with good accuracy.

[0106] (2-3-9c) Posture estimation based on size ratio conditions Alternatively, the posture estimation unit 223 may estimate the posture of the human body 301 based on a size ratio condition related to the ratio (hereinafter referred to as "size ratio") of the length D, width W, and height H of the three-dimensional figure 502 surrounding the cluster (CL, CL1, CL2).

[0107] The three-dimensional figure 502 in this embodiment is, for example, a cuboid with a length D, a width W, and a height H as shown in FIG. 14A to FIG. 14C , and the size ratio is, for example, the ratio between the length D or the width W and the height H.

[0108] However, the three-dimensional figure 502 may be, for example, an elliptical cylinder (not shown). In an elliptical cylinder, the minor axis corresponds to the length D, and the major axis corresponds to the width W.

[0109] The size ratio condition includes a lying position condition. The lying position condition (second lying position condition) constituting the size ratio condition may be, for example, "the length D or width W of the three-dimensional figure 502 is greater than a predetermined ratio (e.g., three times) relative to its height H."

[0110] The size ratio condition further includes a posture condition. The posture condition constituting the size ratio condition may be, for example, "the height H of the three-dimensional figure 502 is greater than a predetermined ratio (e.g., three times) relative to its length D or width W."

[0111] For example, the posture estimation unit 223 can also judge that the posture of the human body 301 is a lying posture when the size ratio satisfies the above-mentioned lying posture condition (second lying posture condition), or judge that the posture is a standing posture when the size ratio satisfies the above-mentioned standing posture condition, or judge that the posture is a sitting posture when the size ratio does not satisfy the above-mentioned lying posture condition or the above-mentioned standing posture condition.

[0112] In this way, the posture of the human body 301 can be estimated with good accuracy based on the ratios related to the length D, width W and height H of the three-dimensional figure 502 surrounding the cluster (CL, CL1, CL2).

[0113] (2-3-9d) Posture estimation based on distribution size ratio conditions For example, the posture estimating unit 223 may estimate whether the posture of the human body 301 is lying, sitting, or standing based on a distribution size ratio condition which is a condition related to the distribution and size ratio.

[0114] The distribution size ratio condition includes a lying position condition. The lying position condition (third lying position condition) constituting the distribution size ratio condition may be, for example, "only one cluster CL is obtained, and the length D or width W of the three-dimensional figure 502 is greater than a predetermined ratio (e.g., three times) relative to its height H."

[0115] The distribution size ratio condition further includes a sitting posture condition, for example. The sitting posture condition (second sitting posture condition) constituting the distribution size ratio condition may be, for example, "acquire first and second clusters CL1 and CL2, and the distance between the two clusters CL1 and CL2 is close enough to be below a critical value."

[0116] For example, the posture estimation unit 223 can also judge that the posture of the human body 301 is a lying posture when the distribution and size ratio meet the above-mentioned lying posture condition (third lying posture condition), and can also judge that the posture is a sitting posture when the distribution and size ratio meet the above-mentioned sitting posture condition (second sitting posture condition), and can also judge that the posture is a standing posture when the distribution and size ratio does not meet the above-mentioned lying posture condition or the above-mentioned sitting posture condition.

[0117] (2-4) Equipment control function of equipment control system As shown in FIG. 1 , the device control system 100 further includes a motion detection unit 224 and a device control unit 225 .

[0118] (2-4-1) Action Detection Department The action detection unit 224 detects the action of the person corresponding to the human body 301 based on the estimation result of the human body 301 by the posture estimation unit 223.

[0119] The action detection unit 224 in this embodiment detects the non-operation action based on at least the change in the estimation result of the posture estimation unit 223.

[0120] In this embodiment, the change in the estimated result is a change between standing, sitting, and lying positions, specifically, for example, a change from standing to sitting, a change from sitting to standing, a change from standing or sitting to lying, a change from lying to standing or sitting, etc.

[0121] Furthermore, the action detection unit 224 may detect a non-operational action when a state in which no change in the estimation result of the posture estimation unit 223 is detected continues for a predetermined period of time or longer. For example, the action detection unit 224 may detect a non-operational action when no change in the estimation result of the posture estimation unit 223 is detected for a predetermined period of time or longer.

[0122] Specifically, the action detection unit 224 detects a non-operational action such as continued recumbent state (e.g., falling asleep) if, for example, the posture estimation unit 223 estimates that the user has changed from a standing or sitting position to a recumbent position, no change from the recumbent position to another posture has been detected for a predetermined period of time or longer. Furthermore, the action detection unit 224 detects a non-operational action such as continued sitting state (e.g., doing paperwork or relaxing) if, for example, the posture estimation unit 223 estimates that the user has changed from a standing or recumbent position to a sitting position, no change from the sitting position to another posture has been detected for a predetermined period of time or longer.

[0123] (2-4-1a) Non-operational actions and operational actions The so-called non-operational actions are actions different from operational actions.

[0124] An operation action is an action intended to operate a device 200. Examples of the device 200 include a lighting fixture 200a mounted on the ceiling of a room 400, a television (TV) 200b placed on a bed, or an electric blind 200c mounted on a window 402, as shown in FIG2 . Examples of operation actions include turning lighting fixture 200a on / off via a wall switch, turning TV 200b on / off and switching channels via a remote control, and opening and closing electric blinds 200c via a remote control.

[0125] Non-operational actions are, for example, everyday actions that are not intended to operate the device 200. Examples of everyday actions include entering the room 400 (entering the room from the entrance 401 of the room 400), lying down on the bed 302b (going to bed), getting up from the bed 302b (getting up), and leaving the room 400 (leaving the room from the entrance 401). Examples of everyday actions include sitting down at the desk 302a (sitting down) (doing paperwork or relaxing) and leaving the desk 302a (leaving the desk).

[0126] Furthermore, sitting can be distinguished into sitting for working at table 302a (desk work) and sitting for watching TV 200b and relaxing (relaxing). Desk work and relaxation can also be determined based on the time period (daytime or nighttime) when sitting is detected.

[0127] Furthermore, the action detection unit 224 may also take into account environmental information, historical information, and the like (described below) to detect non-operational actions. For example, waking up can be distinguished as waking up in the morning and waking up in the middle of the night. Waking up in the morning or waking up in the middle of the night can also be determined based on the time period of the change from lying to sitting or standing.

[0128] (2-4-2) Equipment Control Unit The device control unit 225 controls the device 200 based on at least the non-operation action detected by the action detection unit 224.

[0129] Furthermore, in addition to performing control in response to non-operational actions (automatic control), the device control unit 225 also performs control in response to manipulation of the device 200 (manual control). Manipulation refers to, for example, the manipulation of the operating elements (touch panel, operation buttons, etc.) of the control device 2, but may also refer to the manipulation of the operating elements (wall switches, remote controls, etc.) of the device 200, or to gestures used to operate the device 200. The manipulation of the operating elements of the device 200 is detected, for example, by monitoring control signals from the wall switches or other operating elements to the device 200. The gesture is detected, for example, by analyzing an image of the human body 301 captured by a camera (not shown), but may also be detected by distinguishing non-operational actions using the radio wave sensor 1.

[0130] In this way, the actions of the human body 301 (especially non-operational actions) are detected based on the estimated posture changes, and device control corresponding to the detection results is performed, thereby enabling device control without operating the device 200 (non-operational device control).

[0131] (2-4-3) Information Acquisition Department As shown in FIG1 , the device control system 100 further includes an information acquisition unit 226. The information acquisition unit 226 acquires various information, such as environmental information and a personal identification code (both to be described later).

[0132] For example, the information acquisition unit 226 acquires environmental information. Environmental information refers to information related to the environment in which the human body 301 exists. Examples of environmental information include time zones (daytime and nighttime, morning and nighttime), weather conditions (sunny and rainy), temperature, and humidity. Environmental information may also include the type of room 400 (bedroom and living room) or direction (southward and northward) in which the human body 301 exists.

[0133] Furthermore, if multiple people (two or more human bodies 301) may be present in the same environment, the information acquisition unit 226 may also acquire a person identification code. Specifically, for example, a combination of a person identification code and characteristic information representing the person's characteristics, i.e., a set of combined information (combined information group), is pre-stored in memory. The person identification code is, for example, an ID such as "1" or "2," but may also be a name, email address, or any other information that can identify a person. The characteristic information combined with the person identification code may be, for example, height information representing the person's height. However, the characteristic information may also be speed information representing the person's walking speed, or information representing characteristics that distinguish the person from others. The characteristic information itself may also be used as the person identification code.

[0134] When the posture estimation unit 223 detects the human body 301, the information acquisition unit 226 can also obtain feature information of the human body 301 from the posture estimation unit 223 and read the human identification code paired with the obtained feature information from the memory.

[0135] Furthermore, the automatic control information (see FIG. 15 ) described later can also be customized for each of the multiple personal identification codes.

[0136] The action detection unit 224 further detects non-operation actions based on the environmental information obtained by the information acquisition unit 226.

[0137] In this way, by utilizing environmental information in addition to posture changes, the detection accuracy of non-operational actions can be improved.

[0138] (2-4-3a) Automatic control and information for automatic control The device control unit 225 controls the device 200 based on non-operational actions using, for example, one or more (here, six) automatic control information (automatic control information groups) as shown in FIG15 . The automatic control information groups are, for example, pre-stored in the memory of the control device 2. In this embodiment, control based on the automatic control information groups is referred to as "automatic control."

[0139] As shown in Figure 15, the one or more pieces of automatic control information that make up the automatic control information group each contain information related to a state change, posture change, environment, non-operational action, control content, and control ID. Specifically, among the six pieces of automatic control information shown in Figure 15, the first piece of automatic control information contains: a state change of "absent to present", a posture change of "-", an environment of "-", a non-operational action of "entering the room", a control content of "normal lighting on (brightness 5)", and a control ID of "1". "-" indicates that the information is not included (the same applies hereinafter).

[0140] Similarly, the second automatic control information includes: state change "-", posture change "standing → sitting", environment "daytime", non-operational action "paperwork", control content "brighten lights (+2)", and control ID "2". The third automatic control information includes: state change "-", posture change "standing → sitting", environment "nighttime", non-operational action "relax", control content "turn on TV", and control ID "3". The fourth automatic control information includes: state change "-", posture change "standing or sitting → lying down", environment "nighttime", non-operational action "sleep", control content "lights off", and control ID "4". The fifth automatic control information includes: state change "-", posture change "lying down → standing or sitting", environment "late night", non-operational action "waking up in the middle of the night", control content "dim lights (brightness 2)", and control ID "5". The sixth automatic control information includes: state change "-", posture change "lying down → standing or sitting", environment "morning", non-operation action "getting up", control content "opening the blinds", and control ID "6".

[0141] (2-4-3b) Manual Control Furthermore, the device control unit 225 can also perform control (automatic control) using automatic control information based on an operation (hereinafter simply referred to as "operation") performed on the device 200. In this embodiment, control based on the operation is referred to as "manual control."

[0142] (2-4-4) Storage and Learning of Historical Records The device control system 100 further includes a history record storage unit 227 and a learning unit 228 .

[0143] (2-4-4a) History storage unit The history storage unit 227 stores control history information. Control history information refers to information related to the control history of the device 200 performed by the device control unit 225. While the control history information may be automatic control history information, it may also be manual control history information. Alternatively, the control history information may include both automatic control history information and manual control history information.

[0144] Automatic control history information refers to information related to the control history of device 200 corresponding to actions (e.g., non-operational actions, but operational actions are also possible). Automatic control history information associates the control content based on the action with the time of control, environmental information, and the user identification code.

[0145] Manual control history information refers to information related to the control history of the device 200 corresponding to the operation. Manual control history information associates the control content of the device 200 based on the operation with one or more of the time of operation, environmental information, and a personal identification code.

[0146] When the device control unit 225 controls the device 200, the history storage unit 227 stores the control history information shown in Figure 16 in the memory of the control device 2. In addition, the storage destination of the control history information can also be an external memory.

[0147] For example, at 9:02, when the device control unit 225 controls the lighting fixture 200a three times, three control history records are stored in the memory of the control device 2, as shown in FIG16 . Each of the three control history records includes information related to the history record ID, time, control ID, and operation.

[0148] Of the three control history records shown in Figure 16, the first control history record includes: history record ID "1", time "9:00", control ID "1", and operation "-". The second control history record includes: history record ID "2", time "9:01", control ID "2", and operation "-". The third control history record includes: history record ID "3", time "9:02", control ID "-", and operation "Operation to slightly dim the lighting (-1)".

[0149] Furthermore, the aforementioned automatic control history information refers to information related to operation with a "-" as the control ID. Of the three control history information pieces mentioned above, the first and second control history information pieces correspond to this information. Furthermore, the aforementioned manual control history information refers to information with a control ID of "-". Of the three control history information pieces mentioned above, the third control history information piece corresponds to this information.

[0150] (2-4-5) Learning Department The learning unit 228 updates the automatic control information when the time and control content shown in the automatic control history information and the time and operation shown in the manual control history information meet the pre-set update conditions.

[0151] (2-4-5a) Update conditions and update of automatic control information based on these update conditions An example of an update condition is "after a device is automatically controlled based on a non-operational action, an operation to cancel the control content of the automatic control is received within a preset time (e.g., two minutes)."

[0152] In the example of FIG16 , the second control history record information, i.e., the automatic control information, shows the time “9:01” and the control content “brighten the lighting (+2)” (i.e., the control content corresponding to the control ID “2” in FIG15 ), and the third control history record information, i.e., the manual control information, shows the time “9:03” and the operation “slightly dim the lighting (-1)”, satisfy the above-mentioned update conditions.

[0153] Specifically, after the automatic control "brighten the lighting (+2)" based on the non-operational action "paperwork" was performed on one device "lighting fixture 200a," an operation "slightly dim the lighting (-1)" was received within two minutes to cancel the automatic control. Therefore, learning unit 228 determines that the update condition is satisfied. Furthermore, learning unit 228 updates the second automatic control information corresponding to control ID "2" among the six automatic update information in FIG. 15 to, for example, "slightly brighten the lighting (+1)."

[0154] Alternatively, the update condition may be, for example, "receiving an operation different from the content of the automatic control a predetermined number of times or more (for example, "twice a day or more", "three times a week or more") during a predetermined period."

[0155] In this way, historical record information related to automatic control corresponding to non-operational actions (automatic control historical record information) and historical record information related to manual control corresponding to operations are stored, and based on these two types of historical record information, the information for automatic control is updated when the update conditions are met, thereby realizing the learning function.

[0156] (3) Specific example of equipment control system As shown in Figure 1, the device control system 100 of this example includes a radio wave sensor 1 and a control device 2. Each of the radio wave sensor 1 and the control device 2 includes a communication module and is connected to each other via wired or wireless communication. The control device 2 further includes a processor and memory. Programs and various information are stored in the memory. The processor operates based on the programs in the memory (further coordinating with the communication module), thereby achieving the operations of the control device 2, as described below. Furthermore, the radio wave sensor 1 also includes a processor and memory, and performs transmission and reception operations and communication with the control device 2 under program control.

[0157] As shown in FIG2 , the radio wave sensor 1 is installed on the ceiling of the room 400. As shown in FIG2 , the control device 2 is, for example, a compound remote control and is installed on the side wall of the room 400.

[0158] Based on the output of the radio wave sensor 1, the control device 2 estimates the posture (standing, sitting, or lying) of a person 301 in the room 400. Furthermore, based on changes in the estimated posture, the control device 2 detects non-operational actions by the person 301 and automatically controls the plurality of devices 200 (lighting fixture 200a, TV 200b, and electric blinds 200c) based on the detection results. Furthermore, the control device 2 includes operating elements such as a touch panel, allowing manual control of the plurality of devices 200 in response to the operation.

[0159] As shown in Figure 1, the control device 2 includes a receiving unit 21, a processing unit 22, and an output unit 23. The processing unit 22 includes a distance measurement unit 221, a point placement unit 222, a posture estimation unit 223, a motion detection unit 224, a device control unit 225, an information acquisition unit 226, a history storage unit 227, and a learning unit 228.

[0160] The receiving unit 21 receives various information. The so-called various information includes, for example, operation information of the device 200. The receiving unit 21 can also receive information other than operation information, such as personal information such as the height of the person 301.

[0161] The processing unit 22 performs various processes. Examples of these processes include those performed by the distance measurement unit 221, the point placement unit 222, the posture estimation unit 223, the movement detection unit 224, the device control unit 225, the information acquisition unit 226, the history storage unit 227, and the learning unit 228. Furthermore, the processing unit 22 also performs the various determinations described in the flowcharts.

[0162] Specifically, the processing unit 22 can also perform, for example, storage processing, acquisition processing, position determination processing, human body estimation processing, and placement processing. The processing unit 22 performs this series of processing each time the radio wave sensor 1 outputs position-determinable information, but can also perform this series of processing each time a frame of position-determinable information is output.

[0163] The so-called storage process is a process of storing the position-determinable information output by the radio wave sensor 1. The so-called acquisition process is a process of acquiring the difference between the stored multiple pieces of position-determinable information. The so-called position determination process is a process of determining the position of one or more objects (301, 302, 200c) constituting the object group (301, 302, 200c) based on the difference and acquiring position information indicating the determined position. The so-called human body detection process is a process of detecting the human body 301 using the received signal received by the radio wave sensor. The human body estimation process is a process of estimating whether the acquired position information corresponds to the human body 301 or a moving object other than the human body 301 (for example, the electric blind 200c) based on at least the change in the difference. In this embodiment, the change is generally a change in the difference value with the passage of time, and may also be a so-called "time change". However, the change may also be, for example, a change in value corresponding to a position in space (spatial change).

[0164] The placement process is a process of virtually placing points corresponding to the acquired position information in space (three-dimensional space 500) when it is estimated that the acquired position information corresponds to the human body 301. The placement process is performed by the point placement unit 222 constituting the processing unit 22.

[0165] In this embodiment, the storage process can specifically store three or more determinable position information. The acquisition process is to obtain two or more differences of mutually different time differences for the three or more determinable position information stored. The position determination process is to obtain two or more position information corresponding to the two or more differences obtained. The human body detection process is, for example, to estimate which of the two or more acquired position information each corresponds to, the human body 301 and a moving object (200c) other than the human body 301. The arrangement process is to virtually arrange the point corresponding to the position information estimated to correspond to the human body 301 among the two or more acquired position information in the space (500).

[0166] In addition, the configuration processing can also receive the estimated results of the human body detection processing, and display the points corresponding to the object estimated to be the human body 301 and the points corresponding to the object estimated to be a moving object (200c) other than the human body 301 in a manner that can be distinguished from each other (for example, displayed in different colors, different sizes, etc.) via the output unit 23.

[0167] Furthermore, the processing unit 22 in this embodiment further performs device control processing. The so-called device control processing is processing for controlling the device 200 based on at least a set of multiple points arranged in the space (500), that is, the point group 501. The human body detection processing is processing for estimating the posture of the human body 301 or changes in the posture based on the distribution of the point group 501.

[0168] The processing unit 22 may also perform action detection processing. Action detection processing is processing that detects the actions of a person corresponding to the human body 301 based on the estimated posture or posture changes. The device control processing is, for example, processing that controls the device 200 based on the detected actions.

[0169] The processing unit 22 may further perform information acquisition processing. The information acquisition processing is processing for acquiring environmental information related to the environment in which the human body 301 exists. The human body detection processing is further processing for detecting actions based on the environmental information acquired by the information acquisition processing.

[0170] The device control process is, for example, to control the device 200 based on actions using pre-stored automatic control information.

[0171] The processing unit 22 may also perform a history storage process. The history storage process stores history information related to the control history of the device 200. For example, the history information may be automatic control history information. Alternatively, the history information may be manual control history information. The history storage process preferably stores both automatic control history information and manual control history information, but may store only one of the two.

[0172] The processing unit 22 may also perform a learning process. Learning processing is the process of updating automatic control information when the time and operation indicated by the manual control history information meet pre-set update conditions. Alternatively, learning processing may be the process of updating automatic control information when the time and control content indicated by the automatic control history information, and the time and operation indicated by the manual control history information, meet pre-set update conditions.

[0173] Furthermore, the radio wave sensor 1 may also perform the conversion processing described in "(10) Second Modification of the Device Control System," and the processing unit 22 may also perform the storage processing, difference acquisition processing, and motion detection processing described in "(10) Second Modification of the Device Control System." In other words, the radio wave sensor 1 may also include the conversion unit 11 shown in FIG. 17 , and the processing unit 22 may also include the conversion unit 11, storage unit 12, difference acquisition unit 13, and motion detection unit 14 shown in FIG. 17 .

[0174] The output unit 23 outputs various information. For example, the various information is control information for the device 200 (e.g., remote control signal). The output here usually refers to sending to the device 200, but may also include displaying on a display.

[0175] (4) Operation example of equipment control system The control device 2 constituting the equipment control system 100 operates, for example, according to the flowcharts of FIG. 4 to FIG. 7 .

[0176] (4-1) Overall processing The process of FIG. 4 starts upon activation of the device control system 100 and ends upon stopping of the operation.

[0177] First, the processing unit 22 constituting the control device 2 determines whether an FFT result group is output from the radio wave sensor 1 (step S1). If it is determined that an FFT result group is not output, the process proceeds to step S13.

[0178] When it is determined in step S1 that an FFT result group is output, the distance measuring unit 221 constituting the processing unit 22 stores the FFT result group (step S2).

[0179] Next, the distance measuring unit 221 determines whether a group of FFT results for a period of (K+1) or more frames is stored (step S3). If it is determined that a group of FFT results for a period of (K+1) or more frames is not stored, the process returns to step S1.

[0180] If it is determined in step S3 that a group of FFT results of at least (K+1) frames is stored, the distance measuring unit 221 performs a one-to-many inter-frame difference acquisition process (step S4). The one-to-many inter-frame difference acquisition process is described with reference to the flowchart of FIG6 .

[0181] Next, the ranging unit 221 obtains L (or K) ranging results (ranging result groups) corresponding to the L (or K) differences (difference groups) (step S5).

[0182] Next, the point arrangement unit 222 calculates L (or K) three-dimensional coordinates (three-dimensional coordinate group) based on the distance measurement result group obtained in step S5 (step S6).

[0183] Next, the point arrangement unit 222 arranges L (or K) points (point group 501) based on the three-dimensional coordinate group obtained in step S6 in the three-dimensional space 500 (step S7).

[0184] Next, it is determined whether the judgment start condition is satisfied (step S8). The judgment start condition is, for example, the condition that a predetermined number or more points are arranged in the three-dimensional space 500. If it is determined that the judgment start condition is not satisfied, the process returns to step S1.

[0185] If it is determined in step S8 that the determination start condition is satisfied, the posture estimation unit 223 performs posture estimation processing (step S9). The posture estimation processing will be described with reference to the flowchart of FIG7.

[0186] Next, the action detection unit 224 uses the automatic control information (see FIG15 ) to detect non-operational actions based on changes in the estimation results of step S9 (e.g., changes in state, posture, and environment) (step S10). For example, based on the state change from "absent to present," the non-operational action "entering the room" is detected. Furthermore, based on the posture change from "standing to sitting," the environment (current time zone) is determined to be daytime or nighttime. The non-operational action "paperwork" is detected during the daytime, and the non-operational action "relaxing" is detected during the nighttime.

[0187] Next, the device control unit 225 performs device control (automatic control) based on the results of the non-operational action detection in step S10 (step S11). For example, upon detection of the non-operational action "entering the room," automatic control is performed to illuminate the lighting fixture 200a at normal brightness (brightness 5) (control ID "1"). Furthermore, upon detection of the non-operational action "paperwork," automatic control is performed to brighten the lighting fixture 200a (brightness +2) (control ID "2"). Furthermore, upon detection of the non-operational action "relaxing," automatic control is performed to turn on the TV 200b (control ID "3").

[0188] Next, the history storage unit 227 stores the automatic control history information related to the automatic control performed in step S11 (step S12). Thereafter, the process returns to step S1.

[0189] If it is determined in step S1 that the FFT result group has not been output, the processing unit 22 determines whether the receiving unit 21 has received an operation for the device 200 (step S13). If it is determined that the receiving unit 21 has not received an operation for the device 200, the processing returns to step S1.

[0190] If, in step S13, the reception unit 21 determines that an operation has been received for the device 200, the device control unit 225 performs device control (manual control) corresponding to the operation (step S14). For example, if an operation is performed to slightly dim the lighting fixture 200a (brightness -1), manual control is performed to slightly dim the lighting fixture 200a.

[0191] Next, the history storage unit 227 stores the manual control history information related to the manual control performed in step S14 (step S15).

[0192] Next, the learning unit 228 determines whether the update condition is satisfied (step S16). If it is determined that the update condition is not satisfied, the process returns to step S1.

[0193] If the update conditions are determined to be met, the history storage unit 227 updates the automatic control information (step S17). For example, as shown in Figure 16, since manual control of lighting fixture 200a was performed one minute after the start of automatic control of control ID "2," the update conditions are determined to be met, and the control content of the automatic control of control ID "2" is updated to "Brighten lighting (+1)." The process then returns to step S1.

[0194] Furthermore, in the process of Figure 4 , the radio wave sensor 1 may output three or more IF signals (received signals) corresponding to three or more receiving antennas instead of a group of FFT results (which can determine position information), and the processing unit 22 may also obtain FFT conversion results based on the three or more IF signals output. In this case, in step S1 above, the processing unit 22 only needs to determine whether FFT conversion has been performed on each of the three or more IF signals and obtain the FFT conversion results.

[0195] (4-2) One-to-many inter-frame difference acquisition processing The one-to-many inter-frame difference acquisition process in step S4 is performed, for example, according to the flowchart of FIG. 6 .

[0196] The processing unit 22 sets the initial value "1" to the variable i (step S41).

[0197] Next, the processing unit 22 determines whether the variable i is greater than K (step S42). If it is determined that the variable i is greater than K, the process proceeds to step S45.

[0198] When it is determined in step S42 that the variable i is not greater than K (i.e., less than K), the ranging unit 221 obtains the difference between the current FFT result group and the i-th previous FFT result group before the i-th time difference (=i×T seconds) from the current FFT result group, and sets the obtained difference as the variable "i-th difference" (step S43).

[0199] Next, the processing unit 22 increases the variable i (step S44). Thereafter, the processing returns to step S42.

[0200] If the variable i is determined to be greater than K in step S42, the distance measuring unit 221 selects L differences (e.g., three differences ΔA2, ΔA3, and ΔA5; see FIG12 ) corresponding to various movements of the human body 301 from the first through the Kth K differences (e.g., five differences ΔA1 through ΔA5; only a portion is shown in FIG12 ) (step S45 ). The process then returns to the upper flow chart (see FIG4 ).

[0201] (4-3) Posture Estimation Processing The posture estimation process of step S9 is performed, for example, according to the flowchart of FIG. 7 .

[0202] The posture estimating unit 223 clusters the point cloud 501 arranged in the three-dimensional space 500 and obtains clusters (CL, CL1, CL2: see FIG. 13A to FIG. 13C) (step S91).

[0203] Next, the posture estimating unit 223 arranges a three-dimensional figure (rectangular parallelepiped) 502 (see FIG. 14A to FIG. 14C ) surrounding the clusters (CL, CL1, CL2) acquired in step S91 in the three-dimensional space 500 (step S92 ).

[0204] Next, the posture estimating unit 223 obtains the ratio (dimension ratio) of the length D, width W, and height K of the three-dimensional figure 502 arranged in step S92 (step S93).

[0205] Next, the posture estimating unit 223 determines whether the distribution (cluster distribution) of the clusters (CL, CL1, CL2) obtained in step S91 and the size ratio obtained in step S93 satisfy the lying posture condition (step S94).

[0206] If the cluster distribution and size ratio are determined to satisfy the lying posture condition, the posture estimation unit 223 sets the value "lying posture" to the variable "posture" (step S95). Thereafter, the process returns to the upper flowchart (see FIG4 ).

[0207] When it is determined that the cluster distribution and the size ratio do not satisfy the lying posture condition, the posture estimating unit 223 further determines whether the cluster distribution and the size ratio satisfy the sitting posture condition (step S96).

[0208] If the cluster distribution and size ratio are determined to satisfy the sitting posture condition, the posture estimation unit 223 sets the value "sitting posture" to the variable "posture" (step S97). Thereafter, the process returns to the upper flowchart (see FIG4 ).

[0209] If the cluster distribution and size ratio do not satisfy the sitting posture condition, the posture estimation unit 223 sets the value "standing posture" to the variable "posture" (step S98). Thereafter, the process returns to the upper flowchart (see FIG4 ).

[0210] (5) Human body posture estimation method, equipment control method and program The human posture estimation function of the present invention can also be implemented using a human posture estimation method or program. The human posture estimation method, among the various steps described above, includes at least steps S2-S5 (distance measurement steps), steps S6 and S7 (configuration steps), and step S9 (posture estimation step). The program is configured to cause one or more processors to execute the human posture estimation method.

[0211] The device control function of the present invention can also be implemented via a device control method or program. In addition to the distance measurement step, the configuration step, and the posture estimation step, the device control method further includes step S10 (motion detection step) and step S11 (device control step). The program is used to cause one or more processors to execute the device control method.

[0212] (6) Variations of non-operational actions In addition to going to bed and waking up, non-operational actions may also include, for example, falling asleep after bedtime and being half awake before waking up. Falling asleep and being half awake can be detected based on changes in body movement (stop and start). The cessation of body movement can be, for example, the state where no body movement exceeding a threshold value is detected for a predetermined period of time. The start of body movement can be, for example, the state where body movement exceeding a threshold value is detected for a predetermined period of time.

[0213] (7) Variations of the configuration of each part In this embodiment, the control device 2 includes a distance measuring unit 221, a point placement unit 222, a posture estimation unit 223, a motion detection unit 224, a device control unit 225, an information acquisition unit 226, a history storage unit 227, and a learning unit 228. However, all of these elements may be included in the radio wave sensor 1. Alternatively, the radio wave sensor 1 may include only the distance measuring unit 221, while the control device 2 may include the other elements. Alternatively, the control device 2 may be built into the radio wave sensor 1.

[0214] (8) First Variation of the Equipment Control System: Human Body Posture Estimation System This variation omits the elements related to device control functions in device control system 100 shown in FIG1 , namely, motion detection unit 224, device control unit 225, information acquisition unit 226, history storage unit 227, and learning unit 228. This device control system 100 includes at least a distance measurement unit 221, a point placement unit 222, and a posture estimation unit 223, and may also be referred to as "human posture estimation system 100."

[0215] Furthermore, the human body posture estimation system 100 may also include, in addition to the distance measuring unit 221, the point arrangement unit 222, and the posture estimation unit 223, a motion detection unit 224, a device control unit 225, an information acquisition unit 226, a history record storage unit 227, and a learning unit 228.

[0216] (9) Variation of Difference Acquisition: Intra-frame Difference The radio wave sensor 1 may also set a predetermined time as one frame, perform transmission and reception operations in a cycle of N times (N is an integer greater than or equal to 4) per frame, and output N pieces of determinable position information per frame. The processing unit 22 stores at least the N pieces of determinable position information corresponding to one frame. Furthermore, the processing unit 22 selects three or more pieces of determinable position information representing one frame (representative determinable position information) from the stored N or more pieces of determinable position information corresponding to one or more frames, and obtains two or more differences for the selected three or more pieces of representative determinable position information. Furthermore, the processing unit 22 may also obtain two or more pieces of position information based on the two or more differences obtained for the three or more pieces of representative determinable position information representing one frame.

[0217] Furthermore, in this example, the processing unit 22 further stores N pieces of determinable position information corresponding to the frame below the frame. When two or more pieces of position information obtained for three or more representative determinable position information correspond to a moving object (200c) other than the human body 301, the processing unit 22 sets the frame and the next frame as a new frame. Furthermore, the processing unit 22 selects three or more representative determinable position information representing the new frame from the 2×N pieces of determinable position information corresponding to the new frame, and obtains two or more differences for the selected three or more representative determinable position information. The processing unit 22 can also obtain two or more pieces of position information based on the two or more differences obtained for the three or more representative determinable position information representing the new frame.

[0218] (10) Second variant of the device control system: a device control system including a motion detection system In addition, in this modification, descriptions related to matters already mentioned in the implementation mode may sometimes be omitted or simplified.

[0219] As shown in Figure 17, the device control system 100 in this variation includes a motion detection system 100A and a device control device 2B. The motion detection system 100A includes a radio wave sensor 1 and a motion detection device 2A. The motion detection device 2A includes a conversion unit 11, a storage unit 12, a difference acquisition unit 13, a motion detection unit 14, and a point placement unit 222. The motion detection unit 14 includes a distance measurement unit 221, a posture estimation unit 223, and a motion detection unit 224. The device control device 2B includes a device control unit 225, an information acquisition unit 226, a history storage unit 227, and a learning unit 228.

[0220] Furthermore, in the example of FIG17 , the motion detection device 2A includes a converter 11. However, the converter 11 may be built into the radio wave sensor 1 or interposed between the radio wave sensor 1 and the motion detection device 2A. In the aforementioned embodiment, the converter 11 is built into the radio wave sensor 1, although not shown.

[0221] In the example of FIG17 , the motion detection device 2A and the device control device 2B are provided separately, but they may also be integrated. Furthermore, although omitted in FIG17 , the motion detection system 100A typically further includes a receiving unit 21, a processing unit 22 (e.g., a first processing unit on the motion detection device 2A side and a second processing unit on the device control device 2B side), and an output unit 23 (see FIG1 ).

[0222] (10-1) Radio wave sensor The radio wave sensor 1 constituting the motion detection system 100A performs a sending and receiving operation of transmitting a radio wave modulated in a predetermined manner, namely a transmission wave Tr, toward a real space (for example, the interior of the room 400 shown in FIG2 : hereinafter referred to as “real space (400)”), receiving a reflected wave Re from the real space (400), and outputting a reception signal based on the transmission wave Tr and the reflected wave Re.

[0223] The sending and receiving actions are the same as in the implementation mode, and are repeated at a predetermined cycle (ie, regularly), but can also be performed irregularly.

[0224] Furthermore, while the radio wave sensor 1 in the embodiment outputs position-determining information such as a group of FFT results, the radio wave sensor 1 in this modified example outputs a received signal such as an IF signal. The conversion of the received signal into position-determining information can be performed externally to the radio wave sensor 1 (e.g., by the conversion unit 11 constituting the motion detection device 2A).

[0225] (10-1-1) Receiving Signals For example, in the case of radio waves modulated using the FMCW method, the received signal represents the frequency difference Δf between the reflected wave Re and the transmitted wave Tr at the same time (IF signal: see Figure 3). However, the received signal can also be a signal representing the time difference Δt between the transmitted wave Tr and the reflected wave Re, calculated from the frequency difference Δf. Furthermore, in the case of radio waves modulated using pulse modulation, the received signal can also be a signal representing the time difference Δt between the transmitted wave Tr and the reflected wave Re.

[0226] (10-1-2) Modulation method The predetermined method, that is, the modulation method of the radio wave, is the FMCW method in this example, but it may also be a pulse modulation method, for example. Any method is acceptable as long as it can obtain identifiable position information as a result.

[0227] (10-2) Moving body detection device The conversion unit 11 of the motion detection device 2A performs conversion processing. Conversion processing converts the received signal output by the radio wave sensor 1 into information that can determine position. For example, the conversion process involves Fourier transforming the IF signal, which is one type of received signal. The Fourier transform is, for example, an FFT, but can also be an STFT (short-time Fourier transform).

[0228] The so-called position information that can be determined is information that can determine the position of each of one or more objects (human body 301, electric blinds 200c, stationary object 302) existing in real space (400), such as the FFT result group described in the implementation example, but is not limited to this.

[0229] The storage unit 12 performs storage processing. In this example, the storage processing refers to the storage of the position-determining information converted by the conversion processing. The position-determining information is, for example, written to a memory included in the motion detection device 2A and stored in the memory for a predetermined period (e.g., one frame period, three frames period, etc.).

[0230] The difference acquisition unit 13 performs a difference acquisition process. The difference acquisition process is a process of acquiring the difference between the plurality of positional information stored in the storage process.

[0231] The motion detection unit 14 performs motion detection processing. Motion detection processing is processing for detecting a moving object using the received signal received by the radio wave sensor 1. For example, motion detection processing is processing for estimating motion based on the difference obtained using the received signal. Specifically, motion detection processing is processing for estimating one or more objects as moving objects or non-moving objects (stationary objects 302) based on the differences obtained in the difference acquisition process, determining the positions of the objects estimated to be moving objects (human body 301, electric blinds 200c), and obtaining motion position information indicating the determined positions.

[0232] The point placement unit 222 performs point placement processing. The point placement processing is a process of virtually placing points corresponding to the moving body position information obtained by the moving body detection processing in a virtual space corresponding to the real space (400) (for example, the three-dimensional space 500 in the embodiment). The point placement processing may also be, for example, a multi-point placement process described as an operation of the point placement unit 222 in the embodiment.

[0233] Furthermore, the difference acquisition process in this example acquires two or more differences of mutually different time differences with respect to the three or more identifiable position information stored by the storage process. The motion detection process acquires two or more pieces of motion position information corresponding to the two or more differences acquired by the difference acquisition process. The point placement process virtually places two or more points corresponding to the two or more pieces of motion position information acquired by the motion detection process in a virtual space (500).

[0234] Here, the so-called "virtually configuring two or more points in the virtual space (500)" includes "a situation where all points are configured" and "a situation where only a part of the points are configured" (in other words, a situation where at least one point is configured) among the two or more points corresponding to the two or more moving body position information obtained by the moving body detection processing, and may also include "a situation where no point is configured".

[0235] However, the case where "no point is allocated" can be excluded. In this case, the point allocation process is to virtually allocate "all or part" of two or more points corresponding to two or more pieces of moving body position information obtained by the moving body detection process in the virtual space (500). That is, even if all two or more points are determined to be moving objects other than the human body 301, for example, a representative point of the two or more points (any point, or a point corresponding to the average value of the two or more pieces of moving body position information corresponding to the two or more points) can be allocated.

[0236] Furthermore, when two or more points corresponding to two or more pieces of motion position information obtained by the motion detection processing are always configured, there is no need to judge whether the motion body is the human body 301 or a moving object other than the human body 301 (the motion position information is information corresponding to the human body 301, or information corresponding to a moving object other than the human body 301) as performed in this example.

[0237] In this example, the above-described determination is performed for each of the two or more acquired moving body position information. If the information is determined to correspond to human body 301, the point corresponding to the moving body position information is allocated. Therefore, depending on the individual determination results, all of the two or more points corresponding to the two or more moving body position information may be allocated, only some of the points may be allocated (at least one point may be allocated), or no points may be allocated.

[0238] Furthermore, for example, if the target moving body position information is not considered sufficient to yield a valid determination result, the result of being excluded from the determination may be that only some points are located, or no points are located at all. In this example, for example, based on information such as the reception intensity of the reflected wave Re corresponding to the moving body position information, it is also possible to determine whether each of two or more moving body position information obtained through the moving body detection process is a target for the determination, and perform the determination only on the moving body position information determined to be a target.

[0239] (10-2-1) Point configuration details For example, the point placement process may place only the points corresponding to the human body 301 (i.e., a portion of the two or more points). In other words, points corresponding to moving objects other than the human body 301 may not be placed. However, points corresponding to moving objects other than the human body 301 (i.e., all of the two or more points) may also be placed.

[0240] Furthermore, the point placement process may also place points corresponding to the human body 301 and living bodies other than the human body 301 (such as pets; not shown). In other words, points corresponding to moving bodies other than living bodies may be excluded from the placement targets.

[0241] In this way, it is expected that by obtaining two or more differences of mutually different time differences for three or more pieces of positional information that can be determined, the number of points arranged in the virtual space (500) can be increased (multi-pointing of the point group 501). As a result, it is possible to more accurately determine whether the point group 501 corresponds to a moving object such as a human body 301 or to a stationary object 302, and to improve the accuracy of moving object detection using reflected waves of transmitted radio waves. For example, the accuracy of estimation related to a moving object using the radio wave sensor 1 can be improved.

[0242] Furthermore, as a result of improving the estimation accuracy related to the moving body, the feasibility of controlling the device 200 using the radio wave sensor 1 can be improved (for example, the control accuracy of the device control device 2B described later).

[0243] (10-2-2) Estimation related to the position information of a moving object: The moving object is a human body or a moving object other than a human body. The moving object includes any of the human body 301 and any other moving object other than the human body 301. The moving object other than the human body 301 includes, for example, an electrically powered shutter 200c that opens and closes, or a robot vacuum cleaner that cleans while moving. Any moving object is acceptable. The moving object other than the human body 301 also includes the body of a living organism other than a human (e.g., a pet).

[0244] The motion detection process further estimates which of the two or more motion position information obtained corresponds to the human body 301 and the moving object (200c) other than the human body 301 based on at least the change in the difference. The point configuration process virtually configures in the virtual space (500) the points corresponding to the motion position information estimated by the motion detection process as corresponding to the human body 301 among the two or more motion position information obtained by the motion detection process. That is, the points corresponding to the motion position information estimated by the human body detection process as corresponding to the moving object other than the human body 301 are excluded from the configuration objects.

[0245] In this way, in the motion detection system 100A, when detecting the human body 301, for example, performing inferences related to the human body 301, various movements of the human body 301 (for example, body movement, micro-movements of breathing, movements of hands and feet, etc.: refer to Figure 12) can be detected by obtaining two or more differences of time differences that are different from each other for three or more identifiable position information, and the detection accuracy of the human body 301 can be improved, for example, the inference accuracy related to the human body 301 can be improved.

[0246] (10-2-3) Variations of the estimation of moving body position information: A moving object other than a human body is a living organism, or a moving object other than a living organism The moving object other than the human body 301 may be, for example, a living body other than the human body 301 (for example, the body of a living thing such as a pet kept by a person: not shown) or a moving object other than a living body (such as the electric blinds 200c).

[0247] In this case, the point arrangement process arranges points corresponding to the living body in the virtual space (500). That is, points corresponding to moving bodies other than the living body are excluded from the arrangement objects.

[0248] The device control processing in this example (described later) can also be performed when a point group 501 corresponding to a biological body is detected (for example, when a person or a pet enters a room), such as turning on lighting fixtures or turning on the air conditioner.

[0249] Furthermore, the motion detection processing can also distinguish the multiple points in the virtual space (500) into a point group 501 corresponding to the human body 301 and a point group 501 corresponding to a biological body other than the human body based on the distribution of the multiple points in the virtual space (500) (the shape of the cluster CL, etc.).

[0250] Furthermore, device control processing can also be performed differently depending on whether point cluster 501 corresponds to human body 301 or to a living organism other than human body 301 (e.g., a pet). Specifically, the device control processing can be configured to turn on a television when a point cluster 501 corresponding to human body 301 is detected, while not turning on the television even if a point cluster 501 corresponding to a living organism other than human body 301 (e.g., a pet) is detected. By performing different device control depending on whether the target object is human body 301 or a living organism other than human body 301, a more comfortable environment for people and pets can be achieved, achieving both comfort and energy conservation.

[0251] (10-2-4) Estimation taking into account the intensity of the reflected wave In addition to being based on the change in the differential, the motion detection processing can also be based on at least one of the reception intensity of the reflected wave Re corresponding to the motion position information and the change in the reception intensity, to infer whether each of the two or more motion position information obtained corresponds to the human body 301 or a moving object (200c) other than the human body 301.

[0252] In this way, by utilizing at least one of the reception intensity of the reflected wave Re and the change in the reception intensity in addition to the change in the differential, the estimation accuracy related to whether the moving body (301, 200c) is the human body 301 or a moving object (200c) other than the human body 301 is improved.

[0253] (10-2-5) Prediction Related to Point Groups The motion detection process is further based on at least one of the distribution of the point group 501 and the change in the distribution, and estimates that the point group 501 as a whole corresponds to the human body 301 or a moving object (200c) other than the human body 301. In this way, by also utilizing at least one of the distribution of the point group 501 arranged in the virtual space (500) and the change in the distribution, the accuracy of the estimation related to whether the moving body (301, 200c) is the human body 301 or the moving object (200c) other than the human body 301 can be improved.

[0254] (10-2-6) Inter-frame difference The radio wave sensor 1 sets a predetermined time as a frame and performs transmission and reception operations in a cycle of N times (N is an integer greater than or equal to 1) per frame. Thus, the radio wave sensor 1 outputs N pieces of determinable position information per frame. The storage process stores 3×N or more pieces of determinable position information corresponding to three or more frames. The difference acquisition process selects one representative piece of determinable position information representing a representative frame from each of the three or more frames, from the 3×N or more pieces of determinable position information corresponding to the three or more frames stored in the storage process. Furthermore, the difference acquisition process obtains two or more differences for the selected three or more pieces of representative determinable position information. The motion detection process obtains two or more pieces of motion object position information based on the two or more differences obtained for the three or more pieces of representative determinable position information representing the three or more frames, respectively.

[0255] In this way, three or more representative pieces of position information can be selected from three or more frames, and two or more differences can be obtained for the three or more selected representative pieces of position information.

[0256] (10-2-7) Intra-frame Difference The radio wave sensor 1 performs transmission and reception operations N times (N is an integer greater than or equal to 3) per frame, with a predetermined time period being defined as a frame. The radio wave sensor 1 outputs N or more received signals per frame. A conversion process converts each of the N or more received signals output in a frame into information that can determine a position. A storage process stores at least the N pieces of information that can determine a position corresponding to a frame.

[0257] The difference acquisition process selects three or more representative pieces of identifiable position information representing one frame from among the N or more pieces of identifiable position information corresponding to one or more frames stored in the storage process. Furthermore, the difference acquisition process obtains two or more differences for the selected three or more pieces of representative identifiable position information. The moving object detection process obtains two or more pieces of moving object position information based on the two or more differences obtained for the three or more pieces of representative identifiable position information representing one frame.

[0258] In this way, three or more representative pieces of position information can be selected in one frame, and two or more differences can be obtained for the three or more representative pieces of position information selected.

[0259] (10-2-7a) Frame Combination The storage process further stores N pieces of determinable position information corresponding to the next frame. When two or more pieces of motion position information obtained by the motion detection process for three or more representative determinable position information correspond to a moving object (200c) other than the human body 301, the differential acquisition process sets one frame and the next frame as a new frame. Furthermore, the differential acquisition process selects three or more representative determinable position information representing the new frame from the 2×N pieces of determinable position information corresponding to the new frame, and obtains two or more differences for the selected three or more representative determinable position information. The motion detection process obtains two or more pieces of motion position information based on the two or more differences obtained by the differential acquisition process for the three or more representative determinable position information representing the new frame.

[0260] In this way, when more than two differences cannot be obtained in one frame, a new frame can be constructed by using the next frame to obtain more than two differences in the new frame.

[0261] (10-2-8) Specific example of motion detection (10-2-8a) Determination of one-dimensional position due to an antenna The radio wave sensor 1 includes at least one antenna for receiving reflected waves Re. The received signal is an IF signal generated during a single transmission / reception operation based on the transmitted wave Tr and the reflected wave Re received by one antenna. The IF signal represents the frequency difference between the transmitted wave Tr and the reflected wave Re, and is generated by mixing the transmitted wave Tr and the reflected wave Re.

[0262] The conversion process is a Fourier transform process. Fourier transform processing is the process of applying a Fourier transform to the IF signal. Position information that can be determined is the Fourier transform result of the Fourier transform process. Storage processing is the process of storing the complex Fourier transform result corresponding to a single antenna.

[0263] The moving object detection process includes a distance measurement process. The distance measurement process in this example measures the distance from one antenna to an object estimated to be a moving object (301, 200c) based on the difference between the complex Fourier transform results corresponding to one antenna. The moving object position information is one-dimensional position information based on the distance measurement result of the distance measurement process.

[0264] This improves the accuracy of detecting a moving object. For example, the accuracy of processing related to detecting a moving object (301, 200c) (moving object detection processing) can be improved by using a radio wave sensor 1 that uses radio waves modulated in an FMCW manner. Furthermore, even if there is only one antenna that receives the reflected wave Re, it is possible to improve the accuracy of at least measuring the distance to the moving object (301, 200c) (determining the one-dimensional position).

[0265] (10-2-8b) Determination of two-dimensional or three-dimensional position by multiple antennas The radio wave sensor 1 includes multiple antennas for receiving reflected waves Re. During a single transmission / reception operation, the radio wave sensor 1 outputs multiple IF signals corresponding to the multiple antennas. For each of these multiple antennas, a conversion unit 11 performs conversion processing, a storage unit 12 performs storage processing, a difference acquisition unit 13 performs difference acquisition processing, and a motion detection unit 14 performs motion detection processing, including distance measurement.

[0266] The position information can be determined as a group of Fourier transform results consisting of complex Fourier transform results corresponding to the complex antennas. The storage process stores the group of complex Fourier transform results. The moving body detection process includes a complex ranging process for measuring the distance from each of the complex antennas to an object estimated to be a moving body (301, 200c) based on the difference between the complex Fourier transform results of each of the complex antennas. The moving body position information is two-dimensional or three-dimensional position information based on the complex ranging results corresponding to the complex ranging process.

[0267] In this way, the accuracy of the motion detection processing for determining the two-dimensional position (for example, the two-dimensional coordinates in the two-dimensional virtual space 500, or the combination of direction and distance, etc.) or the three-dimensional position (for example, the three-dimensional coordinates in the three-dimensional virtual space 500, or the combination of direction and distance, etc.) of the moving body (301, 200c) can be improved.

[0268] (10-2-9) Moving body detection method and program The motion detection method in this example includes at least step S1 (conversion step), step S2 (storage step), step S4 (difference acquisition step), steps S5 and S6 (motion detection steps), and step S7 (point placement step), among the various steps described in the embodiment. The program causes one or more processors to execute this motion detection method.

[0269] (10-3) Equipment control device The device control unit 225 constituting the device control apparatus 2B performs device control processing for controlling the device 200 based on at least a set of multiple points arranged in a virtual space (500), namely, a point group 501. In this way, by obtaining the point group 501 corresponding to the moving body (301, 200c) using the radio wave sensor 1 and controlling the device 200 based on the obtained point group 501, it is possible to control the device 200 using the radio wave sensor 1.

[0270] In particular, by obtaining a point group 501 corresponding to the human body 301 and controlling the device 200 based on the obtained point group 501, it is possible to control the device 200 based on the position and movement of the human body 301.

[0271] (10-3-1) Posture Estimation and Action Detection The motion detection process performed by the motion detection device 2A includes posture estimation processing. The posture estimation process estimates the posture of the human body 301. For example, the posture estimation process estimates the posture of the human body 301 based on the distribution of the point group 501 (Figures 13A to 14C). Specifically, the posture estimation process may be the process described in the embodiment using Figure 7.

[0272] The motion detection unit 224 performs motion detection processing. In this example, the motion detection processing is the process of detecting the motion of a person corresponding to the human body 301 based on the posture or posture changes estimated by the posture estimation processing. The device control processing in this example controls the device 200 based on the motion detected by the motion detection processing.

[0273] In this way, by estimating the posture or posture change of the human body 301 based on the distribution of the point group 501, detecting the actions of the person corresponding to the human body 301 based on the estimation result, and controlling the device 200 based on the detection result, it is possible to control the device 200 based on the posture of the human body 301 and then based on the actions of the person.

[0274] (10-3-2) Acquisition of Environmental Information The information acquisition unit 226 performs information acquisition processing. As previously mentioned, information acquisition processing is the process of acquiring environmental information related to the environment in which the human body 301 exists. The motion detection process further detects movement based on the environmental information acquired by the information acquisition process.

[0275] In this way, by utilizing environmental information in addition to posture or posture changes, the detection accuracy of the action can be improved and the control accuracy of the device 200 can be improved.

[0276] (10-3-3) Equipment control using automatic control information The device control process uses the pre-stored automatic control information to perform action-based control of the device 200. In this way, the action-based control of the device 200 can be automatically performed using the automatic control information.

[0277] (10-3-4) Automatic control of historical record information storage The history storage unit 227 in this example performs history storage processing. This history storage processing in this example stores automatic control history information. By storing automatic control history information in this manner, learning processing based on the automatic control history information can be performed, thereby improving the accuracy of control using the automatic control history information. Furthermore, as previously described, the automatic control history information associates the control details of the device 200 based on its actions with one or more of the following: time, environmental information, and a personal identification code. This allows for control tailored to each individual.

[0278] (10-3-5) Further storage of manual control history information The history storage process further stores manual control history information. Manual control history information refers to information related to the control history of device 200 based on operations performed on device 200. For example, operations such as adjusting the brightness of lighting, as described above, are performed via an operating element such as a remote control. By further storing manual control history information, the accuracy of control using both automatic and manual control history information can be improved.

[0279] For example, by repeatedly performing a learning process such as "When automatic control of brightening the lighting is performed based on the detection of a seat occupancy, and no operation to cancel the control content (for example, an operation to dim the lighting) is performed within a certain period of time (for example, within one minute) (hereinafter referred to as "default no action"), when a predetermined number of times (for example, three times a week or more) is detected within a predetermined period of time, learning is performed to consider the control content as appropriate," the control accuracy can be improved.

[0280] However, the above-described learning process can also be implemented when only automatic control history information is stored, without manual control history information. For example, a learning process can be repeated in which "while storing automatic control history information, a determination is made based on real-time operation information whether a predetermined number of times (for example, three or more times a week) of default no-action has been detected within a predetermined period, and when a predetermined number of times of default no-action has been detected within a predetermined period, learning is performed to determine that the control content is appropriate."

[0281] Alternatively, instead of storing automatic control history information, only manual control history information may be stored. For example, by repeatedly performing a process such as "When an operation to brighten the lighting is received within a certain period of time (e.g., within one minute) from the detection of a seated person based on stored manual control history information, and this operation is detected a certain number of times or more during a certain period (e.g., three or more times per week), then the control content "brighten the lighting in response to the seated person detection" is recorded in the automatic control information," this can improve the control accuracy related to automatic control of lighting brightness and diversify the content of automatic control.

[0282] As previously mentioned, manual control history information associates the details of the control of device 200 based on an operation with one or more of the following: time, environmental information, and a personal identification code. Learning unit 228 performs a learning process to update automatic control information when, for example, the time and operation indicated by the control history information meet pre-set update conditions. This learning process, utilizing automatic control history information, enables control tailored to individual actions.

[0283] (10-3-6) Update of automatic control information Alternatively, the learning unit 228 may execute a learning process to update the automatic control information when the time and control content indicated by the automatic control history information, and the time and operation indicated by the manual control history information related to the control history of the device 200 based on the operation of the device 200, meet pre-set update conditions. In this way, by using both automatic and manual control history information, a control process that is suitable for each individual's behavior and operation can be achieved.

[0284] (10-3-7) Cluster The virtual space (500) in this example is a three-dimensional space 500, similar to the embodiment. The motion detection process is performed by clustering the point group 501 to obtain a set of one or more clusters (CL, CL1, CL2), namely, a cluster group (CL, CL1, CL2), and the posture of the human body 301 is estimated based on the distribution of the cluster groups (CL, CL1, CL2) in the three-dimensional space 500. In this way, the posture of the human body 301 can be estimated with good accuracy based on the distribution of the cluster groups (CL, CL1, CL2).

[0285] (10-3-8) Estimation of Posture The motion detection process estimates the posture of the human body 301 based on the ratios of the length, width, and height of the three-dimensional figure 502 surrounding the cluster (CL, CL1, CL2). Thus, the posture of the human body 301 can be estimated with high accuracy and simplicity based on the ratios of the length D, width W, and height H of the three-dimensional figure 502 surrounding the cluster (CL, CL1, CL2).

[0286] (10-3-9) Action Detection The action detection process detects actions other than human body 301 operating device 200, namely, non-operational actions, based at least on the posture or posture changes estimated by the posture estimation process. The device control process controls device 200 based at least on the detected non-operational actions. By controlling device 200 based on non-operational actions (e.g., actions without the intention to operate device 200, such as daily activities), device control without the intention to operate device 200 (non-operational device control) is possible.

[0287] (10-3-10) Equipment control method and program The device control method is a device control method for detecting a moving object and controlling the device 200 based on the detection result. The device control method is, for example, a device control method for using a radio wave sensor 1 to perform an estimation related to a moving object, i.e., a moving object (human body 301, electric blinds 200c), and controlling the device 200 based on the estimation result. The radio wave sensor 1 performs a transmission and reception operation of transmitting a radio wave modulated in a predetermined manner, i.e., a transmission wave Tr, toward a real space (inside a room 400), receiving a reflected wave Re from the real space (400), and outputting a reception signal based on the transmission wave Tr and the reflected wave Re. The device control method includes: a conversion step (S1), a storage step (S2), a difference acquisition step (S4), a moving object detection step (S5, S6), a point configuration step (S7), and a device control step (S11). The conversion step (S1) is a process for converting a reception signal outputted by the radio wave sensor 1 into positional information capable of determining the position of each of one or more objects (301, 200c, 302) existing in a real space (400). The storage step (S2) is a process for storing the positional information converted by the conversion process. The difference acquisition step (S4) is a process for acquiring the difference between the plurality of positional information stored by the storage process. The moving body detection steps (S5, S6) are a process for detecting the moving body (301, 200c) based on the difference acquired by the difference acquisition process. The motion detection step (S5, S6) is, for example, a motion detection process that estimates one or more objects as being a moving object or a non-moving object (302) based on the obtained difference, determines the position of the object estimated to be a moving object (301, 200c), and obtains motion position information representing the determined position. The point configuration step (S7) is a point configuration process that virtually configures points corresponding to the motion position information obtained by the motion detection process in a virtual space (500) corresponding to the real space (400). The device control step (S11) controls the device 200 based on at least a set of multiple points configured in the virtual space (500), that is, a point group 501. The program causes one or more processors to execute this device control method.

[0288] (Second embodiment) Next, the second embodiment and modified examples of the present invention are described using Figures 1 to 22 (especially Figures 4, 5 and 18 to 22).

[0289] The second embodiment is identical to the first embodiment, except that it uses a distribution of multiple points (CL, CL1, CL2) to determine the processing of the object. Below, descriptions of previously mentioned matters will be omitted or simplified, and the similarities and differences with the first embodiment will be explained in detail.

[0290] (1) Commonalities with the first embodiment The radio wave sensor 1 transmits radio waves (transmitted waves Tr: hereinafter sometimes referred to as "radio waves Tr") in a predetermined direction, receives reflected waves Re, and outputs a received signal.

[0291] Furthermore, the term "transmitting in a predetermined direction" means emitting the radio wave Tr in a certain range centered on the predetermined direction. In other words, by pointing the radio wave sensor 1 in a predetermined direction, the radio wave Tr is propagated in a certain range centered on the predetermined direction.

[0292] The radio wave sensor 1 in the first or second embodiment is located above region R1 and transmits radio waves Tr in the height direction of region R1 (specifically, vertically downward, i.e., vertically downward). In other words, the predetermined direction is the height direction of region R1 (see FIG. 2 ), which is the Z direction in the three-dimensional space 500 shown in FIG. 13A and other figures. Thus, the radio wave sensor 1 transmits radio waves Tr in the height direction and receives the reflected waves Re of the transmitted radio waves Tr, outputting a received signal.

[0293] The processing unit 22 detects objects (301, 200c) using the received signal output by the radio wave sensor 1. Examples of the objects include a human body 301, a moving object other than the human body 301, namely, a motorized blind 200c, and a stationary object 302, such as a table 302 (see FIG2 ). The human body 301 is a moving object that performs actions such as walking, in other words, a non-periodic (or irregular) motion. The motorized blind 200c is a moving object that performs periodic (or regular) motion, such as opening and closing.

[0294] (2) Differences from the first embodiment (2-1) Object Determination in the First Implementation: Determination Based on the Dimension Ratio of Length, Width, and Height The processing unit 22 of the first embodiment determines objects such as the human body 301 and the stationary object 302 based on the size ratio of the length D, width W and height H of the rectangular parallelepiped 502 that surrounds one or more points (i.e., a collection of one or more points, i.e., a point group 501, or one or more clusters CL, CL1, CL2 formed by the point group 501) in the three-dimensional space 500 corresponding to the region R1 (see Figures 13A to 14C).

[0295] (2-2) Object Determination in the Second Embodiment 1: Determination Based on the Positional Relationship Between Ends in the Height Direction In contrast, the processing unit 22 of the second embodiment identifies the object based on the positional relationship between two ends of a distribution of one or more points (one or more clusters CL, CL1, CL2 or point group 501).

[0296] In this embodiment, the objects include, in addition to the moving objects (301, 200c) such as the human body 301 and the electric blinds 200c shown in FIG2 , a cleaning robot 303 shown in FIG20A , an electric fan 304 shown in FIG20B , and a cat 305 shown in FIG21A and FIG21B (hereinafter sometimes referred to as "objects 301-305"). The cleaning robot 303 is an example of a moving object that moves automatically (e.g., walks on its own). The electric fan 304 is an example of a moving object that partially moves (e.g., blades rotate, including the top of the blades swinging). The cat 305 is an example of an animal (e.g., a pet) other than the human body 301.

[0297] (2-2-1) Both ends of the distribution in the height direction The two ends of the distribution (CL, CL1, CL2) in the height direction are, for example, the upper cluster CL1 and the lower cluster CL2 in the distribution shown in FIG13A (a point group 501 including two clusters CL1 and CL2).

[0298] The positional relationship between the two ends in the height direction is, for example, the distance between the upper cluster CL1 and the lower cluster CL2. The distance between the upper cluster CL1 and the lower cluster CL2 is, for example, the difference between the height of the upper cluster CL1 relative to the XY plane shown in FIG13A (corresponding to the bottom surface of the region R1 shown in FIG2 ) and the height of the lower cluster CL2 relative to the XY plane (i.e., the height difference DH between the upper cluster CL1 and the lower cluster CL2: see FIG20A and FIG20B ).

[0299] Furthermore, the height of cluster CL1 is the Z component of the position information corresponding to cluster CL1, and the height of cluster CL2 is the Z component of the position information corresponding to cluster CL2.

[0300] Alternatively, the two ends of the distribution (CL, CL1, CL2) in the height direction can be, for example, the distance between two horizontal planes passing through the highest point and the lowest point of a cluster CL. In other words, it can also be the difference between the Z component of the coordinate of the highest point and the Z component of the coordinate of the lowest point.

[0301] Furthermore, the positional relationship between the two ends can include not only distances such as the height difference DH but also temporal changes in distance, i.e., fluctuations. Variations include, for example, periodic or regular changes in the object (such as the movement of the robot vacuum 303 or the swinging of the fan 304), non-periodic or irregular changes in the object (such as the walking of a person 301 or a cat 305), changes in the entire object (such as the movement of the robot vacuum 303 or the walking of a person 301), and changes in a portion of the object (such as the swinging of the fan 304 or the movement of a person 301's hand).

[0302] Specifically, the processing unit 22 obtains multiple pieces of positional information corresponding to multiple parts of the object (301-305), and obtains the distribution (CL, CL1, CL2) of multiple points (point group 501) corresponding to each of the multiple pieces of positional information. Furthermore, the processing unit 22 identifies the object (301-305) based on the positional relationship between the two ends of the distribution of the obtained point group 501 in a predetermined direction.

[0303] As previously mentioned, the predetermined direction in this embodiment is the height direction of region R1, i.e., the Z direction of three-dimensional space 500 corresponding to region R1 (see Figures 20A-21B). The positional relationship in this embodiment is the distance between the two ends in the height direction (Z direction). Specifically, it is the height difference DH between the upper cluster CL1 and the lower cluster CL2, or the height H of the cuboid 502.

[0304] The distance between the two ends in the height direction is the difference in Z-component between the two pieces of position information corresponding to the two ends. For example, the processing unit 22 can also calculate the difference in Z-component between the position information corresponding to the upper cluster CL1 and the position information corresponding to the lower cluster CL2 as the height difference DH.

[0305] Furthermore, the position information corresponding to the upper cluster CL1 may be, for example, the coordinates of the representative point of cluster CL1. The representative point of cluster CL1 may be, for example, the highest point of cluster CL1, but may also be the center of gravity or the lowest point. Furthermore, the position information corresponding to the lower cluster CL2 may be, for example, the coordinates of the representative point of cluster CL2. The representative point of cluster CL2 may be, for example, the lowest point of cluster CL2, but may also be the center of gravity or the highest point.

[0306] In this way, the processing unit 22 uses the reception signal of the reflected wave Re of the radio wave Tr transmitted in the height direction (Z direction) from the radio wave sensor 1 located above the region R1 to obtain the complex position information corresponding to the complex parts constituting the object (301-305), and obtains the distribution (CL, CL1, CL2) of the complex points (501). In addition, the object (301-305) is identified based on the positional relationship (DH, H) between the two ends in the height direction of the distribution (CL, CL1, CL2) of the complex points (501), making it easier to identify the object (301-305).

[0307] Furthermore, by making the positional relationship include the distance (height difference DH) between the two ends of the distribution (CL, CL1, CL2) in the height direction, it is possible to make it easier to identify the object (301~305) and improve the accuracy.

[0308] (2-3) Object Determination 2 in the Second Implementation: Determination of Discreteness Based on Distribution in a Plane Perpendicular to the Height Direction The processing unit 22 further identifies the objects (301 to 305) based on the dispersion of the distribution (CL, CL1, CL2) of the point group 501 on a plane whose normal direction is the height direction (Z direction).

[0309] The plane whose normal direction is the height direction (Z direction) refers to a horizontal plane, such as the XY plane along the X and Y directions shown in FIG. 20A , or a plane parallel to the XY plane.

[0310] The so-called horizontal dispersion refers to, for example, the distance DW between the two ends in the horizontal direction (X direction) as shown in Figures 20A to 21B. The distance DW is equivalent to the width W of the rectangular parallelepiped 502 in the first embodiment.

[0311] The discreteness on the horizontal plane (XY plane) may include, for example, at least one of the discreteness in the transverse direction (X direction), i.e., the positional relationship between the two end portions in the transverse direction, and the discreteness in the longitudinal direction (Y direction), i.e., the positional relationship between the two end portions in the longitudinal direction.

[0312] In this way, the processing unit 22 determines the object based on the dispersion (DW, W) of the distribution (CL, CL1, CL2) of the point group 501 in a plane (horizontal plane) perpendicular to the height direction in addition to the positional relationship between the ends of the distribution (CL, CL1, CL2) of the point group 501 in a predetermined direction, thereby making it easier to determine the object (301-305).

[0313] However, even if the positional relationship between the ends of the distribution (CL, CL1, CL2) of the point group 501 in the height direction is not taken into consideration, but the dispersion (DW, W) of the distribution (CL, CL1, CL2) of the point group 501 in the plane (horizontal plane) perpendicular to the height direction is taken into consideration, it is possible to make it easier to identify the object (301~305).

[0314] (2-4) Further determination of objects based on positional relationships and temporal changes in positional relationships, as well as discreteness and temporal changes in discreteness The aforementioned one or more points are specifically referred to as moving points, that is, moving points. That is, the so-called "points" in the first embodiment or the second embodiment are moving points, and hereinafter, they may be appropriately described as "moving points".

[0315] The positional relationship further includes temporal variations in distance (height difference DH), such as variations in the height of the object. Furthermore, the aforementioned dispersion further includes temporal variations in dispersion, such as expansion and contraction of the object within the horizontal plane. Thus, by including temporal variations in distance and dispersion in the positional relationship, identification of the objects (301-305) can be made easier.

[0316] (2-4-1) Obtaining temporal changes such as positional relationships based on multiple time differences The radio wave Tr is transmitted in each of the multiple frames, and the processing unit 22 calculates the multiple time differences by repeatedly calculating the difference, that is, the time difference, between two reception signals received in two different frames in the multiple frames.

[0317] The processing unit 22 obtains a distribution (CL, CL1, CL2) of one or more moving points, which are points where the time difference is greater than a critical value, from the multiple time differences. Furthermore, the processing unit 22 identifies the object (301-305) based on at least one of a positional relationship (DH, H) of the distribution (CL, CL1, CL2) of the one or more moving points in a predetermined direction (height direction) and a temporal change in the positional relationship, and a dispersion (DW, W) on a plane (XY plane) with the predetermined direction (height direction) as a normal direction and a temporal change in the dispersion (DW, W).

[0318] In this way, the processing unit 22 identifies the object (301-305) based on the positional relationship (DH, H) between the two ends of the distribution (CL, CL1, CL2) of one or more moving points obtained from the complex time difference and the time change of the positional relationship, and the discreteness (DW, W) in the horizontal plane (XY plane) and at least one of the time change of the discreteness (DW, W), thereby making it easier to identify the object (301-305).

[0319] (2-4-2) Action determination based on the dispersion of the distribution level The processing unit 22 determines the movement of the object (301-305) based on the temporal change of the dispersion (DW, W) of the distribution (CL, CL1, CL2) of one or more moving points on the horizontal plane.

[0320] In this way, the processing unit 22 determines the object (301~305) based on the time change of the discreteness (DW, W) of the distribution (CL, CL1, CL2) of the complex moving points in the horizontal plane (XY plane), making it easier to determine the object (301~305).

[0321] (2-4-3) Determination of the action based on the positional relationship between the two ends in the height direction of the distribution Alternatively, the processing unit 22 may determine the movement of the object (301-305) based on the temporal change of the positional relationship (DH, H) of the distribution (CL, CL1, CL2) of one or more moving points in the height direction (Z direction).

[0322] In this way, the processing unit 22 determines the object (301~305) based on the time change of the position relationship (DH, H) of the distribution (CL, CL1, CL2) of the multiple moving points in a predetermined direction (height direction), making it easier to determine the object (301~305).

[0323] (2-4-4) Action determination based on the dispersion of the distribution level and the time variation of the dispersion The processing unit 22 can also determine the object (301~305) and the movement of the determined object (301~305) based on the discreteness (DW, W) of the distribution (CL, CL1, CL2) of one or more moving points on a plane with the height direction as the normal direction, that is, a horizontal plane (the XY plane in the three-dimensional space 500, or a plane parallel to the XY plane), and the time change of the discreteness (DW, W).

[0324] However, the processing unit 22 can also determine the object (301~305) and the action based on either the dispersion (DW, W) on the horizontal plane of the distribution (CL, CL1, CL2) or the time change of the dispersion (DW, W).

[0325] Furthermore, the processing unit 22 can also perform either determination of the object (301~305) or determination of the action based on the dispersion (DW, W) on the horizontal plane of the distribution (CL, CL1, CL2) and the time change of the dispersion (DW, W).

[0326] Furthermore, the processing unit 22 can also perform either the determination of the object (301~305) or the determination of the action based on either the dispersion (DW, W) on the horizontal plane of the distribution (CL, CL1, CL2) or the time change of the dispersion (DW, W).

[0327] In this way, the processing unit 22 determines the object (301-305) based on at least one of the dispersion (DW, W) of the distribution (CL, CL1, CL2) of the multiple moving points on a plane perpendicular to the height direction and the time change of the dispersion (DW, W), making it easier to determine the object (301-305).

[0328] (2-4-5) Determination of the motion based on the positional relationship between the two ends in the height direction of the distribution and the time change of the positional relationship The processing unit 22 can also determine the object (301~305) and the movement of the determined object (301~305) based on the positional relationship (DH, H) in the height direction of the distribution (CL, CL1, CL2) of one or more moving points and the time change of the positional relationship (DH, H).

[0329] However, the processing unit 22 can also determine the object (301~305) and the action based on either the positional relationship (DH, H) of the distribution (CL, CL1, CL2) in a predetermined direction (height direction) or the time change of the positional relationship (DH, H).

[0330] Furthermore, the processing unit 22 can also perform either determination of the object (301~305) or determination of the action based on the positional relationship (DH, H) of the distribution (CL, CL1, CL2) in a predetermined direction (height direction) and the temporal change of the positional relationship (DH, H).

[0331] Furthermore, the processing unit 22 can also perform either the determination of the object (301~305) or the determination of the action based on either the positional relationship (DH, H) of the distribution (CL, CL1, CL2) in a predetermined direction (height direction) or the time change of the positional relationship (DH, H).

[0332] In this way, the processing unit 22 determines the object (301-305) based on at least one of the positional relationship (DH, H) between the ends of the distribution (CL, CL1, CL2) of the multiple moving points in a predetermined direction (height direction) and the time change of the positional relationship, thereby making it easier to determine the object (301-305).

[0333] (2-4-6) Calculation of time differences between multiple different time intervals The processing unit 22 obtains mutually different multiple time differences (ΔA1, ΔA2, ΔA3, etc.) by calculating time differences at mutually different multiple time intervals (T, 2×T, ... 5×T: refer to FIG. 10 ).

[0334] Thereby, the accuracy of determining the object (301-305) can be improved, and the determination of the object (301-305) can be made easier.

[0335] (2-5) Action Example The control device 2 of this embodiment is similar to the control device 2 of the first embodiment and operates basically according to the flowcharts of Figures 4 to 7. However, in this embodiment, the processing from steps S8 to S10 in the processing shown in Figures 4 and 5, namely, "the processing of estimating the posture of the human body and detecting non-operational actions based on the estimation results," is replaced with the processing of six steps S8a, S8b, S9a to S9c, and S10a shown in Figure 18, namely, "the processing of determining the type of the object based on the height of the object."

[0336] (2-5-1) Processing for determining the height of an object After step S7, the processing unit 22 determines whether four or more points are arranged (step S8a). If it is determined that four or more points are not arranged (step S8a: No), the processing returns to step S1 (see FIG. 4).

[0337] If it is determined in step S8a that four or more points have been arranged (yes), the processing unit 22 determines whether two or more point clusters have been formed (step S8b). If it is determined that two or more point clusters have not been formed (no in step S8ba), the process returns to step S1 (see FIG4).

[0338] If it is determined in step S8b that two or more point clusters have been formed (Yes), the processing unit 22 sets the point cluster with the highest height among the two or more point clusters formed as the upper end of the object (step S9a). Next, the processing unit 22 sets the point cluster with the lowest height among the two or more point clusters formed as the lower end of the object (step S9b). Next, the processing unit 22 sets the height difference between the upper end and the lower end as the height of the object (step S9c).

[0339] Next, the processing unit 22 performs a type identification process to identify the type of the object based on the height of the object (step S10a). Thereafter, the process proceeds to step S11 (see FIG. 5 ).

[0340] (2-5-2) Type determination process The type determination process of step S10 a is executed according to the flowchart of FIG. 19 , for example.

[0341] In this example, the memory of control device 2 pre-stores a first threshold value and a second threshold value (provided the first threshold value is less than the second threshold value), as well as multiple feature information corresponding to multiple types of objects. The multiple types of objects include, for example, a robot vacuum cleaner 303, an electric fan 304, a cat 305, and a human body 301. Human body 301 includes three types of bodies: adults, children, and infants. The multiple feature information includes the first feature information corresponding to robot vacuum cleaner 303 (Figure 20A), the second feature information corresponding to fan 304 (Figure 20B), the third feature information corresponding to cat 305 (Figures 21A and 21B), the fourth feature information corresponding to an adult (Figure 13A), the fifth feature information corresponding to a child, and the sixth feature information corresponding to an infant.

[0342] Furthermore, the child is shorter in height (standing height) than the adult shown in FIG13A . Furthermore, the infant is shorter in height than the child because of its recumbent position.

[0343] The first characteristic information corresponding to the robot vacuum 303 is information regarding "the height (DH, H) is below a first threshold and is periodically operating." The second characteristic information corresponding to the fan 304 is information regarding "the height is greater than a first threshold and below a second threshold and is periodically operating."

[0344] The third characteristic information corresponding to the cat states that "the height is below the first threshold, and the point cluster will extend horizontally when it begins to change." Figure 21A shows the state of the point cluster before the change begins, and Figure 21B shows the state of the point cluster after the change begins. The horizontal (X-direction) dispersion (DW, W) of the point cluster (CL1, CL2) in Figure 21B is significantly greater than the horizontal (X-direction) dispersion (DW, W) of the point cluster (CL1, CL2) in Figure 21A.

[0345] The fourth characteristic information corresponding to adults is information about "height greater than the second threshold and performing non-periodic movements." The fifth characteristic information corresponding to children is information about "height greater than the first threshold and below the second threshold and performing non-periodic movements." The sixth characteristic information corresponding to infants is information about "height less than the first threshold and performing non-periodic movements."

[0346] In the type determination process, the processing unit 22 initially determines whether the height of the object is below a first threshold (step S101). If the height of the object is determined to be not below the first threshold (no in step S101, i.e., greater than the first threshold), the process proceeds to step S107.

[0347] If the object's height is determined to be below the first threshold value (Yes) in step S101, the processing unit 22 determines whether the fluctuation of at least one of the two or more point clusters is periodic (step S102). If the fluctuation of any of the point clusters is determined to be non-periodic (No in step S102), the process proceeds to step S104.

[0348] If the fluctuation of at least one point group is determined to be periodic (Yes) in step S102, processing unit 22 sets the type information representing the object type to "cleaning robot" based on the first feature information (step S103). The process then returns to the upper flow chart (see Figure 18).

[0349] In step S104, the processing unit 22 determines whether the point group extends in the horizontal direction (X direction) when the point group starts to change. If it is determined that the point group does not extend in the horizontal direction (X direction) when the point group starts to change (step S104 is no), the processing proceeds to step S106.

[0350] If it is determined in step S104 that the point cloud is extending horizontally (in the X direction) when it begins to move (Yes), the processing unit 22 sets the category information to "cat" based on the third feature information (step S105). The process then returns to the upper flowchart (see Figure 18).

[0351] In step S106, the processing unit 22 sets the category information to "baby" based on the sixth characteristic information. Thereafter, the processing returns to the upper flowchart (see FIG. 18 ).

[0352] In step S107, it is determined whether the height of the object is greater than the first threshold and less than the second threshold. If it is determined that the height of the object is not greater than the first threshold and less than the second threshold (step S107 is no: that is, greater than the second threshold), the process proceeds to step S111.

[0353] If it is determined in step S107 that the height of the object is greater than the first threshold value and less than the second threshold value (Yes), the processing unit 22 determines whether the fluctuation of at least one of the two or more point clusters is periodic (step S108). If it is determined that the fluctuation of any point cluster is non-periodic (step S108: No), the process proceeds to step S110.

[0354] If the fluctuation of at least one point group is determined to be periodic (Yes) in step S108, the processing unit 22 sets the category information to "electric fan" based on the second feature information (step S109). The process then returns to the upper flowchart (see FIG18 ).

[0355] In step S110, the processing unit 22 sets the category information to "children" based on the fifth characteristic information. Thereafter, the processing returns to the upper flowchart (see FIG. 18 ).

[0356] In step S111, the processing unit 22 sets the category information to "adult" based on the fourth characteristic information. Thereafter, the process returns to the upper flowchart (see FIG. 18 ).

[0357] (2-6) Detection methods and procedures A detection method uses a received signal, i.e., a signal based on a reflected wave Re of a radio wave Tr transmitted in a predetermined direction (height direction), to detect objects (301-305) within region R1. The detection method includes processing steps for obtaining information related to the position of the objects (301-305) within region R1 by performing ranging processing based on the received signal. In the first embodiment, these processing steps are steps S8-S10, and in the second embodiment, these steps are steps S8a-10a and steps S101-S111. In the processing steps (S8-S10, S8a-10a, and S101-S111), multiple position information corresponding to multiple parts constituting the object (301-305) is obtained, and the object (301-305) is determined based on the positional relationship (height difference DH, height H of the rectangular parallelepiped 502) between the two ends in a predetermined direction (height direction) of the distribution of multiple points (CL, CL1, CL2) corresponding to the multiple position information.

[0358] The program is used to enable one or more processors to execute the detection method.

[0359] (3) Modification Next, a modified example of this embodiment is described. The description of previously mentioned matters is omitted or simplified, and the differences are described in detail.

[0360] (3-1) First Variation of the Detection System In the process of FIG. 18 , the category determination process of step S10 a may be followed by the individual determination process shown in FIG. 22 .

[0361] In this example, two sets of personal information related to two adults and two conditions corresponding to the two sets of personal information are pre-stored in the memory of control device 2. One set of personal information is related to Mr. AA and includes identification information "AA" for identifying Mr. AA and his personal information "H = 180 cm, W = 40 cm." The other set of personal information is related to Mr. BB and includes identification information "BB" for identifying Mr. BB and his personal information "H = 160 cm, W = 50 cm."

[0362] Mr. AA's personal information "H=180cm, W=40cm" corresponds to the point distribution shown in FIG. 23A , and Mr. BB's personal information "H=160cm, W=50cm" corresponds to the point distribution shown in FIG. 23B .

[0363] Of the two conditions, the first condition corresponding to the personal information "H=180cm, W=40cm" is, for example, "175≦H<185 and 35≦W<45", and the second condition corresponding to the personal information "H=160cm, W=50cm" is, for example, "155≦H<165 and 45≦W<55".

[0364] In the individual identification process of FIG22, the processing unit 22 first determines whether the category information is "adult" (step S201). If the category information is not determined to be "adult" (step S201: No), the process returns to the upper flowchart (see FIG18).

[0365] If the category information is determined to be "adult" (Yes) in step S201, the processing unit 22 determines whether the height H and width W of the point cluster satisfy the first condition (step S202). If the height H and width W of the point cluster do not satisfy the first condition (No in step S202), the process proceeds to step S204.

[0366] If it is determined in step S202 that the height H and width W of the point group satisfy the first condition (yes), the processing unit 22 sets "AA" to the identification information (step S203). Thereafter, the processing returns to the upper flowchart (see FIG. 18 ).

[0367] In step S204, the processing unit 22 determines whether the height H and width W of the point group satisfy the second condition. If it is determined that the height H and width W of the point group do not satisfy the second condition (step S204: No), the process proceeds to step S206.

[0368] If it is determined in step S204 that the height H and width W of the point group satisfy the second condition (yes), the processing unit 22 sets "BB" to the identification information (step S205). Thereafter, the processing returns to the upper flowchart (see FIG. 18).

[0369] In step S206, the processing unit 22 sets the identification information to "unregistered person." Thereafter, the processing returns to the upper flowchart (see FIG18 ).

[0370] Returning to the process in the flowchart of Figure 18, the process proceeds to step S11 (see Figure 5). In the automatic control of step S11, when the category information associated with the detected object is "adult," control can be performed that takes into account the identification information used to identify the individual. For example, if the identification information is "AA," the power of television 200b can be turned on, while if the identification information is "BB," the power of 200b cannot be turned on.

[0371] Furthermore, in this variation, the height H can be represented by the height difference DH, and the width W can be represented by the distance DW. Furthermore, in this variation, even when the subject is a child, individual identification can be achieved in the same manner as when the subject is an adult. Furthermore, the specific numerical values for the first and second conditions are merely examples and can be modified as appropriate.

[0372] According to this variation, when the object (301-305) is determined to be a human body 301, the individual corresponding to the human body 301 can be further determined based on pre-stored personal information, height H (DH), and width W (DW). As a result, for example, device control that can be tailored to individual needs can be achieved.

[0373] (3-2) Second Modification of the Detection System The detection system 100 may not include the radio wave sensor 1. In this variation, the radio wave sensor 1 is located outside the detection system 100. Similar to the radio wave sensor 1 of the first or second embodiment, the radio wave sensor 1 transmits radio waves Tr in a predetermined direction (height direction), receives reflected waves Re, and outputs a received signal. However, in this variation, the output is the transmission of the received signal to the detection system 100. In the detection system 100, the receiving unit 21 receives the received signal from the radio wave sensor 1, and the processing unit 22 processes the received signal received by the receiving unit 21.

[0374] This modification is also the same as the first embodiment or the second embodiment, and can make it easier to identify the object (301~305).

[0375] (3-3) Third Variation of the Detection System The detection system 100 may also be a radio wave sensor 1. Similar to the radio wave sensor 1 in the first or second embodiment, the radio wave sensor 1 in this variation transmits radio waves Tr in a predetermined direction (height direction) and receives reflected waves Re of the radio waves Tr. The radio wave sensor 1 detects objects (301-305) within the region R1 using signals based on the reflected waves Re, i.e., received signals. Specifically, the radio wave sensor 1 in this variation includes a control device 2, as shown in FIG. 24 .

[0376] This modification is also the same as the first embodiment or the second embodiment, and can make it easier to identify the object (301~305).

[0377] (3-4) Other variations of the detection system The detection system 100 may also be a wiring device including the control device 2. The wiring device may further include the radio wave sensor 1. The wiring device may also be a socket connected to the lighting device 200a shown in FIG. 2, for example.

[0378] The detection system 100 may also be a lighting fixture comprising a control device 2 and a fixture body. The fixture body may include, for example, light-emitting elements such as LEDs, a driver circuit for driving the light-emitting elements, and a power supply for supplying power to the driver circuit. This lighting fixture may also include a radio wave sensor 1. For example, the lighting fixture 200a shown in FIG2 may also have a built-in radio wave sensor 1 and a control device 2.

[0379] (3-5) Variations on the Configuration of Radio Wave Sensors and the Direction of Radio Wave Transmission The radio wave sensor 1 does not necessarily need to be located above the region R1. In this variation, the radio wave sensor 1 is located to the side of the region R1. In this case, the direction of transmission of the radio wave Tr, or the "predetermined direction," is the horizontal direction (lateral direction).

[0380] According to this modification, there is a possibility that the detection accuracy of the object will be reduced compared to the case where the predetermined direction is the height direction (vertically downward), but the processing unit 22 can improve the detection accuracy by determining the object (301~305) based on the positional relationship between the ends of the distribution (CL, CL1, CL2) in the horizontal direction (for example, the X direction, the Y direction, or both the X direction and the Y direction).

[0381] Furthermore, the radio wave sensor 1 can be located on any side relative to the region R1, as long as it can transmit the radio wave Tr to the region R1 in a predetermined direction. In this manner, the processing unit 22 identifies the target object (301-305) based on the positional relationship between the ends of the distribution (CL, CL1, CL2) in the predetermined direction and the temporal change in the positional relationship, as well as the dispersion in a plane perpendicular to the predetermined direction and the temporal change in the dispersion, thereby improving detection accuracy.

[0382] (3-6) Variations of the Detection Method The detection process of the detection system 100 can also detect stationary objects, not just moving ones. For example, such a process can be achieved by omitting the calculation of the time difference of the received signal from the detection process in the second embodiment. While omitting the calculation of the time difference may make it difficult to identify the motion of the object, it is possible to identify the object based on the positional relationship between the ends of the point group distribution in a predetermined direction, as well as the dispersion within a plane perpendicular to the predetermined direction.

[0383] (4) Summary According to a first embodiment of the detection system (100), the detection system (100) detects objects (301-305) within an area (R1) using a received signal. The received signal is based on a signal of a reflected wave (Re) of an electric wave (transmitted wave Tr) transmitted in a predetermined direction (height (Z) direction). The detection system (100) includes a processing unit (22) for obtaining information related to the position of the objects (301-305) within the area (R1) by performing ranging processing based on the received signal. The processing unit (22) obtains plural position information corresponding to plural parts constituting the objects (301-305), and determines the objects (301-305) based on the positional relationship (height difference DH, height H of the rectangular parallelepiped 502) between the two ends in the predetermined direction (height (Z) direction) of the distribution of plural points (CL, CL1, CL2) corresponding to the plural position information.

[0384] According to this aspect, when detecting an object (301-305) using a reflected wave (Re) of a radio wave (Tr) transmitted in a predetermined direction (height direction), the positional relationship (DH, H) between the ends of a distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction) is taken into consideration, thereby making it easier to identify the object (301-305). Furthermore, the predetermined direction (height direction) is preferably the height direction, but directions other than the height direction can also make it easier to identify the object (301-305).

[0385] According to the second aspect of the detection system (100), in the first aspect, the positional relationship (DH, H) includes the distance (DH, H) between the two ends.

[0386] In this manner, by taking into account the minimum distance (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in a predetermined direction (height direction), the determination of the object (301~305) can be made easier and the accuracy can be improved.

[0387] According to the third embodiment of the detection system (100), in the first or second embodiment, the processing unit (22) further determines the object (301-305) based on the discreteness (the distance DW between the ends in the horizontal direction, the width W of the rectangular parallelepiped 502) of the distribution (CL, CL1, CL2) on a plane (horizontal plane (XY plane, or a plane parallel to the XY plane)) with a predetermined direction (height direction) as the normal direction.

[0388] In this manner, by (further) taking into account the dispersion (DW, W) of the distribution of the plurality of points (CL, CL1, CL2) in a plane (horizontal plane) perpendicular to a given direction (height direction), it is possible to more easily identify the object (301-305).

[0389] According to the fourth embodiment of the detection system (100), in the third embodiment, the positional relationship includes a distance (DH, H) between the two ends and a dispersion (DW, W) of the distribution (CL, CL1, CL2) on a plane (horizontal plane). The processing unit (22) determines the type of the object (301-305) based on the distance (DH, H) and the dispersion (DW, W).

[0390] In this manner, the type of the object (301-305) can be determined by taking into account the distance (DH, H) between the ends of the distribution (CL, CL1, CL2) in a predetermined direction (height direction) and the dispersion (DW, W) in a plane (horizontal plane) perpendicular to the predetermined direction (height direction).

[0391] According to a fifth aspect, the detection system (100) stores personal information including information related to the body dimensions of each of the plurality of individuals in advance in the fourth aspect. When the type is determined to be a human body (301), the processing unit (22) further determines the individual corresponding to the human body (301) among the plurality of individuals based on the individual personal information corresponding to the plurality of individuals, the distance (DH, H) and the dispersion (DW, W).

[0392] In this manner, when the type of the object (301-305) is determined to be a human body (301), the individual corresponding to the human body (301) can be further determined by taking the distance (DH, H) and the dispersion (DW, W) into consideration.

[0393] According to the sixth aspect of the detection system (100), in the first aspect, the radio wave (Tr) is transmitted in each frame of the plurality of frames. The processing unit (22) calculates the plurality of time differences by repeatedly calculating the difference between two reception signals received in two different frames in the plurality of frames, i.e., the time difference. The processing unit (22) obtains points with time differences above a critical value, i.e., a distribution of one or more moving points (CL, CL1, CL2) from the plurality of time differences. The processing unit (22) determines the object (301-305) based on the positional relationship (DH, H) and the time change of the positional relationship of the distribution of one or more moving points (CL, CL1, CL2) in a predetermined direction (height direction), and the discreteness (DW, W) on a plane (XY plane or a plane parallel to the XY plane) with the predetermined direction (height direction) as the normal direction and at least one of the time change of the discreteness (DW, W).

[0394] In this manner, by taking into account at least one of the positional relationship (DH, H) of the distribution (CL, CL1, CL2) of the complex moving points obtained from the complex time differences and the temporal variation of the positional relationship, and the discreteness (DW, W) in the plane (horizontal plane) and the temporal variation of the discreteness (DW, W), the identification of the object (301-305) can be made easier.

[0395] According to the seventh aspect of the detection system (100), in the sixth aspect, the processing unit (22) determines the movement of the object (301-305) based on the time change of the discreteness (DW, W) of the distribution (CL, CL1, CL2) of one or more moving points on a plane (horizontal plane).

[0396] In this manner, by taking into account the temporal variation of the discreteness (DW, W) of the distribution (CL, CL1, CL2) of the plurality of moving points in a plane (horizontal plane), it is possible to more easily identify the object (301-305).

[0397] According to the eighth aspect of the detection system (100), in the sixth or seventh aspect, the processing unit (22) determines the movement of the object (301-305) based on the time change of the position relationship (DH, H) of the distribution (CL, CL1, CL2) of one or more moving points in a predetermined direction (height direction).

[0398] In this manner, by taking into account the temporal changes in the positional relationship (DH, H) of the distribution of multiple moving points (CL, CL1, CL2) in a predetermined direction (height direction), it is possible to more easily identify the object (301-305).

[0399] According to a ninth aspect of the detection system (100), in the sixth aspect, the processing unit (22) determines an object (301-305) or determines the movement of the object (301-305) based on at least one of the dispersion (DW, W) of a distribution of one or more moving points (CL, CL1, CL2) on a plane (XY plane, or a plane parallel to the XY plane) with a predetermined direction (height direction) as the normal direction, and the time change of the dispersion (DW, W).

[0400] In this manner, by taking into consideration at least one of the dispersion (DW, W) of the distribution (CL, CL1, CL2) of the plurality of moving points on a plane perpendicular to a predetermined direction (height direction) and the temporal variation of the dispersion (DW, W), it is possible to more easily identify the object (301-305).

[0401] According to the detection system (100) of the tenth aspect, in the sixth or ninth aspect, the processing unit (22) determines the object (301-305) or the movement of the object (301-305) based on the positional relationship (DH, H) of the distribution (CL, CL1, CL2) of one or more moving points in a predetermined direction (height direction) and at least one of the time change of the positional relationship (DH, H).

[0402] In this manner, by taking into consideration at least one of the positional relationship (DH, H) between the ends of the distribution of multiple moving points (CL, CL1, CL2) in a predetermined direction (height direction) and the temporal change of the positional relationship, it is possible to make it easier to identify the object (301~305).

[0403] According to the eleventh aspect, the detection system (100) is in any one of the sixth to tenth aspects, wherein the processing unit (22) performs a process of calculating a time difference at a plurality of mutually different time intervals.

[0404] According to this aspect, the accuracy of determining the object (301-305) can be improved, and the determination of the object (301-305) can be made easier.

[0405] The detection system (100) according to the twelfth aspect is any one of the first to eleventh aspects, further comprising a radio wave sensor (1). The radio wave sensor (1) transmits radio waves (Tr) in a predetermined direction (height direction), receives reflected waves (Re), and outputs a received signal.

[0406] In this manner, when a radio wave (Tr) is transmitted in a predetermined direction (height direction) and a reflected wave (Re) of the transmitted radio wave (Tr) is used to detect an object (301-305), the positional relationship (DH, H) between the ends of a distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction) is taken into consideration, thereby making it easier to identify the object (301-305).

[0407] According to the thirteenth aspect, the detection system (100) is in the twelfth aspect, wherein the radio wave sensor (1) is located above the region (R1), and the predetermined direction (height direction) is the height direction of the region (R1).

[0408] According to this aspect, when a radio wave sensor (1) transmits radio waves (Tr) from above a region (R1) and detects an object (301-305) using a reflected wave (Re) of the transmitted radio wave (Tr), the positional relationship (DH, H) between the ends of a distribution of multiple points (CL, CL1, CL2) in the height direction of the region (R1) is taken into consideration, thereby making it easier to identify the object (301-305).

[0409] According to the fourteenth embodiment, the radio wave sensor (1) transmits radio waves (transmitted waves Tr) in a predetermined direction (height direction), receives reflected waves (Re) of the radio waves (Tr), and uses a signal based on the reflected waves (Re), i.e., a received signal, to detect objects (301-305) within an area (R1). The radio wave sensor (1) includes a processing unit (22) for obtaining information related to the positions of the objects (301-305) within the area (R1) by performing ranging processing based on the received signal. The processing unit (22) obtains multiple position information corresponding to multiple parts constituting the objects (301-305), and determines the objects (301-305) based on the positional relationship (height difference DH, height H of the rectangular parallelepiped 502) between the two ends of the distribution (CL, CL1, CL2) of the multiple points corresponding to the multiple position information in the predetermined direction (height direction).

[0410] In this manner, when detecting an object (301-305) using a reflected wave (Re) of an electric wave (Tr) transmitted in a predetermined direction (height direction), the positional relationship (DH, H) between the ends of a distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction) is taken into consideration, thereby making it easier to identify the object (301-305).

[0411] The radio wave sensor (1) according to the fifteenth aspect is located above the region (R1) in the fourteenth aspect. The predetermined direction (height direction) is the height direction of the region (R1).

[0412] In this manner, when radio waves (Tr) are transmitted from above a region (R1) and the reflected waves (Re) of the transmitted radio waves (Tr) are used to detect objects (301-305), the positional relationship (DH, H) between the ends of the distribution of multiple points (CL, CL1, CL2) in the height direction of the region (R1) is taken into consideration, thereby making it easier to identify the objects (301-305).

[0413] According to a sixteenth embodiment, a detection method is provided for detecting objects (301-305) within a region (R1) using a signal, i.e., a received signal, based on a reflected wave (Re) of a radio wave (transmitted wave Tr) transmitted in a predetermined direction (height direction). The detection method includes processing steps (steps S8-S10 in the first embodiment and steps S8a-10a and steps S101-S111 in the second embodiment) for obtaining information related to the position of the objects (301-305) within the region (R1) by performing ranging processing based on the received signal. In the processing steps (S8-S10, S8a-10a, and S101-S111), multiple position information corresponding to multiple parts constituting the object (301-305) is obtained, and the object (301-305) is determined based on the positional relationship (height difference DH, height H of the rectangular parallelepiped 502) between the two ends in a predetermined direction (height direction) of the distribution of multiple points (CL, CL1, CL2) corresponding to the multiple position information.

[0414] In this manner, when detecting an object (301-305) using a reflected wave (Re) of an electric wave (Tr) transmitted in a predetermined direction (height direction), the positional relationship (DH, H) between the ends of a distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction) is taken into consideration, thereby making it easier to identify the object (301-305).

[0415] The program according to the seventeenth aspect is used to enable one or more processors to execute the detection method according to the sixteenth aspect.

[0416] In this manner, when detecting an object (301-305) using a reflected wave (Re) of an electric wave (Tr) transmitted in a predetermined direction (height direction), the positional relationship (DH, H) between the ends of a distribution of multiple points (CL, CL1, CL2) in the predetermined direction (height direction) is taken into consideration, thereby making it easier to identify the object (301-305).

[0417] 1: Radio wave sensor 2: Control device 2A: Motion detection device 2B: Equipment control device 11:Conversion Department 12: Preservation Department 13: Difference acquisition unit 14: Motion detection unit 21: Reception Department 22: Processing Department 23: Output 100: Detection system (equipment control system) 100A: Detection system (moving body detection system) 200: Equipment 200a: Lighting fixtures 200b: Television (TV) 200c: moving body (electric blinds) 221: Distance Measurement Department 222:Point configuration unit 223: Posture Estimation Unit 224: Action Detection Department 225: Equipment Control Department 226: Information Acquisition Department 227: History Record Storage 228: Learning Department 301: Moving body (human body) 302:Stationary 302a: Table 302b:Bed 303: Moving body (sweeping robot) 304: Moving body (electric fan) 305: Moving body (cat) 400: Room (real space) 401: Entrance and Exit 402: Window 500: Space 501: Point Group 502: 3D graphics CL, CL1, CL2: Cluster D: Length DH: Height Difference DW: distance Fr0~Fr5: frame H: Height Re:Reflected wave R1: Area S: slope S1~S17, S41~S45, S91~S98, S8a, S8b, S9a, S9b, S9c, S10a, S101~S109, S201~S206: Steps T: scheduled time Tc: Chirp length Tr~TrN: Transmitted waves (radio waves) t: time W: width c: speed of light d: distance f: frequency f0: starting frequency ΔA1~ΔA5: Inter-frame difference (time difference)

Claims

1. A detection system that detects an object in the room by receiving a signal, i.e., a signal, of a reflected wave of an electromagnetic wave emitted in a predetermined direction from an FMCW-type electromagnetic wave sensor installed on the ceiling of the room; the detection system includes: a processing unit that acquires information related to the position of the object in the room by performing ranging processing based on the received signal; the processing unit performs the following processing: acquiring multiple position information corresponding to multiple parts constituting the object; and determining the object based at least on the positional relationship between the two ends of multiple points distributed in the predetermined direction corresponding to the multiple position information.

2. The detection system as described in claim 1, wherein, This positional relationship includes the distance between the two ends.

3. The detection system as described in claim 1, wherein, The processing unit further determines the object based on the dispersion of the distribution on the plane with the predetermined direction as the normal direction.

4. The detection system as described in claim 3, wherein, The positional relationship includes: the distance between the two ends; and the dispersion of the distribution on the plane; the processing unit determines the type of the object based on the distance and the dispersion.

5. The detection system as described in claim 4, wherein, For each of the multiple individuals, personal information containing information related to the individual's body size is pre-stored; when the type has been determined to be the human body, the processing unit further determines the individual among the multiple individuals that corresponds to the human body based on the multiple personal information corresponding to the multiple individuals, the distance and the dispersion.

6. The detection system as described in claim 1, wherein, The radio wave is transmitted in each of a plurality of frames; the processing unit performs the following processing: calculating a plurality of time differences by repeatedly performing the processing of calculating the difference, i.e., the time difference, between the two received signals received in two different frames in the plurality of frames; obtaining the distribution of the point, i.e., one or a plurality of moving points, above the critical value of the time difference from the plurality of time differences; and determining the object based on at least one of the positional relationship of the distribution of the one or a plurality of moving points in a predetermined direction and the time change of the positional relationship, and the dispersion of the distribution on a plane with the predetermined direction as the normal direction and the time change of the dispersion.

7. The detection system as described in claim 6, wherein, The processing unit performs the following processing: Based on the time change of the dispersion of the distribution of one or more moving points on the plane, it determines the movement of the object.

8. The detection system as described in claim 6 or 7, wherein, The processing unit performs the following processing: Based on the time change of the positional relationship of the distribution of one or more moving points in the predetermined direction, it determines the movement of the object.

9. The detection system as described in claim 6, wherein, The processing unit performs the following processing: based on at least one of the dispersion of the distribution of one or more moving points on a plane with the predetermined direction as the normal direction and the time variation of the dispersion, it determines the object or the action of the object.

10. The detection system as described in claim 6 or 9, wherein, The processing unit performs the following processing: determining the object or determining the action of the object based on at least one of the positional relationship of the distribution of one or more moving points in the predetermined direction and the time change of the positional relationship.

11. The detection system as described in claim 6, wherein, The processing unit performs the calculation of the time difference using multiple distinct time intervals.

12. The detection system as described in claim 1, wherein, The detection system further includes the radio wave sensor; the radio wave sensor sends the radio wave in the predetermined direction and receives the reflected wave to output the received signal.

13. The detection system as described in claim 12, wherein, The predetermined direction is the height direction of the room; the processing unit determines the object based on the positional relationship between the two ends of the distribution of the plurality of points in the height direction and the dispersion on the plane perpendicular to the height direction.

14. An electromagnetic wave sensor, which is of the FMCW (Free-Mechanical Controlled Wave) type and is installed on the ceiling of an indoor space, transmits electromagnetic waves in a predetermined direction, receives reflected waves of the electromagnetic waves, and detects objects in the space using signals based on the reflected waves, i.e., received signals; the electromagnetic wave sensor includes: a processing unit that acquires information related to the position of the object in the space by performing ranging processing based on the received signals; the processing unit performs the following processing: acquiring multiple position information corresponding to multiple parts constituting the object; and determining the object based at least on the positional relationship between the two ends of multiple points corresponding to the multiple position information distributed in the predetermined direction.

15. The electromagnetic wave sensor as described in claim 14, wherein, The predetermined direction is the height direction of the room; the processing unit determines the object based on the positional relationship between the two ends of the distribution of the plurality of points in the height direction and the dispersion on the plane perpendicular to the height direction.

16. A detection method for detecting an object in an indoor space using a signal, i.e., a received signal, of a reflected wave of an electromagnetic wave emitted in a predetermined direction from an FMCW-type electromagnetic wave sensor installed on the ceiling of an indoor space; the detection method includes the following steps: a processing step, which obtains information related to the position of the object in the indoor space by performing ranging processing based on the received signal; the processing step performing the following processing: obtaining multiple position information corresponding to multiple parts constituting the object; and determining the object based at least on the positional relationship between the two ends of multiple points corresponding to the multiple position information distributed in the predetermined direction.

17. A computer program for causing one or more processors to execute the detection method as described in claim 16.

Images