Action recognition system

Abstract

To provide an action recognition system that accurately detects and classifies action information for an individual person in a dwelling space where multiple persons live.SOLUTION: An action recognition system 1000 includes: a sensor group containing a plurality of presence sensors 100 for sensing a person and a plurality of person recognition sensors 200 which senses a person and is capable of identifying a person; and an action estimation device 300 that recognizes an action of an identified person on the basis of sensing information received from the sensor group. The action estimation device 300 includes: a storage unit 350 that stores movement information related to a route through which a person moves from one region where at least one sensor in the sensor group is installed to another region where at least another sensor in the sensor group is installed; and a calculation unit 340 that performs time series analysis of information sensed by the sensor group, creates candidate information for a movement route through which the person to be recognized can move on the basis of a result of the analysis and the movement information, and estimates the action of the person identified by the plurality of human recognition sensors on the basis of the candidate information for the movement route and the information sensed by the plurality of human recognition sensors.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technology for a system that recognizes the behavior of people living in a living environment by combining detection information of sensors such as home appliances and human presence sensors provided in the living environment with a sensor that detects and identifies people.

Background Art

[0002] In the prior art that utilizes the operation information of home appliances and sensor information to control devices according to the state of users living in the living environment or to grasp the activity state of users, for example, there is Patent Document 1. Patent Document 1 discloses a remote care system for apartment houses that uses white goods for general households to implement and operate remote care. As a remote care system for apartment houses, a detection unit for detecting the behavior of residents existing in the living space is provided in home appliances installed in the living spaces that make up the apartment house, and based on the detection information of the detection unit provided in the home appliances, a technology for remotely controlling home appliances installed in a plurality of living spaces or remotely monitoring based on the detection information is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a system for confirming the safety of residents from a remote location, in a normal general household, since a plurality of residents share the living space, for the behavior information for determining the safety of the residents, it is required to detect and identify the individual behaviors of the residents. On the other hand, in order to individualize the behavior information of the residents, a plurality of sensors such as images and voices must be provided at various locations in the living space, which causes problems in terms of ensuring the privacy of the residents, introduction costs, operation costs, etc.

[0005] For example, in a two-family home, if a safety confirmation system is introduced that identifies behavioral information solely through motion sensors and appliance operation data, when multiple residents share a common space such as a dining room or living room, and then one resident remains in the common space while the others move to a different living space, the system will be unable to determine which resident's actions the motion sensor detection data originates from. Therefore, if a safety confirmation system that identifies behavioral information solely through motion sensors and appliance operation data is introduced in a living space shared by two or more people, it will be impossible to detect and identify the individual actions of the residents. On the other hand, if multiple image or sound sensors are installed in the living environment to solve the above problem, challenges arise in terms of ensuring privacy, installation costs, and operating costs.

[0006] This invention addresses the above-mentioned problems and aims to provide an action recognition system that can accurately detect and classify the actions of individuals in a living space where multiple people reside, even when the number of human recognition sensors installed is limited to ensure privacy. [Means for solving the problem]

[0007] To solve the above problems, the behavior recognition system of the present invention comprises a sensor group including a plurality of human presence sensors that detect people, a plurality of human recognition sensors that detect people and identify them, and a behavior estimation device that recognizes the behavior of a person identified by the human recognition sensors based on detection information received from the sensor group. The behavior estimation device comprises a storage unit that stores at least movement information relating to the path a person takes when moving from an area where at least one of the sensor group is installed to another area where at least one other of the sensor group is installed, and a calculation unit that analyzes the information detected by the sensor group in a time series, creates candidate movement path information that the person to be recognized can take based on the analysis results and the movement information, and recognizes the behavior of a person identified by the plurality of human recognition sensors based on the candidate movement path information and the information detected by the plurality of human recognition sensors. [Effects of the Invention]

[0008] According to the present invention, by connecting multiple motion sensors and human recognition sensors installed in a living environment with a behavior estimation device also installed in the living environment, it becomes possible to determine the behavior of individual residents living in the target living environment using an estimation unit that estimates the behavior of each resident. Furthermore, since human behavior information is created by utilizing detection information from motion sensors and home appliances, the number of sensors that recognize people can be reduced, thus providing a human behavior recognition system that takes user privacy into consideration. Further features related to the present invention will become apparent from the description herein and the accompanying drawings. Problems, configurations, and effects other than those described above will be revealed by the following description of the embodiments. [Brief explanation of the drawing]

[0009] [Figure 1] A schematic diagram showing the configuration of the behavior recognition system of the present invention. [Figure 2] A diagram illustrating an example of installing multiple sensors in a living environment. [Figure 3] A diagram illustrating the various types of information used in the behavior recognition system of the present invention. [Figure 4] A flowchart illustrating the procedure for processing information detected by sensors in the behavior recognition system of the present invention. [Figure 5] A diagram illustrating the behavior detection information and individual behavior detection information used in the behavior recognition system of the present invention. [Figure 6] A flowchart illustrating the process of identifying an individual's movement information based on information detected by sensors in the behavior recognition system of the present invention. [Figure 7] A diagram illustrating the information on potential travel routes. [Modes for carrying out the invention]

[0010] An embodiment of the present invention will be described below with reference to the drawings. Figure 1 shows an outline of the configuration of the behavior recognition system of the present invention. The behavior recognition system 1000 of the present invention includes a group of motion sensors 100 (inexpensive, simple motion sensors or home appliances) that detect people, a group of motion sensors 200 (sensors that can detect and identify people, such as image, sound, ToF, and acceleration sensors), and a behavior estimation device 300 that identifies and recognizes people and their actions based on the detection information from the group of motion sensors. In this embodiment, the behavior recognition system 1000 further includes a remote management unit 400 that receives information from the behavior estimation device 300 and can remotely check the safety and status of the detected resident.

[0011] Each of the multiple motion sensors 100 includes a power supply unit 110 for activating the motion sensors 100, a communication unit 120 for transmitting detection information and unique IDs of the motion sensors 100 to the behavior estimation device 300, a storage unit 130 for storing unique information of the motion sensors 100, a detection unit 140 for detecting people, and a control unit 150 for controlling the operation of the motion sensors 100.

[0012] Similarly, each of the multiple human recognition sensors 200 also includes a power supply unit 210 for activating the human recognition sensors 200, a communication unit 220 for transmitting detection information and unique IDs of the sensors to the behavior estimation device 300, a storage unit 230 for storing unique information of the human recognition sensors 200, a detection unit 240 for detecting and identifying people, and a control unit 250 for controlling the operation of the human recognition sensors 200.

[0013] Furthermore, the behavior estimation device 300 includes a communication unit 310, an external communication unit 320, a control unit 330, a calculation unit 340, a storage unit 350, a timer 360, and an external power supply 370. The behavior estimation device 300 is composed of a server or a PC, etc.

[0014] The communication unit 310 receives detection information and sensor unique IDs transmitted from a plurality of human presence sensors 100 and a plurality of human recognition sensors 200. The external communication unit 320 transmits and receives detection information and analysis information to and from an external network other than the sensor network constructed by the plurality of human presence sensors 100, the plurality of human recognition sensors 200, and the behavior estimation device 300. The control unit 330 analyzes information from the sensor network, external network, etc., and controls the device based on the analysis results. The arithmetic unit 340 differentiates the behaviors of individual residents and determines the safety of the residents based on the input information from the external communication unit 320, the control unit 330, the detection information of the plurality of human presence sensors 100 and the plurality of human recognition sensors 200, and the space movement information, residence information, etc. described later. The storage unit 350 stores the movement path information obtained by differentiating the behaviors of individual residents determined by the arithmetic unit 340. The timer 360 evaluates the time when the detection information of the plurality of human presence sensors 100 and the plurality of human recognition sensors 200 is received. Also, the remote management unit 400 can transmit and receive information regarding the safety of the residents analyzed by the behavior estimation device 300 via the external communication unit 320.

[0015] By constructing a behavior recognition system with the above configuration, the detection information of the residents and the unique IDs of the sensors detected by the plurality of human presence sensors 100 and the plurality of human recognition sensors 200 installed in the living environment can be transmitted to the behavior estimation device 300. The arithmetic unit 340 of the behavior estimation device 300 that has received the detection information and the sensor ID manages the reception time by the timer 360. When the sensor that transmitted the detection information is the human presence sensor 100, the arithmetic unit 340 can determine the room (the location where the behavior occurred) and the time when the sensor reacted from the sensor ID and the sensor installation information, and store it in the storage unit 350 as behavior detection information.

[0016] Similarly, the arithmetic unit 340 recognizes the type of the sensor from the unique ID of the sensor. When the sensor that transmitted the detection information is the human recognition sensor 200, the arithmetic unit 340 can identify the resident to be detected by comparing with the resident information, and store it in the storage unit 350 as personal detection information.

[0017] The calculation unit 340 identifies the movement information of an individual by analyzing the individual's movement route using the personal detection information and behavior detection information stored in the storage unit 350, the residential information, and the spatial movement information. Then, the storage unit 350 can store the identified movement routes of the individuals. By monitoring the stored movement routes, if detection information that deviates from the daily movement information of an individual is detected, it can be determined that something abnormal has occurred to the individual being detected, and the abnormal information can be transmitted to the remote management unit 400 through the external communication unit 320.

[0018] FIG. 2 is a diagram showing an overview when the behavior recognition system 1000 having the configuration as shown in FIG. 1 is introduced into a living environment including a residential space 2000. The human presence sensor 100 is composed of a human presence sensor for detecting the presence or absence of a person, a door opening and closing sensor for detecting the opening and closing of a door, an illuminance sensor for detecting the brightness of a room, and the like. These sensors simply detect the presence or absence of a person, etc., without identifying personal information. Therefore, in order not to invade the privacy of the residents living in the detection target, there is no restriction on the installation location within the residential space 2000. In FIG. 2, the human presence sensor 100 is installed in the living room L and the dining room D.

[0019] On the other hand, the person recognition sensor 200 includes sensors that acquire personal information such as images, voices, ToF, etc. and identify individuals, sensors that measure the acceleration of door opening and closing, and the like. Since these sensors detect information including the privacy of residents, it is necessary to consider privacy when installing them. For example, it is preferably installed in a place with relatively high sharing, such as the entrance or corridor. In FIG. 2, the person recognition sensor 200 is installed at the entrance E, etc.

[0020] Although the robot vacuum cleaner 500 is illustrated in FIG. 2 as a representative of home appliances, home appliances that handle image information such as a robot vacuum cleaner can be utilized as the person recognition sensor 200. Also, when using the sensors and operation information of a refrigerator, a washing machine, an air conditioner, etc. as the detection information of the sensors, these home appliances can be utilized as the human presence sensor 100. The behavior estimation device 300 and the remote management unit 400 are not shown in the figure.

[0021] Figure 2 is an example illustrating the schematic method for installing the motion sensor 100 and the human recognition sensor 200 in the living space 2000 according to the present invention, and the method for installing the motion sensor 100 and the human recognition sensor 200 according to the present invention is not limited to Figure 2.

[0022] The procedure for detecting and recognizing the individual behavioral information of residents to be detected in the behavioral recognition system 1000 configured as described above will be explained using Figures 3 to 7.

[0023] First, the various types of information used in the behavior recognition system 1000 according to the present invention will be explained. Multiple motion sensors 100 and multiple human recognition sensors 200 are installed in the living space 2000 to be detected, and basic information such as residents, houses, and sensors is pre-registered in the storage unit 350 of the behavior estimation device 300. Examples of information to be registered include resident information, room information, sensor installation information, residence information, spatial movement information, etc. An example of this information will be explained using Figure 3.

[0024] As shown in Figure 3, resident information is constructed from resident names, personal IDs to distinguish residents from each other, and feature data related to resident features used to identify individuals by the human recognition sensor 200. These features include, for example, height, build, posture, gait, dominant hand, and information detectable by various sensors such as images and sounds. Room information is constructed from room names and room IDs. Sensor installation information manages the installation details of the human motion sensor 100 and human recognition sensor 200, and is constructed from the sensor ID, the room ID in which it was installed, and the type of sensor (0: human recognition sensor, 1: non-human recognition sensor).

[0025] Resident information is a database of information about the spaces where the detected resident is normally present or absent, and consists of a personal ID, characteristic date and time information (for example, day of the week or weekend; in this embodiment, it is divided into WD (weekday) / WE (weekend), but the division is not limited to this embodiment), time of day (for example, in this embodiment, it is divided into 0: morning (4am to 10am), 1: noon (10am to 4pm), 2: evening (4pm to 10pm), 3: late night (10pm to 4am), but the division method is not limited to this embodiment), and space utilization status (divided into 0: present / 1: absent / -: unknown, but the division is not limited to this embodiment).

[0026] Spatial movement information (movement information) is information that shows a list of ON and OFF sensor ID sequences of multiple sensors that react when a person moves from one room to another. As described later, the movement path can be estimated by comparing the spatial movement information with the sensor detection information.

[0027] The basic information described above is registered in the memory unit 350 of the behavior estimation device 300, and the behavior recognition system 1000 is activated. Figure 4 is a flowchart showing the process from when the behavior recognition system 1000 receives a detection signal from the human presence sensor 100 or human recognition sensor 200, until it stores the information in the memory unit 350 as behavior detection information or personal detection information.

[0028] When the behavior recognition system 1000 is activated, it first proceeds to operation step S100. In operation step S100, the behavior recognition system 1000 determines whether either the motion sensor 100 or the motion recognition sensor 200 has detected the resident's actions. If either the motion sensor 100 or the motion recognition sensor 200 has detected the resident's actions and the sensor has responded, the operation steps proceed to S110, and the system proceeds to the process of selecting / identifying the sensor information.

[0029] If the sensor does not respond, the operation step moves to S200 to determine if the operating time (T) since the start of sensor response detection has exceeded a predetermined elapsed time (Tan). The predetermined elapsed time (Tan) can be set to, for example, 6 hours. If the operating time (T) ≥ elapsed time (Tan) in operation step S200, the operation step moves to S210 in Figure 6 to proceed to the process of estimating the moving person. This process will be described later. If the operating time (T) < elapsed time (Tan), the elapsed time is added by T + ΔT, and then the operation step returns to S100.

[0030] Having moved to operation step S110, the action recognition system 1000 proceeds to the process of selecting sensor information and accumulating detection data according to the sensor information. In operation step S110, the action recognition system 1000 reads the sensor installation information and then proceeds to operation step S120.

[0031] In operation step S120, the behavior recognition system 1000 compares the detection sensor ID obtained from the detection information received from the human presence sensor 100 or the human presence sensor 200 with the sensor installation information to determine whether the sensor that transmitted the detection information is the human presence sensor 200 or not (human presence sensor 100). In operation step S120, if it is determined that the sensor transmitting the detection information is not the human presence sensor 200, i.e., it is the human presence sensor 100, the behavior recognition system 1000 moves the operation step to S121, creates behavior detection information, moves the operation step to S160, and saves the behavior detection information in the storage unit 350 of the behavior estimation device 300.

[0032] Here, the behavior detection information is a data sequence consisting of a room ID and a detection time, as shown in Figure 5. In other words, the behavior detection information does not contain information that can identify a person, but it contains information about the location and time when the sensor reacted. If, in operation step S120, the sensor transmitting the detection information is determined to be the person recognition sensor 200, the behavior recognition system 1000 proceeds to operation step S130.

[0033] After the behavior recognition system 1000 has moved to operation step S130, it reads resident information from the memory unit 350 of the behavior estimation device 300, and then moves to operation step S140.

[0034] In the operation step S140, the behavior recognition system 1000 compares the detection information received from the person recognition sensor 200 with the individual characteristic data registered in the resident information. If no individual characteristic data matches the detection information, the behavior recognition system moves to the operation step S121 and stores the behavior detection information in the storage unit 350. If individual characteristic data matches the detection information, the behavior recognition system 1000 moves to the operation step S150.

[0035] The action recognition system 1000, having moved the operation step to S150, determines that the detected information is information about an individual's actions and creates individual detection information and action detection information. As shown in Figure 5, the individual detection information is composed of a data sequence consisting of room ID, individual ID, and detection time. In other words, individual detection information includes information about the identified individual in addition to the action detection information.

[0036] The behavior recognition system 1000 creates individual detection information and behavior detection information in operation step S150, or after creating behavior detection information in operation step S121, it moves the operation step to S160. After storing the behavior detection information and individual detection information in the storage unit 350 of the behavior estimation device 300, it moves the operation step to S200 again. When the operating time (T) ≥ elapsed time (Tan), it moves the operation step to S210 and proceeds to the process of estimating movement information and identifying the individual held by the movement information, as shown in Figure 6, which is the process of estimating movement information and identifying the individual held by the movement information.

[0037] In operation step S210, the behavior recognition system 1000 reads spatial movement information from the memory unit 350, then compares the behavior detection information and spatial movement information, which are divided into analysis time widths w, to create inter-room movement information. Here, inter-room movement information is information about the rooms a person moved to (rooms where the reacting sensors were installed) during a certain time period included in the behavior detection information. Specifically, for example, using the behavior detection information in Figure 5 as an example, a person moved from room R1 to R2 between time t1 and time t2. Referring to this and the spatial movement information in Figure 3, it can be seen that there are two routes for movement from room R1 to room R2 as a sequence of reacting sensors: S1→S2 and S1→S6→S5→S2. In other words, inter-room movement information is information about these two routes. This inter-room movement information is calculated according to the detection times included in the behavior detection information.

[0038] The created inter-room movement information is combined to create candidate movement path information, and the operation step is moved to S220. As shown in Figure 7, the candidate movement path information is information that connects the inter-room movement information that occurred within the elapsed time (Tan), and each candidate movement path information is information arranged in chronological order by pairing the room ID to which the movement was moved and the time when the movement was completed.

[0039] The behavior recognition system, having moved to operation step S220, reads residence information from the memory unit 350, and then, using the personal detection information and the presence / absence / unknown information for each room ID included in the residence information, determines the weight coefficient (w) of the room ID and time indicated by each candidate movement route information in operation steps S235 to S265.

[0040] Here, the weight coefficient (w) is an index that represents the likelihood that the proposed travel path is indeed a travel path taken by a person identified by their personal ID, and is calculated by the following flow.

[0041] The behavior recognition system 1000, having moved to operation step S230, determines whether there is personal detection information that includes a room ID and detection time that matches the room ID and inter-room travel time shown in the travel route candidate information. Here, inter-room travel time refers to, for example, the time t1-t2 for the movement from R5 to R2 in travel route candidate information #1 in Figure 7. This is compared with the personal detection information to determine whether there is detection information indicating that the person was in room R2 between times t1-t2. This process is performed for all route candidates.

[0042] If it is determined in S230 that there is personal detection information that matches the candidate movement path information, the behavior recognition system 1000 proceeds to operation step S235, sets the weight coefficient to +1, and proceeds to operation step S270. If there is no matching personal detection information in S230, the behavior recognition system proceeds to operation step S240.

[0043] In the operation step S240, the behavior recognition system 1000 determines whether there is a room ID in the residence information that matches the room ID and inter-room travel time shown in the travel route candidate information and whether there is a room ID that matches "occupied" in the residence information. If there is a room ID that matches "occupied" in the residence information DB, the behavior recognition system 1000 moves the operation step to S245, sets the weight coefficient to +1, and moves the operation step to S270. For example, in the travel route candidate information #1 in Figure 7, room R2 at time t2 is listed as a candidate. Referring to the residence information in Figure 3, for example, if this time t2 is Day:WD and TimeZone:2, then the person identified by personal ID:P1 would be the person in question, and this person's action is weighted +1 as a candidate. If there is no room ID that matches "occupied" in the residence information in operation step S240, the behavior recognition system moves the operation step to S250.

[0044] In the operation step S250, the behavior recognition system 1000 determines whether there is a room ID in the residence information DB that matches the room ID and inter-room travel time shown in the candidate travel route information and matches "absence". If there is a room ID that matches "absence" in the residence information DB, the behavior recognition system 1000 moves the operation step to S255, sets the weight coefficient to -1, and moves the operation step to S270. For example, in the residence information in Figure 3, if there is sensor detection information for room ID:R1 in Day:WD, TimeZone:1, the person identified by personal ID:P1 is considered "absent" during this time period, and therefore the possibility of this person being a candidate route is low, so a weight of -1 is assigned. If there is no room ID that matches "absence" in the residence information DB, the behavior recognition system 1000 moves the operation step to S265, sets the weight coefficient to 0, and moves the operation step to S270.

[0045] The action recognition system 1000, having moved to operation step S270, calculates an occupancy score using the occupancy time and weight coefficient for each room ID of each candidate movement path, and moves the operation step to S280. Note that the occupancy score S i w(RID) is the weight coefficient for each room. k ) and the time spent in the room (T k+1 -T k It is obtained by multiplying ) by the value obtained and adding it from the start time of measurement to the end time of measurement.

[0046] The action recognition system 1000, having moved to operation step S280, determines whether calculations have been performed for all candidate movement paths. If the calculation of all occupancy scores is complete, it moves to operation step S290. If the calculation of all occupancy scores is not complete, it moves to operation step S220.

[0047] The behavior recognition system 1000, having moved to operation step S290, identifies the movement route candidate with the highest occupancy score among the results of the weighting calculation for each movement route candidate for the individual ID being calculated as that individual's movement route, and moves to operation step S300. The behavior recognition system 1000, having moved to operation step S300, determines whether or not it has determined the movement routes of all residents in the detected living environment, and if it has determined the movement routes of all residents, moves to operation step S310.

[0048] If the movement routes of all residents have not been determined, the operation step proceeds to S220. The behavior recognition system 1000, having proceeded to operation step S310, saves the movement information of the detected individual to the memory unit 350 of the behavior estimation device 300, resets the operating time (T) to 0, and then proceeds to operation step S100.

[0049] The embodiments of the present invention described above provide the following effects. (1) An action recognition system according to one embodiment of the present invention comprises a sensor group including a plurality of human presence sensors that detect a person, and a plurality of human recognition sensors that detect a person and identify the person, and an action estimation device that recognizes the actions of a person identified by the human recognition sensors based on detection information received from the sensor group, wherein the action estimation device comprises a storage unit that stores at least movement information relating to a path a person takes when moving from an area where at least one of the sensor group is installed to another area where at least one other of the sensor group is installed, and a calculation unit that analyzes the information detected by the sensor group in a time series, creates candidate movement path information that the person to be recognized can take based on the results of the analysis and the movement information, and recognizes the actions of a person identified by the plurality of human recognition sensors based on the candidate movement path information and the information detected by the plurality of human recognition sensors.

[0050] With the above configuration, by connecting multiple sensors and motion sensors installed in the living environment with a behavior estimation device installed in the living environment, it becomes possible to determine the behavior of individual residents living in the target living environment using an estimation unit that estimates the behavior of each resident. Furthermore, by utilizing detection information from motion sensors and home appliances to create human behavior information, the number of sensors that recognize people can be reduced, thus providing a human behavior recognition system that takes user privacy into consideration.

[0051] (2) The behavior estimation device further includes an external communication unit that transmits information about human behavior to an area outside the area where the sensor group is installed via a network. This makes it possible to send an alert to an external monitoring device if an abnormality is detected in the resident's movement information.

[0052] (3) The memory unit further stores person information related to features for identifying the person being detected, and the person recognition sensor identifies the person by comparing the detected content with the person information. This allows for the use of information detectable by various sensors such as height, build, posture, gait, dominant hand, images, and sounds, enabling identification of individuals from various angles and improving the accuracy of identification.

[0053] (4) The memory unit further stores records of the actions of the person to be detected within the area where the sensor group is installed, and the calculation unit recognizes the person's actions based on the information detected by the multiple person recognition sensors, candidate movement path information, and the action records. This makes it possible to recognize a person's actions based on pre-stored action records (for example, which room belongs to whom, etc.) even if a person cannot be identified based on the person's features.

[0054] (5) Before referring to the action record, the calculation unit refers to the information detected by multiple person recognition sensors to recognize a person's actions. This prioritizes the processing to be performed, which is expected to reduce the load on calculation processing and speed it up.

[0055] (6) The calculation unit, when it is able to recognize a person's behavior by referring to the information detected by multiple person recognition sensors, assigns a positive weight and does not refer to the behavior record. When it is not able to recognize a person's behavior by referring to the information detected by multiple person recognition sensors, it refers to the behavior record without assigning a weight and recognizes the person's behavior. This makes it possible to quantitatively recognize a person's behavior based on the detected information, improving recognition accuracy.

[0056] (7) The calculation unit refers to the action record and assigns a positive weight if it can recognize a person's action, and a negative weight if it cannot recognize a person's action. This makes it possible to quantitatively recognize actions, similar to (6), and furthermore, by assigning a negative weight, it is possible to eliminate information due to false detections by the sensor.

[0057] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are possible. For example, the embodiments described above are explained in detail to make the present invention easier to understand, and the present invention is not necessarily limited to embodiments having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment. It is also possible to add configurations from other embodiments to the configuration of one embodiment. Furthermore, it is possible to delete parts of the configuration of each embodiment, or to add or replace other configurations. [Explanation of symbols]

[0058] 100 Human presence sensor, 200 Human recognition sensor, 300 Action estimation device, 320 External communication unit, 340 Calculation unit, 350 Memory unit

Claims

1. An activity recognition system that recognizes human activity, a sensor group including a plurality of human presence sensors that detect people and a plurality of human recognition sensors that detect people and identify the people; a behavior estimation device that recognizes the behavior of the person identified by the human recognition sensor based on the detection information received from the group of sensors, the behavior estimation device, a storage unit that stores at least movement information regarding a route along which a person moves from an area where at least one of the sensors is installed to another area where at least another of the sensors is installed; a calculation unit that performs time series analysis on information detected by the group of sensors, creates candidate movement route information that the person to be recognized may take based on the result of the analysis and the movement information, and recognizes the behavior of the person identified by the plurality of human recognition sensors based on the candidate movement route information and information detected by the plurality of human recognition sensors; An activity recognition system characterized by:

2. The activity recognition system according to claim 1 , the behavior estimation device further includes an external communication unit that transmits information about the person's behavior to an area outside where the sensor group is installed via a network. An activity recognition system characterized by:

3. The activity recognition system according to claim 1 , the storage unit further stores person information relating to feature amounts for identifying a person to be detected, The human recognition sensor identifies the person by comparing the detection content with the person information. An activity recognition system characterized by:

4. The activity recognition system according to claim 1 , the storage unit further stores a record of the person's behavior within the area where the group of sensors is installed, the calculation unit recognizes the behavior of the person based on the behavior record in addition to the information detected by the plurality of human recognition sensors and the travel route candidate information; An activity recognition system characterized by:

5. The behavior recognition system according to claim 4, the calculation unit recognizes the behavior of the person by referring to information detected by the plurality of human recognition sensors before referring to the behavior record. An activity recognition system characterized by:

6. The behavior recognition system according to claim 5, the calculation unit, when it is possible to recognize the person's behavior by referring to the information detected by the plurality of human recognition sensors, assigns a positive weight to the person without referring to the behavior record, and when it is not possible to recognize the person's behavior by referring to the information detected by the plurality of human recognition sensors, it assigns a positive weight to the person without referring to the behavior record and recognizes the person's behavior. An activity recognition system characterized by:

7. The behavior recognition system according to claim 6, the calculation unit performs a positive weighting when the person's behavior can be recognized by referring to the behavior record, and performs a negative weighting when the person's behavior cannot be recognized. An activity recognition system characterized by:

Images