Interior monitoring system, interior monitoring method, and interior monitoring program

Abstract

To provide an interior monitoring system, interior monitoring method, and interior monitoring program capable of detecting an intruder even in the event that there is a spot incapable of being caught with a monitoring camera in an interior or a monitoring camera has failed, and detecting in an early stage a case that an entering person gets sick or injured and needs to be rescued.SOLUTION: An interior monitoring system includes a monitoring sensor 100 including at least one of an air flow sensor 6 that detects a change in an air flow in an interior, an odor sensor 7 that detects an odor in the interior, a floor pressure sensor 8 that detects a pressure applied to the floor of the interior, and an acoustic sensitivity sensor 9 that detects a sound in the interior. The interior monitoring system includes a learning model 11 that has learned a correlation between learning data including data, which is detected by the monitoring sensor 100, as input data, and the propriety or condition of an entering person of the interior. The input data including data detected by the monitoring sensor 100 is acquired and inputted to the learning model 11 in order to estimate the propriety or condition of an entering person in the interior.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a room monitoring system, a room monitoring method, and a room monitoring program that can be used to monitor a room such as a high-security room. [Background technology]

[0002] Only those who have registered in advance can enter the high-security room, and surveillance cameras are installed in the room to monitor for trespassers. However, surveillance cameras are not used 24 hours a day, and since there are equipment and terminals such as racks and computers placed inside the room, there are quite a few blind spots for the surveillance cameras. Therefore, as shown in Patent Document 1, a surveillance camera system has been devised in which multiple surveillance cameras are installed side by side, and when a monitored object (such as an intruder) enters the blind spot of a surveillance camera, the rotation of the other surveillance cameras is controlled to track the monitored object (such as an intruder). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-055180 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the problem of "blind spots of surveillance cameras" can be solved by increasing the number of surveillance cameras and installing them continuously. However, as mentioned above, surveillance cameras do not provide constant monitoring, so this is not necessarily sufficient in terms of security, and it also makes it difficult to respond to sudden events such as the appearance of an injured person.

[0005] The present invention has been made in consideration of the above circumstances, and its main objective is to provide an indoor monitoring system, an indoor monitoring method, and an indoor monitoring program that can accurately detect intruders even when there are areas in the room that cannot be captured by a surveillance camera, and that can quickly detect if an entrant becomes injured or ill and requires first aid. [Means for solving the problem]

[0006] In order to achieve the above object, the indoor monitoring system is a system for monitoring an indoor space, a monitoring sensor including at least one of an airflow sensor that detects changes in the airflow in the room, an odor sensor that detects odors in the room, a floor pressure sensor that detects pressure on the floor surface in the room, and a sound sensor that detects sounds in the room; a learning model storage unit that stores a learning model that has learned a correlation between learning data including data detected by the monitoring sensor as input data and the suitability or status of a person entering the room; an input data acquisition unit that acquires input data including data detected by the monitoring sensor; an inference unit that inputs the input data acquired by the input data acquisition unit into the learning model and infers the suitability or status of the person entering the room; The present invention is characterized by the following:

[0007] Therefore, using a learning model that has learned the correlation between input data detected by a monitoring sensor consisting of at least one of an airflow sensor, an odor sensor, a floor pressure sensor, and a sound sensor, and the suitability or status of a person occupying the room, the inference unit infers the suitability or status of a person occupying the room from input data including data detected by the monitoring sensor acquired by the input data acquisition unit.Therefore, even if the person occupying the room is in a blind spot of the monitoring camera installed to monitor the room, or if the monitoring camera is malfunctioning, it is possible to accurately grasp the suitability or status of the person occupying the room.

[0008] Here, the legitimacy of the entrant may be determined to be whether or not the entrant is an intruder, and the condition of the entrant may be determined to be whether or not the entrant is injured or ill. This makes it possible to confirm the presence of an intruder even if they are in a blind spot of the surveillance camera or are disguised, and also makes it possible to know if the entrant is injured or ill in a position that cannot be captured by the camera.

[0009] The learning model used to confirm the suitability of the entrant (whether or not there is an intruder) and the condition of the entrant (whether or not the entrant has developed an injury or illness) may be trained using supervised learning or unsupervised learning.

[0010] That is, in the former case, the learning data is associated with the input data and further includes, as output data, information indicating whether the person entering the room is suitable or not or whether the person is in one of a plurality of states; The learning model may be configured to learn the correlation between input data and output data through supervised learning. In the latter case, the learning data includes only the input data when it indicates whether the person entering the room is suitable or not or when the person is in a predetermined state, The learning model may be configured to learn the correlation between the input data and information indicating whether the person entering the room is suitable or in a predetermined state through unsupervised learning. In addition, when a monitoring camera is installed in the room, it is preferable that the monitoring sensor is mainly installed in a position that is in the blind spot of the monitoring camera. [Effects of the Invention]

[0011] As described above, according to the present invention, in a room requiring indoor monitoring, such as a high-security room, even if there are blind spots that cannot be captured by a surveillance camera or if the installed surveillance camera breaks down, if there is an entrant in the room, the suitability and condition of the entrant can be determined using a learning model based on data detected by the surveillance sensor, thereby further strengthening indoor security and enabling early detection of injuries or illnesses of entrants and immediate appropriate action to be taken. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic configuration diagram showing an indoor monitoring system according to the present invention; [Figure 2] FIG. 1 is a block diagram illustrating an example of a machine learning device for an indoor monitoring system according to the present invention. [Figure 3] 1 is a flowchart showing an example of a machine learning method performed by a machine learning device according to the present invention, illustrating an example of supervised learning. [Figure 4] 1 is a flowchart showing an example of a machine learning method by the machine learning device according to the present invention, illustrating an example of unsupervised learning. [Figure 5] FIG. 1 is a block diagram illustrating an example of an entrant estimation device. [Figure 6] 10 is a flowchart illustrating an example of a room monitoring method using an entrant estimation device. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. 1 shows an example in which an indoor monitoring system 1 according to the present invention is applied to, for example, a high-security room equipped with indoor monitoring cameras. In this example, monitoring cameras 2 are installed high in the room, one on each of the room's opposing walls, but there is a large installation 3 such as a rack or large computer in the middle of the room, and the area behind this installation 3 (area α indicated by a hatch) is a blind spot for the monitoring cameras 2. In this example, an air current is formed behind the installation 3, with conditioned air blown out from an air conditioner 4 moving toward an exhaust duct 5.

[0014] As shown in the figure, this indoor monitoring system 1 comprises an airflow sensor 6 (airflow sensor 6a used in the learning phase, airflow sensor 6b used in the estimation phase) that detects changes in airflow in the room, an odor sensor 7 (odor sensor 7a used in the learning phase, odor sensor 7b used in the estimation phase) that detects odors in the room, a floor pressure sensor 8 (floor pressure sensor 8a used in the learning phase, floor pressure sensor 8b used in the estimation phase) that detects pressure on the floor surface in the room, a sound sensor 9 (sound sensor 9a used in the learning phase, sound sensor 9b used in the estimation phase) that detects sounds in the room, a machine learning device 10, an entrant estimation device 20, and a display 30.

[0015] The airflow sensor 6, odor sensor 7, floor pressure sensor 8, and sound sensor 9 function as surveillance sensors 100, and having at least one of them can reinforce surveillance in area α, which is a blind spot for the surveillance camera 2. However, by combining each sensor, the reinforcing effect of surveillance can be increased, and it is most preferable to install all four types of sensors mentioned above.

[0016] The airflow sensor 6 detects changes in the airflow from the air conditioner 4 to the exhaust duct 5, particularly changes in the airflow that occur behind the installed object 3, and can thereby detect the presence of a person entering the area a that is a blind spot of the surveillance camera 2 and capture the trajectory of the person's movement. For this reason, it is desirable to install the airflow sensor 6 in a position where changes in the airflow that occur behind the installed object 3 can be easily detected. One or more airflow sensors 6 may be installed. In this example, the airflow sensors 6 are installed on the wall on which the exhaust duct 5 is installed, in front of the exhaust outlet of the exhaust duct 5, and on the back of the installed object 3, near the floor, in front and behind the airflow direction (on both sides of the back of the installed object 3). Here, the airflow sensor 6a used in the learning phase and the airflow sensor 6b used in the estimation phase may be the same or different.

[0017] The odor sensor 7 detects odors behind the installation 3 to detect the presence of a person entering area a, which is a blind spot of the surveillance camera 2. It is desirable to install the odor sensor 7 in a position where it is easy to detect odors behind the installation 3, and one or more sensors may be installed. In this example, the odor sensor 7 is installed in the center near the floor on the back side of the installation 3. Here, the odor sensor 7a used in the learning phase and the odor sensor 7b used in the estimation phase may be the same or different.

[0018] The floor pressure sensor 8 detects changes in pressure on the floor that occur behind the installed object 3, in order to detect the presence of an entrant in area a, which is a blind spot of the surveillance camera 2, and to capture the entrant's movement trajectory. While it is conceivable to install sensors throughout the room, installing sensors throughout the area that can be captured by the surveillance camera 2 would be costly, so in this example, the sensor is installed mainly on the floor of area a, which is a blind spot of the surveillance camera 2 behind the installed object 3. Here, the floor pressure sensor 8a used in the learning phase and the floor pressure sensor 8b used in the estimation phase may be the same or different.

[0019] The sound sensor 9 is intended to detect the presence of a person entering area a, which is a blind spot of the surveillance camera 2, by detecting sound behind the installed object 3. It is desirable to install it in a position where it is easy to detect sound behind the installed object 3, and one or more sound sensors may be installed, and in this example, it is installed approximately in the center of the back surface of the installed object 3. Here, the sound sensor 9a used in the learning phase and the sound sensor 9b used in the estimation phase may be the same or different.

[0020] The machine learning device 10 operates as the main subject of the learning phase, and generates a learning model 11 by machine learning, which is used to estimate the suitability or status of a person entering a room, from data obtained from the monitoring sensors 100 (in this example, four sensors: an airflow sensor 6, an odor sensor 7, a floor pressure sensor 8, and a sound sensor 9). Either "supervised learning" or "unsupervised learning" can be used for machine learning; "supervised learning" is used when learning is performed by comparison with supervised data, and "unsupervised learning" is used when there is no particular supervised data and steady-state (normal) conditions are learned.

[0021] The entrant estimation device 20 operates as the main body of the estimation phase, and uses the trained learning model 11 generated by the machine learning device 10 to estimate the suitability or status of the entrant from the data obtained by the monitoring sensor 100.

[0022] The display device 30 can be replaced by the display screen of various terminals; for example, the monitor of the worker terminal 31 described below can be used to display the suitability and status of the entrant estimated by the entrant estimation device 20.

[0023] (machine learning device) The machine learning device 10 is configured by a general-purpose or dedicated computer. This computer may be a desktop computer or a portable computer, and may also be a client computer, a server computer, or a cloud computer. As shown in FIG. 2, the machine learning device 10 includes a learning data acquisition unit 12, a learning data storage unit 13, a machine learning unit 14, and a learning model storage unit 15.

[0024] The learning data acquisition unit 12 functions as an interface unit that acquires learning data including input data via a communication network 32. This learning data acquisition unit 12 is connected via the communication network 32 to the monitoring sensors 100 (airflow sensor 6, odor sensor 7, floor pressure sensor 8, sound sensor 9), worker terminals 31 used by workers, and the like.

[0025] The learning data storage unit 13 is a database that stores a plurality of sets of learning data acquired by the learning data acquisition unit 12.

[0026] The machine learning unit 14 performs machine learning using the learning data stored in the learning data storage unit 13. The machine learning unit 14 inputs multiple sets of learning data into the learning model 11, and causes the learning model 11 to learn the correlation between the input data included in the learning data and the judgment information indicating the suitability or status of the entrant, thereby generating a trained model.

[0027] There are various methods for generating trained models, but it is best to use both "supervised learning" and "unsupervised learning" as machine learning methods depending on the training data.

[0028] First, when adopting "supervised learning" as machine learning, it is advisable to adopt, for example, a neural network as a specific method.

[0029] The learning model storage unit 15 is a database that stores the learned learning model 11 generated by the machine learning unit 14. The learning model 11 stored in the learning model storage unit 15 is provided to the entrant estimation device 20 (described later) via any communication network or storage medium. The learning data storage unit 13 and the learning model storage unit 15 may be configured as a single storage device or as separate storage devices.

[0030] The learning data includes, as input data, at least data detected by the monitoring sensor 100. The input data may include other data as well, but may also include data that can collect information about area α, which is a blind spot of the monitoring camera 2, and that correlates with the suitability and status of the person entering the room.

[0031] In the case of "supervised learning," the learning data further includes, as output data (teacher data) associated with the input data, determination information indicating whether the person entering the room is a person who has been registered in advance. The determination information is information that characterizes a person who has been registered in advance, and is at least one of the registered person's movements in the room, the odor emitted by the registered person, the pressure on the floor when the registered person walks, the rhythm of the person's walking, the sound of the person walking, etc. When diagnosing the presence or absence of an abnormal condition as the suitability of the person entering the room (whether or not the person is registered, i.e., whether or not the person is an intruder) or the condition of the person entering the room (whether or not the person is injured or ill), the determination information may be configured as information that indicates whether the person entering the room is normal or abnormal. In this case, the determination information may be defined as "0" if the person entering the room is a registered person and has not developed an injury or illness, and as "1" if the person entering the room is a person other than a registered person (an intruder) or is injured or ill.

[0032] Here, we will explain the correlation between the input data contained in the learning data (data detected by the monitoring sensors 100, i.e., data detected by the airflow sensor 6, data detected by the odor sensor 7, data detected by the floor pressure sensor 8, and data detected by the sound sensor 9) and the information determining the suitability and status of the person entering the room.

[0033] (Correlation between airflow sensor detection data and people entering the room) A registered person can move freely around the room without worrying about the surveillance cameras, and if they have a set routine, they will move accordingly, and the airflow will change accordingly to track the registered person's movements. However, an intruder will be concerned about the surveillance cameras and will hide behind objects (installation 3), so by learning the airflow changes associated with such movements in advance, it is possible to identify the entrant as an intruder. Also, if an entrant becomes injured or falls or sits down, the airflow from the air conditioning will change and become constant. Therefore, by learning the airflow in advance when the entrant falls or sits down, it is possible to estimate the presence or absence of an injured or ill person. Furthermore, by learning how a registered person's normal breathing affects the airflow, it is possible to estimate the presence or absence of an intruder or an injured or ill person. Therefore, the detected data from the airflow sensor can be used to monitor for the presence or absence of intruders or injured people based on the presence or absence of changes in the airflow and changes in the airflow over time.

[0034] (Correlation between odor sensor detection data and people entering the room) By having the system learn beforehand the body odor and other odors emitted by registered individuals, it becomes possible to determine whether or not the person entering the room is someone other than a registered individual. Also, if the injured or sick person is bleeding, the smell of iron in the blood will be detected, and if the injured or sick person is sweating abnormally, the smell of that sweat will be detected, so by having the system learn beforehand the smells of blood and sweat, it becomes possible to determine whether or not there is an injured or sick person. Therefore, the data detected by the odor sensor can be used to detect odors that are different from registered odors, or to monitor for the presence or absence of trespassers or injured or sick people by detecting changes in odors over time.

[0035] (Correlation between floor pressure sensor detection data and occupants) The detection data obtained from the pressure sensor on the floor according to the registered person's walking style and weight is data specific to the registered person. Therefore, if a person other than the registered person enters the room, the load on the floor and walking style will be different, and the detection data from the pressure sensor will be different. Also, if the injured person falls to the floor or sits down and does not move, the corresponding sensor output will continue, making it possible to detect an abnormality. In addition, detecting the pressure-sensitive area when the injured person falls or sits down can also make it possible to estimate the onset of the injury or illness. Therefore, the detection data from the floor pressure sensor can be used to determine whether the movement is that of a registered person, an intruder, or whether the person entering the room is injured or ill, based on the detection of different detection data, pressure sensing time, pressure sensing area, etc.

[0036] (Correlation between sound sensor detection data and people entering the room) By registering the sound of a registrant's breathing, walking rhythm, etc. in advance, the detection data obtained from the sound sensor becomes data unique to the registrant. Furthermore, by training the robot to detect sounds such as breaking glass and carelessly turning knobs, it can estimate whether or not there is an intruder. Also, by training the robot to detect sounds made when someone falls or sits down, and the state of breathing of an injured person, it can determine whether or not there is an injured person. Therefore, the detection data from the sound sensor can be used to monitor the presence or absence of trespassers or injured people by detecting different sound data and changes in sound over time.

[0037] The correlations for determining the suitability and status of an entrant based on the data detected by each sensor are as described above, but the suitability and status of an entrant can be determined with a certain degree of accuracy even when judging any of the above data individually.

[0038] (machine learning methods) In the case of supervised learning, the machine learning device 10 forms a learning model using a neural network model as described above. This neural network model is a well-known model, and learns the correlation between the input data and the output data by inputting input data included in the learning data to an input layer and comparing the output data output from the output layer as the inference result with the output data (teacher data) included in the learning data.

[0039] An example of a supervised learning method using a machine learning device is shown as a flowchart in Figure 3. This machine learning method corresponds to the learning phase, and the machine learning method will be explained below based on this flowchart. First, in step S100, the learning data acquisition unit 12 prepares a desired number of pieces of learning data as a preliminary preparation for starting machine learning, and stores the prepared learning data in the learning data storage unit 13. The number of pieces of learning data to be prepared here is set appropriately in consideration of the inference accuracy required for the learning model 11.

[0040] There are various methods available for preparing training data. <Preparing training data for airflow sensors> For example, the system simulates the behavior of hiding behind an object (installed object 3) and detects changes in the airflow over time during this behavior. Then, in association with this detected data, determination information (in this case, it is assumed that an intruder has entered, so output data "1") is input as output data from the operator terminal 31. In addition, the system simulates a person falling or sitting down, and acquires the airflow (data detected by the airflow sensor) at that time. Furthermore, in association with the acquired assumed data, judgment information (in this case, it is assumed that an injured person has occurred, so output data "1") is input from the operator terminal. Furthermore, the system simulates normal behavior when there are no intruders or injured people, and detects changes in airflow over time. In this case, since there are no intruders or injured people, output data "0" is input from the operator terminal. By repeating the above steps, learning data (input data and output data) for a plurality of sets of airflow sensors 6 are prepared.

[0041] <Preparing training data for odor sensors> For example, an odor such as body odor emanating from a registered person is detected, and in association with this, judgment information (which is the normal odor of the registered person himself / herself, so output data "0" is input from the worker terminal 31) is generated. In addition, the odor of a person's blood or sweat is detected in advance by an odor sensor, and corresponding judgment information (in this case, it is assumed that the person is bleeding or sweating due to an injury or illness, so output data "1") is input from the worker terminal 31. Furthermore, the system detects odors in the room under normal circumstances when there are no intruders or injured people, and in association with this, inputs output data "0" from the worker terminal. By repeating the above steps, learning data (input data and output data) for multiple pairs of odor sensors 7 are prepared.

[0042] <Preparing floor pressure sensor learning data> For example, the registered person is made to walk normally, and the load received by the floor pressure sensor 8 at that time and any changes over time are detected. Then, in association with this detected data, determination information (in this case, since this is the registered person's normal movement, the output data is "0") is input as output data from the worker terminal 31. Also, the person collapses or sits on the floor to simulate a motionless state, and the data detected by the floor pressure sensor 8 at that time is acquired. Then, in association with this detected data, judgment information (in this case, output data "1" since it is assumed that an injured person has occurred) is input as output data from the worker terminal 31. Furthermore, the expected movements of an intruder are simulated, and changes over time in the floor pressure sensor 8 are detected, and judgment information (in this case, output data "1" since the state is one in which an intruder is expected to enter) is input from the worker terminal in correspondence with this. By repeating the above steps, learning data (input data and output data) for a plurality of sets of floor pressure sensors 8 are prepared.

[0043] <Preparing training data for the sound sensor> In the learning data for the sound sensor 9, for example, the sound of the registered person's breathing or the rhythmic sound of walking is detected in advance, and judgment information (in this case, the output data is "0" since it is the detected information of the registered person) is input as output data from the worker terminal 31 in correspondence with this detected data. In addition, the sound of breaking glass or carelessly turning a knob is simulated, and the change in sound that occurs is detected. Then, in association with this detected data, determination information (in this case, since it is assumed that an intruder has entered, output data "1" is input as output data from the worker terminal 31). Furthermore, the system simulates the sounds that occur when someone falls or sits down, as well as the state of the victim's breathing, and detects the changes in sound that occur.Then, in association with the acquired detection data, judgment information (in this case, it is assumed that an injured person has occurred, so output data "1") is input from the worker's terminal. By repeating the above steps, learning data (input data and output data) for a plurality of sets of sound sensors 9 are prepared.

[0044] Next, in order to start machine learning, the machine learning unit 14 prepares a pre-learning learning model 11 configured as a neural network model (step S110). The input layer of this learning model 11 is associated with data detected by the monitoring sensors 100 (airflow sensor 6, odor sensor 7, floor pressure sensor 8, and sound sensor 9) as input data included in the learning data, and the output layer is associated with judgment information as output data included in the learning data.

[0045] Next, in step S120, the machine learning unit 14 acquires one set of learning data for each monitoring sensor 100 from the multiple sets of learning data obtained by each monitoring sensor stored in the learning data storage unit 13. The learning data may be acquired in a predetermined order or randomly.

[0046] Next, in step S130, the machine learning unit 14 inputs the input data included in the acquired set of learning data into the prepared learning model, and outputs an inference result. Then, the output data (teacher data) included in the learning data acquired in step S120 is compared with the output data output from the output layer as an inference result in step S130, and machine learning is performed using a known neural network method. That is, the machine learning unit 143 causes the learning model 11 to learn the correlation between the input data and the output data (information on the suitability or status of the person entering the room).

[0047] Thereafter, using this learning model under learning, the above-mentioned operation is performed on the remaining prepared learning data, and learning is continued in the machine learning unit 14 (step S140). At this time, the machine learning unit 14 may determine whether or not to continue machine learning based on the error between the output data and the teacher data, the remaining amount of unlearned learning data stored in the learning data storage unit 142, etc.

[0048] That is, if the machine learning unit 14 determines in step S140 to continue the machine learning, the process returns to step S120, and performs steps S120 to S130 on the currently-trained learning model 11 using untrained learning data. On the other hand, if the machine learning unit 14 determines in step S140 to end the machine learning, the machine learning unit 14 stores the generated trained learning model 11 in the learning model storage unit 15 in step S150, and ends the machine learning.

[0049] As described above, the machine learning device 10 and machine learning method according to this embodiment make it possible to provide a learning model 11 that can accurately infer (estimate) the suitability and status of a person entering a room from data obtained by a monitoring sensor.

[0050] Here, the learning model 11 may be formed by inputting the learning data of each sensor one set at a time, or may be formed by having each sensor learn using the machine learning unit 14 to form a respective learning model (a learning model based on the airflow sensor 6, a learning model based on the odor sensor 7, a learning model based on the floor pressure sensor 8, and a learning model based on the sound sensor 9).

[0051] Next, when "unsupervised learning" is adopted as the machine learning method, the learning data includes only input data when the room is in a steady state with no occupants. In other words, the learning data does not include output data, and is composed only of data from the airflow sensor, odor sensor, floor pressure sensor, and sound sensor in a steady state (normal state) with no occupants.

[0052] The learning data acquisition unit 12 acquires data from each sensor, and at the same time, the worker inputs into the worker terminal that the detected data is from a steady state (normal state) when no one is entering the room, thereby creating a set of learning data and storing this learning data in the learning data storage unit 13. By repeating this process, multiple sets of learning data are stored in the learning data storage unit 13.

[0053] The machine learning unit 14 inputs one or more sets of learning data into the learning model 11, and causes the learning model to learn the correlation between the input data included in the learning data and the determination information indicating that the room is in a steady state with no one in it, thereby generating a trained learning model. In this embodiment, an autoencoder, for example, can be used as a specific method for unsupervised learning by the machine learning unit 14.

[0054] This autoencoder model is a well-known model that inputs input data contained in training data into an input layer and compares this input data with output data output from the output layer as inference results to learn patterns and trends in the input data. This series of processes is repeatedly performed on the training model, and when a predetermined training termination condition is met, the machine learning is terminated and a trained autoencoder model is generated.

[0055] The learning model storage unit 15 is also a database that stores the learned learning model 11 generated by unsupervised learning by the machine learning unit 14. The learning model 11 stored in the learning model storage unit 15 is provided to the entrant estimation device 20 via any communication network or storage medium. The learning data storage unit 13 and the learning model storage unit 15 may be configured as a single storage device or as separate storage devices.

[0056] Here, the steady-state data from the airflow sensor 6 is detected as the steady-state airflow that occurs in the room when no one is in the room, and includes airflow caused by cooling air from electronic devices in the room and air conditioning air from air conditioning equipment. If there is a change in the detection value of the airflow sensor 6 even though it is not captured by the surveillance camera 2, there is a possibility that a person has entered the room behind the installed object, and if the detection value continues to change for a long period of time, there is a possibility that the person has collapsed or sat down behind the installed object.

[0057] The steady-state data from odor sensor 7 is detected when there is no one in the room and the electronic devices are in operation and the air conditioner is creating airflow in the room, and this is the odor data detected in that state. If the data from the odor sensor 7 deviates from the normal data even though it is not captured by the surveillance camera 2, it is possible that a change in the state of an object in the room (smoking, decay, etc.) has occurred, or that the smell of iron or an abnormal smell associated with bleeding in an injured person has been detected, which could indicate the presence of an injured person.

[0058] The steady-state data of the floor pressure sensor 8 is pressure data detected in an area that is a blind spot of the monitoring camera 2 when no one is in the room. If pressure is detected by the floor pressure sensor 8 even though it is not captured by the surveillance camera 2, and this state continues for a predetermined time or more, it is possible that a person has entered the room behind the installed object and is unable to move, such as having collapsed or sat down on the floor. It is also possible to determine from the pressure-sensitive area whether the person entering the room is sitting or collapsed.

[0059] The steady-state data from the sound sensor is detected when there is no one in the room, and includes the sounds of electronic devices operating, the air conditioning unit's fan noise, and outside noise. If the sound sensor detects the sound of something falling or rough breathing even though it is not captured by the surveillance camera, it is possible that an injured person has collapsed behind the object. Also, the sound of glass breaking or the careless turning of knobs could indicate that an intruder has entered.

[0060] FIG. 4 is a flowchart showing an example of a method for unsupervised learning using a machine learning device. As a preliminary preparation for having the machine learning device 10 perform machine learning, the learning data acquisition unit 12 acquires a desired number of sets of learning data and stores the acquired sets of learning data in the learning data storage unit 13. The number of sets of learning data to be prepared here is set appropriately taking into consideration the inference accuracy required for the learning model 11.

[0061] Various methods can be used to acquire learning data, but for example, input data that constitutes learning data is prepared by acquiring output data from each sensor during steady-state conditions when no one is entering the room. That is, the detection data from the airflow sensor 6 detects steady airflow when no one is in the room. The detection data from the odor sensor 7 detects steady odors when no one is in the room. The detection data from the floor pressure sensor 8 detects pressure (pressure when no pressure is applied to the pressure sensor) when no one is in the room. The detection data from the sound sensor 9 detects steady sound when no one is in the room. By repeating this process, multiple sets of learning data are prepared and stored in the learning data storage unit 13 (step S200).

[0062] Furthermore, before starting machine learning, a pre-learning learning model 11 configured as an autoencoder model is prepared. The input layer of this learning model 11 is associated with the detection data of the monitoring sensors 100 (airflow sensor 6, odor sensor 7, floor pressure sensor 8, and sound sensor 9) as input data included in the learning data (step S210).

[0063] After the advance preparation is completed, first, in step S220, the machine learning unit 14 acquires one set of learning data from the multiple sets of learning data stored in the learning data storage unit 13. The learning data may be acquired in a predetermined order or randomly.

[0064] Next, in step S230, the machine learning unit 14 performs unsupervised machine learning. That is, the machine learning unit 14 inputs input data included in the acquired set of learning data into a prepared learning model, causes an inference result to be output, compares the input data included in this learning data with output data output as an inference result from the output layer, and performs machine learning using a known autoencoder method.

[0065] Thereafter, using this learning model under learning, the above-mentioned operation is performed on the remaining prepared learning data, and learning continues in the machine learning unit 14. At this time, the machine learning unit 14 determines whether or not to continue machine learning based on the number of learning times, the remaining amount of unlearned learning data stored in the learning data storage unit 13, etc. (step S240). That is, if the machine learning unit 14 determines in step S240 to continue machine learning (YES), it performs steps S220 to S230 on the currently-trained learning model 11 using untrained learning data. On the other hand, if the machine learning unit 14 determines in step S240 not to continue machine learning (NO), then in step S250 the machine learning unit 14 stores the generated trained learning model 11 in the learning model storage unit 15 and ends the machine learning. The above machine learning method can provide a learning model 11 that can accurately infer (estimate) the suitability or state of a person entering a room from the monitoring sensor 100 installed in the room.

[0066] (Room occupant estimation device) As shown in FIG. 5, the entrant estimation device 20 includes an input data acquisition unit 21, an inference unit 22, a learning model storage unit 23, and an output processing unit 24. The input data acquisition unit 21 is an interface unit that is connected to each monitoring sensor 100 (airflow sensor 6, odor sensor 7, floor pressure sensor 8, sound sensor 9) and acquires the data detected by these monitoring sensors 100 as input data via the communication network 32. The inference unit 22 inputs the input data acquired by the input data acquisition unit 21 into the trained learning model 11 and performs an inference process to infer judgment information regarding the suitability or status of the person entering the room.The inference process uses the trained learning model 11, which has undergone supervised or unsupervised learning in the machine learning device 10.

[0067] The inference unit 22 may not only have the function of performing inference processing using the learning model 11, but may also include a pre-processing function of adjusting the input data acquired by the input data acquisition unit 21 into a desired format, etc., and inputting it into the learning model 11 as pre-processing of the inference processing, and a post-processing function of applying a predetermined logical formula or calculation formula to the value of the output data output from the learning model 11 as post-processing of the inference processing, thereby making a final judgment on the suitability and condition of the person entering the room.

[0068] The learning model storage unit 23 is a database that stores trained learning models 11 used in the inference process of the inference unit 22. The learning models 11 stored in the learning model storage unit 23 include, but are not limited to, the above-mentioned supervised learning models and unsupervised learning models. For example, multiple learning models for different input data, different numbers of input data, or different machine learning methods may be further stored and appropriately selected for use.

[0069] The output processing unit 24 performs output processing to output the inference result of the inference unit 22, i.e., judgment information on the suitability and status of the person entering the room. Specific output means can be various. For example, the output processing unit 24 may notify the worker of the judgment information by display or sound, or may transmit the judgment information to the management device as a judgment history and store it.

[0070] 6 is a flowchart showing an example of a method for determining the suitability or status of an entering person by the room monitoring system 1. In this example, a case will be described in which the determination information for the suitability or status of an entering person is defined as either "0" if normal or "1" if abnormal.

[0071] First, in step S300, the input data acquisition unit 21 acquires data detected by the monitoring sensors 100 (airflow sensor 6, odor sensor 7, floor pressure sensor 8, and sound sensor 9).

[0072] Next, in step S310, the inference unit 22 inputs the input data to the input layer of the learning model 11, and obtains the output data output from the output layer of the learning model 11.

[0073] Then, in step S320, as an example of post-processing of supervised or unsupervised learning, the inference unit 22 compares the value of the output data (a number between 0 and 1) with a predetermined threshold value to determine whether the entrant is normal (the entrant is a registered user) or abnormal (the entrant is a trespasser or has suffered an injury or illness). For example, if the value of the output data is less than the predetermined threshold value, the diagnostic information is determined to be "normal," and if the value is equal to or greater than the predetermined threshold value, the diagnostic information is determined to be "abnormal," and this is output as the inference result.

[0074] Next, in step S330, the output processing unit 24 determines whether the judgment information regarding the suitability or status of the person entering the room, which is the inference result of the inference unit 22, is "normal" or "abnormal." If it judges it to be "normal," it outputs information indicating "normal" (step S340). If it judges it to be "abnormal," it outputs information indicating "abnormal" to the display 30 together with an alarm (step S350). After the diagnostic information is output in step S340 or S350, the process of determining the suitability or status of the person entering the room ends.

[0075] As described above, according to the indoor monitoring system and indoor monitoring method of this embodiment, if there is a person entering the blind spot area α that cannot be captured by the monitoring camera 2, it is possible to determine whether the person is an intruder or an injured person without relying on the experience or intuition of the monitor.Furthermore, even if the monitoring camera 2 malfunctions or if the intruder disguises themselves and pretends to be a registered person, and would be overlooked based on the judgment of the monitoring camera 2 alone, it is possible to appropriately determine the suitability and condition of the person entering the room, thereby making it possible to assist monitoring by the monitoring camera 2.

[0076] In the above embodiment, a case has been described in which a neural network or an autoencoder is used as a specific method of machine learning by the machine learning unit 14, but the machine learning unit 14 may employ any other machine learning method. For example, tree-type methods such as decision trees and regression trees, ensemble learning such as bagging and boosting, neural network-type methods (including deep learning) such as recurrent neural networks and convolutional neural networks, clustering-type methods such as hierarchical clustering, non-hierarchical clustering, k-nearest neighbors, and k-means, multivariate analysis such as principal component analysis, factor analysis, and logistic regression, support vector machines, etc. may be employed.

[0077] In addition, the above-mentioned machine learning device 10 and entrant estimation device 20 can also be provided in the form of a program (machine learning program or room monitoring program) for executing each step of the above-mentioned machine learning method and entrant estimation method.

[0078] Furthermore, although the above-mentioned configuration has been used as an example to determine the suitability or condition of a person entering a room, a similar configuration can be used for other determinations. For example, to determine the condition of an object in a room, a learning model that has learned the correlation between learning data including data detected by the above-mentioned monitoring sensor 100 as input data and the condition of the object in the room (smoking, decay, overturning, collapse, etc.) can be stored in the learning model memory unit 15, the input data including the data detected by the monitoring sensor 100 can be acquired by the input data acquisition unit 21, and the inference unit 22 can input the input data acquired by the input data acquisition unit 21 into the learning model to infer the condition of the object in the room. [Explanation of symbols]

[0079] 1. Indoor surveillance system 2. Surveillance cameras 3 Installations 6 Airflow Sensor 7 Odor Sensor 8 Floor pressure sensor 9 Sound Sensor 11 Learning Model 15,23 Learning model memory unit 21 Input data acquisition unit 22 Reasoning part 100 Monitoring Sensor

Claims

1. An indoor monitoring system for monitoring an indoor space, a monitoring sensor including at least one of an airflow sensor that detects a change in airflow in the room, an odor sensor that detects an odor in the room, a floor pressure sensor that detects pressure on a floor surface in the room, and a sound sensor that detects sound in the room; a learning model storage unit that stores a learning model that has learned a correlation between learning data including data detected by the monitoring sensor as input data and the suitability or status of a person entering the room; an input data acquisition unit that acquires input data including data detected by the monitoring sensor; an inference unit that inputs the input data acquired by the input data acquisition unit into the learning model and infers the suitability or status of the person entering the room; An indoor monitoring system comprising:

2. The learning data is associated with the input data and further includes, as output data, information indicating whether the person entering the room is suitable or in one of a plurality of states; the learning model learns a correlation between the input data and the output data through supervised learning; The indoor monitoring system according to claim 1 .

3. the learning data includes only the input data when it indicates the suitability or the state of the person entering the room is a predetermined state, the learning model learns a correlation between the input data and information indicating whether the person in the room is suitable or in a predetermined state through unsupervised learning; The indoor monitoring system according to claim 1 .

4. Equipped with surveillance cameras to monitor the room, 4. The indoor monitoring system according to claim 1, wherein the monitoring sensor is installed in a blind spot of the monitoring camera.

5. The indoor monitoring system according to any one of claims 1 to 3, characterized in that the suitability of the person entering the room is determined by determining whether or not the person is an intruder, and the condition of the person entering the room is determined by determining whether or not the person is injured or ill.

6. a monitoring sensor including at least one of an airflow sensor that detects a change in airflow in the room, an odor sensor that detects an odor in the room, a floor pressure sensor that detects pressure on the floor surface of the room, and a sound sensor that detects sound in the room; An indoor monitoring method for monitoring the suitability or status of a person entering the room using a learning model that has learned a correlation between learning data including data detected by the monitoring sensor as input data and the suitability or status of the person entering the room, an input data acquisition step of acquiring input data including data detected by the monitoring sensor; an inference step of inputting the input data acquired by the input data acquisition step into the learning model and inferring the suitability or status of the person occupying the room; An indoor monitoring method comprising:

7. A room monitoring program for causing a computer to execute each step of the room monitoring method according to claim 6.

8. 4. The room monitoring system according to claim 1, wherein the state of an object in the room is detected instead of the suitability or state of the person entering the room.

Images