Abnormality detection system, abnormality detection method, and abnormality detection program

Abstract

To provide an abnormality detection system and the like that can detect an occurrence of an abnormality in a monitoring target person such as overturning.SOLUTION: There is provided an abnormality detection system which detects an occurrence of an abnormality in a person who is in a predetermined area by scanning the inside of the predetermined area using a three-dimensional sensor installed in the predetermined area, the system comprising: a scan data acquisition unit that acquires scan data within the predetermined area from the three-dimensional sensor; a scan data determination unit that determines whether or not there is an occurrence of an abnormality in the person in the predetermined area based on a difference between scan data acquired by the scan data acquisition unit and scan data of the predetermined area acquired in advance when a person in the predetermined area is normal; and a notification unit that notifies a monitoring person in the predetermined area that an abnormality has occurred in the person when the scan data determination unit determines that an abnormality has occurred in the person located in the predetermined area.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an anomaly detection system, an anomaly detection method, and an anomaly detection program, and more particularly to an anomaly detection system, an anomaly detection method, and an anomaly detection program that detect an anomaly in a person in a non-contact manner. [Background technology]

[0002] Conventionally, surveillance cameras have been used to remotely and non-contactly monitor people who are receiving nursing care, caregiving, nursing care, childcare, etc. Surveillance cameras are installed in the rooms of people who are being monitored, and the behavior of the people being monitored while they are sleeping and after they wake up is monitored through the surveillance cameras. If the monitor suspects that something is wrong with the people being monitored, the monitor goes to the rooms of people who are being monitored and checks their condition with his or her own eyes.

[0003] When a person is monitored by a surveillance camera, there is a risk that the person may overlook an abnormality occurring in the person being monitored. For example, when a person falls and leans against a bed, the person watching the monitor of the surveillance camera may not notice that the person has fallen. It is also difficult to distinguish between a person who has fallen off a bed and a person who is sitting leaning against the bed through a surveillance camera. In this way, when a person is monitored by a surveillance camera, it is sometimes difficult to distinguish whether the person has fallen or fallen down, or is simply crouching or sitting.

[0004] In recent years, a cardiopulmonary function monitoring device has been proposed that uses a radio wave sensor to detect the movement of a person being monitored (see Patent Document 1). This radio wave sensor uses the Doppler effect to irradiate the person being monitored with radio waves, detect the reflected waves, and analyze the detected reflected waves to detect the movement of the person being monitored.

[0005] The cardiopulmonary function monitoring device disclosed in Patent Document 1 is capable of calculating and monitoring the respiratory rate and heart rate of the person being monitored without contact. However, the cardiopulmonary function monitoring device disclosed in Patent Document 1 does not take into consideration the occurrence of abnormalities in the person being monitored that involve whole-body movement, such as falling or tripping, and is still unable to detect the occurrence of abnormalities that involve whole-body movement in the person being monitored. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2000-83927 A Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, an object of the present invention is to provide an anomaly detection system, an anomaly detection method, and an anomaly detection program that can detect the occurrence of an abnormality in a monitored person, such as a fall. [Means for solving the problem]

[0008] That is, the anomaly detection system of the first aspect is an anomaly detection system that detects the occurrence of an abnormality in a person in a specified area by scanning the inside of the specified area using a three-dimensional sensor installed in the specified area, and is characterized by comprising a scan data acquisition unit that acquires scan data of the inside of the specified area from the three-dimensional sensor, a scan data determination unit that determines whether or not an abnormality has occurred in the person in the specified area based on the difference between the scan data acquired by the scan data acquisition unit and previously acquired scan data of the specified area when the person in the specified area is normal, and an alarm unit that, when the scan data determination unit determines that an abnormality has occurred in the person in the specified area, alerts a supervisor of the specified area that an abnormality has occurred in the person.

[0009] As a second aspect, in the anomaly detection system according to the first aspect, the anomaly may be a person falling down within the predetermined area.

[0010] In a third aspect, in the anomaly detection system according to the first aspect, the scan data determination unit may input the scan data acquired by the scan data acquisition unit into a scan data learning model that has previously learned the correspondence between the occurrence of an abnormality in a person and the scan data, thereby determining whether or not an abnormality has occurred in the person based on the scan data acquired by the scan data acquisition unit.

[0011] In a fourth aspect, in the anomaly detection system according to the first aspect, an absence determination unit is further provided which determines that no person is present in the specified area when there is no person in the specified area based on a difference between the scan data acquired by the scan data acquisition unit and scan data of the specified area in an unoccupied state acquired in advance, and the notification unit may notify a monitor of the specified area that no person is present in the specified area when the absence determination unit determines that no person is present.

[0012] In a fifth aspect, the anomaly detection system according to the first aspect may further include an imaging data acquisition unit that is installed in a specified area and acquires imaging data from an imaging unit that captures an image of the interior of the specified area, a judgment time acquisition unit that, when the scan data determination unit determines that an abnormality has occurred in a person, acquires the time when the scan data determination unit made the judgment as a judgment time, and a memory control unit that stores in a memory unit the imaging data acquired by the imaging data acquisition unit for a specified period starting from a specified time before the judgment time acquired by the judgment time acquisition unit.

[0013] A sixth aspect is an anomaly detection system according to the fifth aspect, further comprising a code issuing unit that issues an access code for the memory control unit to access the imaging data stored in the memory unit, and the notification unit may notify a monitor in a specified area that an abnormality has occurred with a person, together with the access code issued by the code issuing unit.

[0014] A seventh aspect is an anomaly detection system according to the fifth aspect, further comprising an imaging data determination unit that inputs the imaging data acquired by the imaging data acquisition unit into an imaging data learning model that has previously learned a correspondence between the occurrence of an abnormality in a person and the imaging data, and determines whether or not an abnormality has occurred in the person based on the imaging data acquired by the imaging data acquisition unit, and when the imaging data determination unit determines that an abnormality has occurred, the determination time acquisition unit may acquire the time at which the imaging data determination unit made the determination as the determination time.

[0015] An eighth aspect is an anomaly detection system according to the first aspect, further comprising a heat distribution acquisition unit that acquires heat distribution inside the specified area as heat distribution data by imaging the inside of the specified area using a thermographic camera installed in the specified area, a body temperature acquisition unit that acquires a body temperature of a person in the specified area based on the heat distribution data acquired by the heat distribution acquisition unit, and a body temperature determination unit that determines that an abnormality has occurred in the person when the body temperature of the person acquired by the body temperature acquisition unit exceeds a threshold value, and the notification unit may be configured to notify a supervisor in the specified area that an abnormality has occurred in the person when the body temperature determination unit determines that an abnormality has occurred.

[0016] A ninth aspect is an anomaly detection system according to the first aspect, further comprising a temperature data acquisition unit that acquires the air temperature inside the specified area as temperature data detected by a temperature sensor that detects the air temperature inside the specified area, and a temperature determination unit that determines that an abnormality has occurred in the specified area if the temperature data acquired by the temperature data acquisition unit exceeds a threshold value, and the notification unit may be configured to notify a supervisor of the specified area that an abnormality has occurred in the specified area when the temperature determination unit determines that an abnormality has occurred.

[0017] A tenth aspect is an anomaly detection system according to the ninth aspect, further comprising a humidity data acquisition unit that acquires the humidity inside the specified area detected by a humidity sensor that detects the humidity inside the specified area as humidity data, and a humidity determination unit that determines that an abnormality has occurred in the specified area when the humidity data acquired by the humidity data acquisition unit exceeds a threshold value, and the notification unit may be configured to notify a monitor of the specified area that an abnormality has occurred in the specified area when the humidity determination unit determines that an abnormality has occurred.

[0018] An eleventh aspect is an anomaly detection system according to the tenth aspect, further comprising a body sensation index calculation unit that calculates a body sensation index of a person in a specified area based on temperature data acquired by the temperature data acquisition unit and humidity data acquired by the humidity data acquisition unit, and a body sensation index determination unit that determines that an abnormality has occurred in the specified area when the body sensation index of the person calculated by the body sensation index calculation unit exceeds a threshold value, and the notification unit may be configured to notify a monitor of the specified area that an abnormality has occurred in the specified area when the body sensation index determination unit determines that an abnormality has occurred.

[0019] A twelfth aspect is an anomaly detection system according to the eleventh aspect, further comprising a remote control unit that controls a smart remote control unit that operates and monitors household electrical appliances within a specified area, wherein the household electrical appliances are air conditioning appliances that condition the air within the specified area, and when at least one of the temperature determination unit, the humidity determination unit, and the body sensation index determination unit determines an abnormality, the remote control unit controls the air conditioning appliance so that the temperature and humidity inside the specified area become preset temperature and humidity.

[0020] According to a thirteenth aspect, in the anomaly detection system according to the eleventh aspect, the sensation index may be at least one of a discomfort index and a sensation temperature.

[0021] A fourteenth aspect is an anomaly detection system according to the first aspect, further comprising a biometric information acquisition unit that acquires, as biometric information data, biometric information of a person detected by a biometric sensor that detects biometric information of the person while sleeping, and a biometric information determination unit that determines that an abnormality has occurred in the person when the biometric information acquired by the biometric information acquisition unit exceeds a biometric threshold value, and the notification unit may be configured to notify a monitor in a specified area that an abnormality has occurred in the person when the biometric information determination unit determines that an abnormality has occurred.

[0022] In a fifteenth aspect, in the anomaly detection system according to the fourteenth aspect, the biometric information acquisition unit may use either or both of scan data from a three-dimensional sensor or thermal distribution data relating to the heat distribution inside a specified area acquired by imaging the inside of the specified area using a thermographic camera installed in the specified area as actual data, and may input the newly acquired actual data into a virtual biometric sensor model that has previously learned the correspondence between a person's biometric information while sleeping and the actual data, thereby acquiring virtual biometric information data relating to the person's biometric information while sleeping based on the newly acquired actual data, and the biometric information determination unit may determine that an abnormality has occurred in the person when the virtual biometric information data acquired by the biometric information acquisition unit exceeds a threshold value of the biometric information.

[0023] According to a sixteenth aspect, in the anomaly detection system according to the fourteenth aspect, the biological information may be at least one of a heart rate and a respiratory rate of the person while sleeping.

[0024] A 17th aspect is an abnormality detection system according to the first aspect, further comprising a body movement information acquisition unit that acquires body movement information detected by a body movement sensor that detects vibrations caused by body movements of a person while sleeping as body movement information data, and a body movement information determination unit that determines that an abnormality has occurred in the person when the body movement information data acquired by the body movement information acquisition unit indicates a state of no body movement for a predetermined period of time or longer, and the notification unit may be configured to notify a monitor in a predetermined area that an abnormality has occurred in the person when the body movement information determination unit determines that an abnormality has occurred.

[0025] In an 18th aspect, in the abnormality detection system according to the 17th aspect, the body movement information acquisition unit uses either or both of scan data from a three-dimensional sensor or thermal distribution data relating to the heat distribution inside a specified area acquired by imaging the inside of the specified area using a thermographic camera installed in the specified area as actual data, and acquires virtual body movement information data relating to vibrations caused by the person's body movements while sleeping based on the newly acquired actual data by inputting the newly acquired actual data into a virtual body movement sensor model that has previously learned the correspondence between vibrations caused by the person's body movements while sleeping, and the body movement information determination unit may determine that an abnormality has occurred in the person when the virtual body movement information data acquired by the body movement information acquisition unit indicates a state in which there is no body movement for a predetermined period of time or longer.

[0026] A 19th aspect is an anomaly detection system according to the 12th aspect, further comprising a setting information acquisition unit that acquires setting information input to a smart remote control unit by a person present in a specified area, a setting information storage unit that separates the setting information acquired by the setting information acquisition unit for each model number assigned to each household electrical appliance and stores the setting information in a storage unit, and a search unit that searches for and identifies the setting information stored in the storage unit based on the model number input by a person present in another specified area, and the setting information identified by the search unit may be used for setting the smart remote control unit in the other specified area.

[0027] The anomaly detection method according to the 20th aspect is an anomaly detection method that detects the occurrence of an abnormality in a person in a specified area by scanning the inside of the specified area using a three-dimensional sensor installed in the specified area, and is characterized in that it has a computer execute a scan data acquisition step of acquiring scan data of the inside of the specified area from the three-dimensional sensor, a scan data determination step of determining whether or not an abnormality has occurred in the person in the specified area based on the difference between the scan data acquired in the scan data acquisition step and scan data of the specified area previously acquired when the person in the specified area is normal, and a notification step of notifying a supervisor of the specified area that an abnormality has occurred in the person if it is determined in the scan data determination step that an abnormality has occurred in the person in the specified area.

[0028] The anomaly detection program of the 21st aspect is an anomaly detection program that detects the occurrence of an abnormality in a person in a specified area by scanning the inside of the specified area using a three-dimensional sensor installed in the specified area, and is characterized in that the computer is caused to exhibit a scan data acquisition function that acquires scan data of the inside of the specified area from the three-dimensional sensor, a scan data determination function that determines whether or not an abnormality has occurred in the person in the specified area based on the difference between the scan data acquired by the scan data acquisition function and scan data of the specified area previously acquired when the person in the specified area is normal, and an alarm function that, when the scan data determination function determines that an abnormality has occurred in the person in the specified area, an alarm function that alerts a supervisor of the specified area that an abnormality has occurred in the person. Effect of the Invention

[0029] The anomaly detection system etc. of the present invention is an anomaly detection system that detects the occurrence of an abnormality in a person in a specified area by scanning the inside of the specified area using a three-dimensional sensor installed in the specified area, and is equipped with a scan data acquisition unit that acquires scan data of the inside of the specified area from the three-dimensional sensor, a scan data determination unit that determines whether or not an abnormality has occurred in the person in the specified area based on the difference between the scan data acquired by the scan data acquisition unit and pre-acquired scan data of the specified area when the person in the specified area is normal, and an alarm unit that, when the scan data determination unit determines that an abnormality has occurred in the person in the specified area, notifies a monitor of the specified area that an abnormality has occurred in the person, and thus it is possible to detect the occurrence of an abnormality in the person being monitored, such as a fall. [Brief description of the drawings]

[0030] [Figure 1] FIG. 1 is a diagram for explaining an overview of an anomaly detection system according to this embodiment. [Diagram 2] FIG. 2 is an example of an infrastructure configuration diagram including the anomaly detection system according to this embodiment. [Diagram 3] FIG. 3 is an example of a block diagram showing the configuration of the anomaly detection system and the monitor terminal according to this embodiment. [Figure 4] FIG. 4 is an example of a block diagram showing the configuration of a terminal for a monitored person according to this embodiment. [Diagram 5] FIG. 5 is an example of a block diagram showing the functional configuration of the anomaly detection system according to this embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a display screen of the observer terminal according to this embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a display screen of the observer terminal according to this embodiment. [Figure 8] FIG. 8 is an example of a flowchart of the anomaly detection program according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] (Overview of Anomaly Detection System 10) An embodiment of an anomaly detection system 10 of the present invention will be described with reference to FIGS. First, an overview of an anomaly detection system 10 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining the overview of an anomaly detection system 10 according to this embodiment.

[0032] The anomaly detection system 10 of this embodiment is used by a user to remotely detect an anomaly occurring in a specific person. A specific person refers to a person who stays in a specific area for any length of time, such as a person who stays overnight in the specific area, a person who stays in the specific area for any length of time, or a person who resides in the specific area.

[0033] A user of the anomaly detection system 10 is a monitor who monitors a specific person by observing or watching over them. Monitors include, for example, operators of senior housing and nursing homes for the purpose of caring for the elderly, operators of hospitals and other medical facilities for the purpose of caring for hospitalized patients, operators of nursery schools and kindergartens for the purpose of caring for infants and small children, operators who monitor lodging facilities such as hotels, etc. The monitored person is a person who is the target of monitoring by the above-mentioned operator, for example, elderly people who use senior housing, hospitalized patients, infants in nursery schools, and hotel guests. Note that users of the anomaly detection system 10 are not limited to the monitors described above, and there may also be cases where a monitored person monitors other monitored people.

[0034] The abnormality detection system 10 detects the occurrence of an abnormality in a person present in the predetermined area 1 by scanning the inside of the predetermined area 1 using a three-dimensional sensor 21 (see FIG. 4) described below that is installed in the predetermined area 1. The predetermined area 1 is a space where the monitored person lives or stays, and is, for example, a room such as a living room, a bedroom, or a kitchen. The predetermined area 1 may also be an entrance hall, a toilet, a bathroom, a dressing room, a washroom, a closet, a storeroom, a corridor, or the like other than a room. In addition, if it is possible to scan outdoors by placing a sensor on a fence in a park, the predetermined area may be outdoors. In this embodiment, a bedroom in which a bed 2 is installed as shown in FIG.

[0035] An abnormality refers to an abnormality occurring to a person in the specified area 1, and an abnormality in the living environment inside the specified area 1. An abnormality occurring to a person refers to a mental or physical problem of the person, specifically, an injury, illness, poor health, accident, etc. Abnormalities in the living environment refer to abnormalities in temperature, humidity, and physical sensation index. An abnormality may be a person falling down in the predetermined area 1. Also, a person falling off a bed 2 placed in the predetermined area 1 may be included in the occurrence of an abnormality. Furthermore, the occurrence of an abnormality includes the occurrence of a symptom of an abnormality. A symptom of an abnormality is a precursor to the occurrence of an abnormality. An example of a symptom of an abnormality is when most of the body of a person in bed 2 is sticking out of bed 2, but the person has not yet fallen off bed 2.

[0036] 1, a bed 2 for use by the person being monitored, an air conditioner 5, and lighting equipment 6 are installed in the predetermined area 1. A terminal 12 for the person being monitored is also installed in the predetermined area 1. The terminal 12 for the person being monitored is installed on a mounting stand 4a provided on a support 4 installed between the floor and the ceiling. The power required to operate the terminal 12 for the person being monitored is supplied from a wiring plug connector in the predetermined area 1. The monitored person terminal 12 may be installed not only on the support pole 4 but also on the ceiling or on top of furniture. The monitored person terminal 12 is preferably installed at a high place so that the entire area of ​​the predetermined area 1 can be viewed from the built-in three-dimensional sensor 21.

[0037] A biological / body movement sensor 3 is installed on the bed 2. The biological / body movement sensor 3 is a rectangular sheet that is inserted under the mattress and detects biological information and body movement information of the monitored person while sleeping.

[0038] (Infrastructure configuration of anomaly detection system 10) The infrastructure configuration of the anomaly detection system 10 will be described with reference to Fig. 2. Fig. 2 is an example of an infrastructure configuration diagram including the anomaly detection system 10 according to this embodiment. The anomaly detection system 10 receives a connection from a monitor terminal 11 used by a monitor via a network 20 including the Internet. The anomaly detection system 10 also communicates with a monitored person terminal 12 via the network 20 to obtain various data described below. The anomaly detection system 10 accumulates various data for use by the monitor and communicates with the monitor terminal 11.

[0039] The monitor uses an information processing device (computer) such as a commercially available general-purpose personal computer (hereinafter referred to as PC), a notebook PC, a tablet PC, or a smartphone as a monitor terminal 11 to connect to the anomaly detection system 10 via a network 20, receive notifications from the anomaly detection system 10 as described below, and use other functions of the anomaly detection system 10. In addition, a web browser runs on the monitor terminal 11, and the monitor terminal 11 can connect to the anomaly detection system 10 through the web browser and display display screens 60, 90 (see FIGS. 6 and 7) as described below and prepared by the anomaly detection system 10 on the monitor of the monitor terminal 11.

[0040] In addition, in addition to using a general-purpose information processing device as the monitor terminal 11, the monitor terminal 11 may be provided exclusively for using the functions of the anomaly detection system 10, such as receiving notifications from the anomaly detection system 10.

[0041] The anomaly detection system 10 may be managed and operated by a monitor, or may be managed and operated by an operator who is different from the monitor and operates a service business that provides administrators with the functions of the anomaly detection system 10. The anomaly detection system 10 may be installed on-premise, or may be a cloud server of a cloud computing service.

[0042] On-premise refers to constructing the anomaly detection system 10 by installing physical equipment on the premises of a facility owned by a person who manages and operates the anomaly detection system 10, and operating the system. On the other hand, a cloud computing service refers to providing a computing service via a network 20, such as GCP (registered trademark) (Google Cloud Platform), AWS (registered trademark) (Amazon Web Services), and AZURE (registered trademark). When constructing the anomaly detection system 10 using a cloud computing service, a multi-tenant system is adopted that uses part of the cloud server of the service.

[0043] The anomaly detection system 10 of the present embodiment is assumed to be installed on-premise as described above, and uses an information processing device (computer) such as a server, a workstation, a personal computer (hereinafter referred to as a PC), or a notebook PC.

[0044] The anomaly detection system 10 and the monitor's terminal 11 have in common the fact that they are equipped with the functions of an information processing device (computer). The difference between the anomaly detection system 10 and the monitor's terminal 11 is that the ROM 10b, RAM 10c, and storage unit 10d described below of the anomaly detection system 10 are desirably of large capacity. In addition, since the anomaly detection system 10 is expected to receive requests (demands) from multiple monitor's terminals 11 simultaneously, it is required to be stable and durable, and the processing performance required of the CPU 10e is high. Furthermore, the storage unit 10d of the anomaly detection system 10 may be configured as a database. A database is a group of information that is accumulated, organized, and stored.

[0045] 2, one monitor terminal 11 is illustrated for convenience, but two or more monitor terminals 11 may connect to the anomaly detection system 10 via the network 20 and transmit requests. Therefore, it is possible to monitor the same monitoring target using two or more monitor terminals 11.

[0046] 2, for convenience, two monitored person terminals 12 (monitored person terminal A 12a and monitored person terminal B 12b) are shown, but one or three or more monitored person terminals 12 can be connected to and communicate with the anomaly detection system 10 via the network 20. Therefore, the monitor terminal 11 can monitor multiple predetermined areas 1 simultaneously.

[0047] Furthermore, two or more monitored person terminals 12 may be installed in one specified area 1. By installing two or more monitored person terminals 12 in one specified area 1 at different positions, the three-dimensional sensor 21, the imaging unit 22 with night vision function, and the thermography camera 23 are installed at different positions, and it is possible to image one specified area 1 by changing the imaging direction and angle of view, thereby reducing blind spots and suppressing the possibility of overlooking the occurrence of an abnormality.

[0048] (Configuration of anomaly detection system 10 and monitor terminal 11) The configurations of the anomaly detection system 10 and the monitor terminal 11 will be described with reference to Fig. 3. Fig. 3 is an example of a block diagram showing the configurations of the anomaly detection system 10 and the monitor terminal 11 according to this embodiment.

[0049] The anomaly detection system 10 and the monitor terminal 11 include a communication interface 10a, a read only memory (ROM) 10b, a random access memory (RAM) 10c, a storage unit 10d, a central processing unit (CPU) 10e, and an input / output interface 10f.

[0050] The communication interface 10a has a function of transmitting and receiving data handled by the anomaly detection system 10 and the monitor terminal 11 to and from other devices via a network 20 including the Internet.

[0051] The storage unit 10d can be used as a storage device for the anomaly detection system 10 and the monitor terminal 11, and is configured, for example, by a hard disk drive, a solid state drive, a flash memory, or the like, and the storage unit 10d can also be configured using cloud storage. As described above, the storage unit 10d of the anomaly detection system 10 may be configured as a database. Configuring the storage unit 10d as a database allows for efficient management of huge amounts of data, provides excellent searchability and accessibility, and allows for quick reference of required information.

[0052] In addition, the memory unit 10d of the anomaly detection system 10 stores the anomaly detection program described below, an OS (Operating System) required for the operation of the anomaly detection system 10, various other applications, and various data used by the applications.

[0053] The storage unit 10d of the monitor terminal 11 stores an OS that is a basic application of a general-purpose PC, various other applications, and various data used by the applications. The OS is a type of application built into the anomaly detection system 10 and the monitor terminal 11, and has a function of managing basic control of the anomaly detection system 10 and the monitor terminal 11.

[0054] The anomaly detection system 10 stores an anomaly detection program, which will be described later, in the ROM 10b or the storage unit 10d, and loads the anomaly detection program into a main memory constituted by the RAM 10c etc. The CPU 10e accesses the main memory into which the anomaly detection program has been loaded, and executes the anomaly detection program.

[0055] The supervisor terminal 11 stores the OS in the ROM 10b or the storage unit 10d, and loads the OS into a main memory configured by the RAM 10c, etc. The CPU 10e accesses the main memory into which the OS has been loaded, and executes the OS.

[0056] The input / output interface 10f transmits and receives data to and from devices external to the anomaly detection system 10 and the monitor terminal 11. The external devices refer to an input device 10g and an output device 10h that input and output data to and from the anomaly detection system 10 and the monitor terminal 11. The input device 10g is, for example, a keyboard and a mouse, and the output device 10h is, for example, a monitor, a printer, and a speaker.

[0057] (Configuration of monitored person terminal 12) Next, the monitored person's terminal 12 will be described with reference to Fig. 4. Fig. 4 is an example of a block diagram showing the configuration of the monitored person's terminal 12 according to this embodiment.

[0058] The monitored person's terminal 12 includes a communication interface 12a, a ROM 12b, a RAM 12c, a storage unit 12d, a CPU 12e, and an input / output interface 12f. The monitored person's terminal 12 further includes a three-dimensional sensor 21, an imaging unit 22 with a night vision function, a thermography camera 23, a temperature sensor 24, a humidity sensor 25, and a smart remote control unit (so-called smart remote control unit) 26. The monitored person's terminal 12 also includes a biological / body movement sensor 3 as an external device.

[0059] The communication interface 12a has a function of transmitting and receiving data handled by the monitored person terminal 12 to and from other devices via a network 20 including the Internet. The monitored person terminal 12 of this embodiment is connected to the network 20 by a wireless LAN using Wi-Fi (registered trademark: Wireless Fidelity). The communication interface 12a transmits and receives data using radio waves in the 2.4 GHz band or 5 GHz band.

[0060] The storage unit 12d is mainly used as a storage area for temporarily storing acquired data. The storage unit 12d is used as a data buffer until various data acquired by the monitored person terminal 12 is transmitted to the anomaly detection system 10 via the network 20 by the communication interface 12a, and is composed of, for example, a hard disk drive, a solid state drive, and a flash memory.

[0061] The storage unit 12d also stores an OS necessary for the monitored person terminal 12 to operate, various other applications, and various data used by the applications.

[0062] The monitored person terminal 12 stores the OS in the ROM 12b or the storage unit 12d, and loads the OS into a main memory configured by the RAM 12c, etc. The CPU 12e accesses the main memory into which the OS has been loaded, and executes the OS.

[0063] Note that the OS required for the operation of the monitored person terminal 12 may have some of its functions virtualized, and in reality, the functions may be handled by the OS of the anomaly detection system 10. Specifically, the OS of the monitored person terminal 12 may have only functions required for communication with the anomaly detection system 10, and the OS of the anomaly detection system 10 may control the three-dimensional sensor 21, the imaging unit 22 with night vision function, the thermography camera 23, the temperature sensor 24, the humidity sensor 25, and the biological body / body movement sensor 3 provided in the monitored person terminal 12, and control related to the transmission of various data acquired by these various sensors, the imaging unit 22, and the thermography camera 23 to the anomaly detection system 10.

[0064] In this case, the anomaly detection system 10 determines the timing of transmitting the above-mentioned various data acquired by the monitored person terminal 12 to the anomaly detection system 10, and also determines which data to transmit. In this way, it becomes possible to perform updates and improvements to the OS of multiple monitored device terminals 12 all at once in the OS of the anomaly detection system 10, which makes it possible to reduce costs such as maintenance and development costs and also improves work efficiency.

[0065] Furthermore, since updating the OS that controls the various sensors, imaging unit 22, and thermographic camera 23, which is required for updating, repairing, and improving the hardware of the three-dimensional sensor 21, the imaging unit with night vision function 22, the thermographic camera 23, the temperature sensor 24, the humidity sensor 25, and the biological / body movement sensor 3, can be achieved simply by updating the OS of the anomaly detection system 10, there is no need to update the OS of the multiple monitored person terminals 12, which also improves work efficiency.

[0066] If the OS of multiple monitored terminals 12 are updated at the same time on the monitored terminal 12 side, a problem occurs in that sensing of the monitored terminals 12 during this period is stopped all at once. For this reason, it is desirable to stagger the update of the OS of each monitored terminal 12, but this work is expected to be cumbersome. However, the configuration in which the OS that controls the monitored terminal 12 can be updated simply by updating the OS of the anomaly detection system 10 as described above solves the problem of stopping sensing of multiple monitored terminals 12 all at once. The OS of the monitored person terminal 12 may be so-called firmware.

[0067] The input / output interface 12 f transmits and receives data to and from the three-dimensional sensor 21 , the imaging unit 22 with night vision function, the thermography camera 23 , the temperature sensor 24 , the humidity sensor 25 , the smart remote control unit 26 , and the biological body / body movement sensor 3 .

[0068] The three-dimensional sensor 21 is installed in the predetermined area 1 by being built into the monitored person terminal 12 and scans the inside of the predetermined area 1. Scanning is the process of measuring the distance to an object by irradiating the object with electromagnetic waves, sound waves, or other waves and capturing the reflected waves. Scanning is the process of irradiating waves from the inside of the specified area 1 to be measured toward the outside, thereby reading the internal conditions of the specified area 1, such as the internal shape of the specified area 1, the layout of furniture, and the positions and postures of people.

[0069] Examples of the three-dimensional sensor 21 include a three-dimensional ultrasonic sensor and a LiDAR. A 3D ultrasonic sensor is a sensor that enables object detection in three dimensions by using one element of a transmitting ultrasonic speaker and three receiving microphones. LiDAR (Light Detection And Ranging) is a technology that irradiates a laser beam and measures the distance to an object and the shape of the object based on the information of the reflected light. There are two types of LiDAR: TOF (Time of Flight) and FMCW (Frequency Modulated Continuous Wave), and either method may be used as the three-dimensional sensor 21. The TOF method is a method that measures the distance to an object by instantaneously irradiating a pulse of laser beam and measuring the time it takes for the pulse wave to be reflected by the object and return. The FMCW method is a method that continuously irradiates a transmitted laser beam while changing the frequency, and measures the distance from the frequency of the received wave reflected by the object and returned.

[0070] The imaging unit 22 with night vision function is a so-called camera that is installed in the specified area 1 by being built into the monitored person's terminal 12, and captures images of the interior of the specified area 1, using the night vision function to capture images of the interior of the specified area 1 when the specified area 1 is dark, such as during bedtime. The night vision function of the imaging unit 22 may be either a night vision correction method that amplifies the sensitivity or a method that visualizes infrared rays emitted from an object. The night vision correction method can capture a color image even in a dim situation with few light sources, but has the disadvantage that it cannot capture an image in complete darkness. The infrared visualization method is sensitive even in complete darkness, but can capture a black and white image or a thermal distribution image colored according to the intensity of infrared rays emitted from the object, but cannot capture the object in its original color. The imaging unit 22 with night vision function of this embodiment uses a night vision correction method, and when the lights in the specified area 1 are turned off, such as at bedtime, the area is not made completely dark, but a night light such as a canopy lamp built into the lighting device 6 (see FIG. 1) is turned on to ensure a light source sufficient to perform night vision correction.

[0071] The thermographic camera 23 is installed in the predetermined area 1 by being built into the monitored person terminal 12, and captures an image of the inside of the predetermined area 1 to obtain a heat distribution inside the predetermined area 1. The thermography camera 23 is a camera that detects the amount of infrared radiation emitted from an object and visualizes the temperature. The amount of infrared radiation increases as the temperature of the object increases, so the thermography camera 23 can measure the temperature without contact. The thermography camera 23 can measure the body temperature of a person in the specified area 1 without contact.

[0072] The temperature sensor 24 is installed in the predetermined area 1 by being built into the monitored person terminal 12, and detects and measures the air temperature inside the predetermined area 1. The air temperature measured by the temperature sensor 24 is used to control the air conditioner 5 installed in the predetermined area 1.

[0073] The humidity sensor 25 is installed in the predetermined area 1 by being built into the monitored person terminal 12, and detects and measures the humidity inside the predetermined area 1. The humidity measured by the humidity sensor 25 is used to control the air conditioner 5 installed in the predetermined area 1.

[0074] The smart remote control unit 26 is capable of collectively controlling a plurality of home electrical appliances that enable remote control via infrared communication. The smart remote control unit 26 is built into the monitored person terminal 12 and is controlled by a remote control unit 48 (described later) of the anomaly detection system 10, and mainly controls the air conditioners 5 and lighting devices 6 installed in the predetermined area 1. The air conditioners 5 are controlled by the smart remote control unit 26 to start and stop operation, adjust temperature and humidity, and adjust wind strength and direction. The lighting devices 6 are controlled by the smart remote control unit 26 to turn on and off the lights, adjust illuminance, etc.

[0075] The smart remote control unit 26, when combined with a smart speaker (not shown), enables voice operation of household electrical appliances that can be controlled by the smart remote control unit 26, such as the air conditioner 5 and the lighting device 6. In addition to the air conditioner 5 and the lighting device 6, for example, an automatic curtain opening and closing device can be operated by the smart remote control unit 26 and the smart speaker.

[0076] The bio-motion sensor 3 is a sensor that combines a bio-sensor and a body motion sensor. The bio-motion sensor 3 is inserted under the mattress of the bed 2, or under the futon if no bed or mattress is used. The bio-sensor detects the heart rate and respiratory rate of a sleeping person by detecting and analyzing changes in pressure and vibration caused by the sleeping person. The body motion sensor detects the body movement of the sleeping person and whether or not the person has left the bed by detecting and analyzing changes in pressure and vibration caused by the sleeping person.

[0077] The biological / body movement sensor 3 is suitable for remotely monitoring sleeping people, reducing the burden of nighttime patrols in hospitals, nursing homes and other accommodation facilities, enabling rapid response to deterioration of the monitored person's condition, and further enabling automatic recording for health management and preventive measures.

[0078] (Functional configuration of anomaly detection system 10) The functional configuration of the anomaly detection system 10 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is an example of a block diagram showing the functional configuration of the anomaly detection system 10 according to this embodiment.

[0079] By executing the abnormality detection program described below, the abnormality detection system 10 provides functional units such as a scan data acquisition unit 31, an image data acquisition unit 32, an absence determination unit 33, a scan data determination unit 34, an image data determination unit 35, a determination time acquisition unit 36, a memory control unit 37, a code issuing unit 38, a heat distribution acquisition unit 39, a body temperature acquisition unit 40, a body temperature determination unit 41, a temperature data acquisition unit 42, a temperature determination unit 43, a humidity data acquisition unit 44, a humidity determination unit 45, a body sensation index calculation unit 46, a body sensation index determination unit 47, a remote control unit 48, a biometric information acquisition unit 49, a biometric information determination unit 50, a body movement information acquisition unit 51, a body movement information determination unit 52, an alarm unit 53, a setting information acquisition unit 54, a setting information memory unit 55, and a search unit 56 in the CPU 10e.

[0080] The scan data acquisition unit 31 acquires scan data of the inside of the predetermined area 1 from the three-dimensional sensor 21 . The scan data acquisition unit 31 communicates with the monitored person terminal 12 via the network 20 to acquire scan data obtained by the three-dimensional sensor 21 scanning the inside of the predetermined area 1. The acquired scan data is stored in the RAM 10c or the memory unit 10d of the anomaly detection system 10.

[0081] The imaging data acquisition unit 32 is installed in the predetermined area 1 and acquires imaging data from the imaging unit 22 that captures an image of the interior of the predetermined area 1. The imaging data acquisition unit 32 acquires imaging data obtained by the imaging unit 22 with night vision function built into the monitored person terminal 12 capturing an image of the inside of the predetermined area 1. The acquired imaging data is stored in the RAM 10c or the memory unit 10d of the anomaly detection system 10.

[0082] The absence determination unit 33 determines that no person is present in the specified area 1 based on the difference between the scan data acquired by the scan data acquisition unit 31 and the scan data of the specified area 1 in an unoccupied state that was acquired in advance, when there is no person in the specified area 1. The scan data of the predetermined area 1 in an unmanned state that has been acquired in advance is stored in the RAM 10c or the storage unit 10d of the anomaly detection system 10 in advance. The difference between the scan data acquired by the scan data acquisition unit 31 and the previously acquired scan data of the predetermined area 1 in an unoccupied state is based on the reflected waves that are captured when a person is present in the predetermined area 1 and the waves irradiated from the three-dimensional sensor 21 are reflected by the person before reaching the inner wall of the predetermined area 1. Therefore, the absence determination unit 33 determines that a person is present in the predetermined area 1 when the difference exists, and determines that no person is present in the predetermined area 1 when the difference does not exist.

[0083] The scan data determination unit 34 determines whether or not an abnormality has occurred in a person in the specified area 1 based on the difference between the scan data acquired by the scan data acquisition unit 31 and the scan data of the specified area 1 acquired in advance when the person in the specified area 1 is normal. The scan data judgment unit 34 judges whether or not an abnormality has occurred in a person in the specified area 1 based on the difference between the previously acquired scan data of the specified area 1 when the person in the specified area 1 is normal, thereby enabling accurate judgment. The pre-acquired scan data of the predetermined area 1 when a person in the predetermined area 1 is normal includes reflected waves that are generated when waves irradiated from the three-dimensional sensor 21 are reflected and captured by a person in a normal state in the predetermined area 1. The scan data determination unit 34 determines whether or not an abnormality has occurred in a person in the predetermined area 1 based on the difference between the scan data acquired by the scan data acquisition unit 31 and the scan data that includes reflected waves that are reflected and captured by a person in a normal state.

[0084] The scan data determination unit 34 may input the scan data acquired by the scan data acquisition unit 31 into a scan data learning model that has previously learned the correspondence between the occurrence of abnormalities in a person and the scan data, thereby determining whether or not an abnormality has occurred in the person based on the scan data acquired by the scan data acquisition unit 31.

[0085] The scan data learning model is based on supervised machine learning using a learning dataset annotated with metadata. A machine learning model is an algorithm that, given a large dataset of training data, focuses on specific features to derive and recognize generic patterns, and when new data is input, uses the recognized generic patterns to infer answers for the new data.

[0086] The dataset of training data used in supervised machine learning of the scan data learning model is created as follows. The learning data annotated with an abnormality is generated by adding metadata indicating that an abnormality has occurred to a person in the specified area 1 to scan data from the three-dimensional sensor 21 in a situation where an abnormality has occurred to a person in the specified area 1. The learning data annotated as normal is generated by adding metadata indicating that there are no abnormalities in the people to scan data from the three-dimensional sensor 21 of the specified area 1 in a situation where there are no abnormalities in the people in the specified area 1.

[0087] The abnormality here may be limited to a person falling inside the predetermined area 1 and a person falling off the bed 2. By limiting the range of abnormalities determined by the scan data learning model, the determination accuracy of the scan data learning model can be improved. In this case, the learning data annotated with a person's fall within the predetermined area 1 is generated by adding metadata indicating that a person has fallen within the predetermined area 1 to scan data of the three-dimensional sensor 21 of the predetermined area 1 in a situation where a person in the predetermined area 1 has fallen. The learning data is given various variations by changing the place of fall, the posture before the fall, the posture after the fall, the posture during the fall, and the type of fall, thereby improving the estimation accuracy of the scan data learning model.

[0088] Furthermore, the learning data annotated with a person falling from the bed 2 in the predetermined area 1 is generated by adding metadata indicating that a person has fallen from the bed 2 to the scan data of the three-dimensional sensor 21 of the predetermined area 1 in a situation where a person in the predetermined area 1 has fallen from the bed 2. The learning data is given various variations by changing the place where the person fell from the bed 2, the posture before the fall, the posture during the fall, the posture after the fall, and the type of fall, thereby improving the estimation accuracy of the scan data learning model. The scan data learning model is stored in advance in the memory unit 10d of the anomaly detection system 10.

[0089] The imaging data determination unit 35 inputs the imaging data acquired by the imaging data acquisition unit 32 into an imaging data learning model that has pre-learned the correspondence between the occurrence of abnormalities in a person and the imaging data, and determines whether or not an abnormality has occurred in a person based on the imaging data acquired by the imaging data acquisition unit 32.

[0090] The imaging data learning model is constructed by supervised machine learning, and performs video recognition on the imaging data acquired by the imaging data acquisition unit 32 to detect abnormalities in people present in the predetermined area 1. Video recognition is the process of finding a target object or movement within a video. Since a video is composed of multiple still images, video recognition involves repeatedly performing image recognition on the multiple still images that make up the video.

[0091] The imaging data learning model determines whether or not the imaging data includes an abnormality by performing image recognition on each of the still images constituting the imaging data acquired by the imaging data acquisition unit 32. The imaging data learning model may detect an abnormality and identify the type of abnormality, or may simply determine the presence or absence of an abnormality in a person in the specified area 1 without identifying the type of abnormality. Furthermore, the abnormality in the predetermined area 1 may be limited to a person falling inside the predetermined area 1 and a person falling off the bed 2. By limiting the range of abnormalities determined by the imaging data learning model, the determination accuracy of the imaging data learning model can be improved.

[0092] The construction of an imaging data learning model involves the steps of collecting image data that will serve as learning data, preprocessing the collected image data, selecting features to focus on when performing image recognition, deriving a generic pattern by focusing on the selected features for the preprocessed image data, and evaluating the performance of the derived generic pattern.

[0093] In collecting image data to be used as learning data, images are collected that capture abnormalities in people in a predetermined area 1. When identifying the type of abnormality using an imaging data learning model, annotation is performed by adding metadata for distinguishing the type of abnormality to each of the collected image data.

[0094] When the abnormality in the specified area 1 that is to be judged by the imaging data learning model is limited to a person falling within the specified area 1, images of a person falling in the specified area 1 are collected as learning data for the imaging data learning model. Furthermore, if the abnormalities in the predetermined area 1 to be judged by the imaging data learning model are a person falling inside the predetermined area 1 and a person falling off the bed 2, an image of a person falling in the predetermined area 1 and an image of a person falling off the bed 2 are collected as learning data. When the imaging data learning model is to identify whether the abnormality is a fall or a fall, metadata is added to the image of a person falling in the predetermined area 1 indicating that the type of abnormality is a fall, and metadata is added to the image of a person falling off the bed 2 indicating that the type of abnormality is a fall, and the collected image data is annotated. When the amount of image data collected as training data is insufficient, data augmentation is performed to increase the amount of training data. Data augmentation is a technique for augmenting the training image data by applying transformations to the data. Transformations include image rotation, left / right translation, up / down translation, enlargement, reduction, shearing, left / right flipping, and up / down flipping. Shearing refers to transforming a rectangular image into a parallelogram.

[0095] Preprocessing of collected image data refers to correcting the image data so that the computer can easily recognize it as learning data. Specifically, preprocessing involves making the contours of the image data clearer if they are blurred, and correcting the rotation and tilt if they are distorted diagonally.

[0096] In selecting features to focus on in image recognition, feature points are detected from image data provided as learning data, and feature values ​​are derived and selected from the pixel values ​​and differential values ​​of those points. Examples of algorithms for deriving features in image recognition include SIFT (Scale-Invariant Feature Transform) and HOG (Histograms of Oriented Gradients).

[0097] In deriving a generic pattern by focusing on features selected from preprocessed image data, an imaging data learning model is constructed by deriving a generic pattern based on the selected features from a large amount of image data that serves as given learning data.

[0098] In the performance evaluation of the derived general-purpose pattern, the constructed image data learning model is quantitatively evaluated in terms of recognition accuracy, processing speed, etc. In the performance evaluation, image data different from the learning data is prepared in advance as test data and the evaluation is performed.

[0099] When the scan data determination section 34 determines that an abnormality has occurred in a person, the determination time acquisition section 36 acquires the time at which the scan data determination section 34 made the determination as the determination time. The determination time is measured using an internal clock built into the anomaly detection system 10. The internal clock maintains its accuracy by synchronizing with an accurate time source on the Internet.

[0100] The storage control unit 37 stores the imaging data acquired by the imaging data acquisition unit 32 for a predetermined period starting from a predetermined time before the determination time acquired by the determination time acquisition unit 36 ​​in the storage unit 10d. The "predetermined time before" refers to a few seconds to a few tens of seconds before the judgment time, and the purpose is to store image data from a little before the time when the abnormality occurred in the memory unit 10d of the abnormality detection system 10. The predetermined period is, for example, five minutes. The storage control unit 37 stores, for example, five minutes of imaging data from 30 seconds before the determination time in the storage unit 10d. The observer can view the five minutes of imaging data stored in the storage unit 10d.

[0101] The code issuing unit 38 issues an access code for accessing the imaging data stored by the storage control unit 37 in the storage unit 10d. The access code is a Uniform Resource Locator (URL). By clicking the access code on the monitor terminal 11, the monitor can transition to the URL linked to the access code and can play back the imaging data stored in the storage unit 10d of the anomaly detection system 10.

[0102] The heat distribution acquisition unit 39 acquires the heat distribution inside the predetermined area 1 as heat distribution data by capturing an image of the inside of the predetermined area 1 using the thermography camera 23 installed in the predetermined area 1. The thermal distribution data refers to thermal image data of the heat distribution inside a specified area 1. The thermal image is obtained by detecting infrared energy naturally radiated from various parts of the surface of an object using a thermographic camera 23, processing the image, converting it into temperature values, and then displaying the temperature distribution image in black and white brightness or pseudo-color color distribution. Based on the heat distribution data, the observer can identify the location of a person in the predetermined area 1 and can measure the body surface temperature of the person.

[0103] The body temperature acquiring section 40 acquires the body temperature of a person present in the predetermined area 1 based on the heat distribution data acquired by the heat distribution acquiring section 39 . The body temperature determination unit 41 determines that an abnormality has occurred in the person when the body temperature of the person acquired by the body temperature acquisition unit 40 exceeds a threshold value. For example, if the threshold temperature of the body temperature is set to 37.5°C, the body temperature determining section 41 determines that an abnormality has occurred in the person when the body temperature of the person acquired by the body temperature acquiring section 40 exceeds 37.5°C.

[0104] The temperature data acquisition unit 42 acquires the air temperature inside the predetermined area 1 detected by the temperature sensor 24 that detects the air temperature inside the predetermined area 1 as temperature data. The temperature determination unit 43 determines that an abnormality has occurred in the predetermined area 1 when the temperature data acquired by the temperature data acquisition unit 42 exceeds a threshold value. The temperature data thresholds are the lower and upper limits of the air temperature. The temperature determination unit 43 determines that an abnormality has occurred in the specified area 1 when the temperature data acquired by the temperature data acquisition unit 42 is below the lower limit or when the temperature data is above the upper limit. The threshold value of the temperature data is changed depending on the season. For example, if it is summer and the temperature inside the predetermined area 1 exceeds 28°C, the temperature determination unit 43 determines that an abnormality has occurred inside the predetermined area 1. Also, if it is winter and the temperature inside the predetermined area 1 falls below 16°C, the temperature determination unit 43 determines that an abnormality has occurred inside the predetermined area 1.

[0105] The humidity data acquisition unit 44 acquires the humidity inside the predetermined area 1 detected by the humidity sensor 25 that detects the humidity inside the predetermined area 1 as humidity data. The humidity determination unit 45 determines that an abnormality has occurred in the predetermined area 1 when the humidity data acquired by the humidity data acquisition unit 44 exceeds a threshold value. The humidity data thresholds are the lower and upper limits of humidity. The humidity determination unit 45 determines that an abnormality has occurred in the specified area 1 when the humidity data acquired by the humidity data acquisition unit 44 is below the lower limit or when the humidity data is above the upper limit. The humidity data threshold changes depending on the season. In summer, a humidity of 50-60% is considered optimal, and in winter, a humidity of 40-50% is considered optimal. The humidity data threshold is set so that the lower limit is 50% and the upper limit is 60% in summer, and the lower limit is 40% and the upper limit is 50% in winter.

[0106] The thermal comfort index calculation unit 46 calculates the thermal comfort index of the person in the predetermined area 1 based on the temperature data acquired by the temperature data acquisition unit 42 and the humidity data acquired by the humidity data acquisition unit 44. The thermal sensation index is at least one of a discomfort index and a thermal sensation temperature. The discomfort index is a temperature sensation expressed in degrees Fahrenheit, calculated from a combination of temperature and humidity. The sensible temperature is an attempt to quantitatively express the sensation of body temperature, which is generally expressed as hot or cold.

[0107] The Discomfort Index (DI) is calculated using the following formula (1).

[0108]

number

[0109] The sensible temperature is calculated using the following formula (2). The Misnar sensible temperature (Tm) is used as the sensible temperature. The Misnar sensible temperature is a method of calculating the sensible temperature from three elements: room temperature, humidity, and wind speed.

[0110]

number

[0111] Here, t is temperature (°C), h is humidity (%), v is wind speed (m / s), and A is a correction value based on the wind speed. In this embodiment, since it is assumed that the predetermined area 1 is indoors, the wind speed v is set to zero, and the correction value A based on the wind speed is 1.76.

[0112] The body sensation index determining unit 47 determines that an abnormality has occurred in the predetermined area 1 when the body sensation index of the person calculated by the body sensation index calculating unit 46 exceeds a threshold value. The thresholds of the thermal index are the lower and upper limits of the thermal index. When the discomfort index is used as the thermal index, for example, the lower limit of the thermal index threshold is set to 60 and the upper limit to 70. In an environment with a discomfort index of 60 or less, people feel chilly, and in an environment with a discomfort index of 70 or more, people start to feel hot.

[0113] Even at the same air temperature, the lower the humidity, the lower the sensible temperature. For example, in summer, if the sensible temperature inside the predetermined area 1 exceeds 28°C, the sensation index determination unit 47 determines that an abnormality has occurred in the predetermined area 1. In winter, if the sensible temperature inside the predetermined area 1 falls below 16°C, the sensation index determination unit 47 determines that an abnormality has occurred in the predetermined area 1.

[0114] The remote control unit 48 controls the smart remote control unit 26 that operates and monitors the home appliances within the predetermined area 1 . The household electrical appliance is an air conditioner 5 that conditions the air within a specified area 1, and when at least one of the temperature determination unit 43, the humidity determination unit 45, and the body sensation index determination unit 47 determines an abnormality, the remote control unit 48 controls the air conditioner 5 so that the temperature and humidity inside the specified area 1 become preset temperature and humidity. The remote control unit 48 controls the air conditioner 5 via the smart remote control unit 26 of the monitored person terminal 12.

[0115] The biometric information acquiring unit 49 acquires, as biometric information data, biometric information of a person detected by a biometric sensor that detects biometric information of the person while sleeping. The biometric information determination unit 50 determines that an abnormality has occurred in the person when the biometric information data acquired by the biometric information acquisition unit 49 exceeds a threshold value of the biometric information. The biological information refers to a person's heart rate and respiratory rate while sleeping. The biosensor is a sensor included in the biosensor / body movement sensor 3. The biosensor acquisition unit 49 acquires the biosensor of the person detected by the biosensor / body movement sensor 3 installed on the bed 2 as biosensor data.

[0116] The thresholds of the biological information are the lower and upper limits of the biological information. For example, when the biological information is a person's breathing rate while sleeping, the lower limit threshold of the breathing rate is set to 10 times / minute. In this case, the biological information determination unit 50 determines that an abnormality has occurred in the person when the heart rate while sleeping throughout the day falls below 10 times / minute.

[0117] When the biometric information is, for example, a person's heart rate while sleeping, the upper threshold of the heart rate is set to a value that exceeds 100 beats / min for 10 or more consecutive minutes, and the lower threshold is set to a value that falls below 40 beats / min. In this case, the biometric information determination unit 50 determines that an abnormality has occurred in the person when the heart rate while sleeping exceeds 100 beats / min for 10 or more consecutive minutes, and when the heart rate while sleeping falls below 40 beats / min.

[0118] In addition, the biometric information acquisition unit 49 may use either or both of the scan data from the three-dimensional sensor 21 or thermal distribution data regarding the thermal distribution inside the specified area 1 acquired by imaging the inside of the specified area 1 using the thermographic camera 23 installed in the specified area 1 as actual data, and acquire virtual biometric information data regarding the person's biometric information while sleeping based on the newly acquired actual data by inputting the newly acquired actual data into a virtual biometric sensor model that has previously learned the correspondence between a person's biometric information while sleeping and the actual data.

[0119] This is a technique in which a biosensor (biome / body movement sensor3) is used as a virtual sensor instead of actually using a biosensor. A virtual sensor is a term used in contrast to sensors that have physical hardware such as normal measuring instruments, and refers to a technique for estimating state quantities that are difficult to measure directly with an actual sensor in real time from measured values ​​of variables that are relatively easy to measure online, and is also called a virtual sensor, soft sensor, estimation sensor, state estimator, virtual metrology, etc. Virtual biometric data is biometric data regarding a person's biometric information while sleeping that is acquired by a virtual sensor, and is obtained by inputting newly acquired actual data into a virtual biometric sensor model to estimate a person's biometric information while sleeping.

[0120] A model for estimating a state quantity that is difficult to measure directly may be constructed by machine learning or a statistical method. In this embodiment, machine learning is used to construct a model for estimating a person's biological information while sleeping, and a virtual biological sensor model is constructed in which the correspondence between the biological information of a person while sleeping and real data is previously machine-learned. The real data is data actually measured in real time, and in this embodiment, it refers to real-time scan data from the three-dimensional sensor 21, real-time heat distribution data from the thermography camera 23, or a combination of real-time scan data from the three-dimensional sensor 21 and real-time heat distribution data from the thermography camera 23.

[0121] When the biometric information is a person's heart rate and respiration rate while asleep, a virtual biometric sensor model that represents the correlation (correspondence) between the above-mentioned actual data and the heart rate and respiration rate is constructed by machine learning. The virtual biosensor model, for example, performs machine learning on a large number of data sets consisting of combinations of past scan data from the three-dimensional sensor 21 and past heart rates at corresponding times, and estimates the heart rate based on newly acquired scan data from the three-dimensional sensor 21. In addition, the virtual biosensor model may be, for example, a model that pre-learns a large number of data sets consisting of combinations of past thermal distribution data from the thermographic camera 23 and past heart rates at corresponding times, and estimates the heart rate based on newly acquired thermal distribution data from the thermographic camera 23. In addition, the virtual biosensor model may, for example, be a model that pre-machines a large number of data sets consisting of combinations of past scan data from the three-dimensional sensor 21 and heat distribution data from the thermographic camera 23, and the heart rate at a corresponding point in time in the past, and estimates the heart rate based on a newly acquired combination of scan data from the three-dimensional sensor 21 and heat distribution data from the thermographic camera 23.

[0122] Furthermore, the virtual biosensor model, for example, machine-learns a large number of data sets consisting of combinations of past scan data from the three-dimensional sensor 21 and the respiratory rate at a corresponding point in time in the past, and estimates the respiratory rate based on newly acquired scan data from the three-dimensional sensor 21. In addition, the virtual biosensor model may be, for example, a model that pre-learns a large number of data sets consisting of combinations of past thermal distribution data from the thermographic camera 23 and the respiration rate at a corresponding point in time in the past, and estimates the respiration rate based on newly acquired thermal distribution data from the thermographic camera 23. In addition, the virtual biosensor model may, for example, be a model that pre-machines a large number of data sets consisting of a combination of past scan data from the three-dimensional sensor 21 and thermal distribution data from the thermographic camera 23, and a combination of the respiration rate at a corresponding point in time in the past, and estimates the respiration rate based on a newly acquired combination of scan data from the three-dimensional sensor 21 and thermal distribution data from the thermographic camera 23.

[0123] The model (virtual biosensor model) for estimating a person's bioinformation while sleeping is not limited to being constructed by machine learning, but can also be constructed by a statistical method. As an example of a statistical method, a virtual biosensor model may be constructed by multiple regression analysis with the heart rate as the objective variable. The explanatory variables are extracted from the past scan data of the three-dimensional sensor 21, and for example, the frequency or peak value of the waveform is selected. The explanatory variables may also be extracted from the past thermal distribution data of the thermographic camera 23, or may be extracted from a combination of the past scan data of the three-dimensional sensor 21 and the past thermal distribution data of the thermographic camera 23.

[0124] As another example of a statistical method, a virtual biosensor model may be constructed by multiple regression analysis with the respiratory rate as the objective variable. The explanatory variables are the same as those in the case of constructing a virtual biosensor model by multiple regression analysis with the respiratory rate as the objective variable.

[0125] In constructing a virtual biosensor model, if the model is a linear model, or if the model is a nonlinear model but approximation to a linear model is effective, the model can be constructed by a linear regression method such as multiple regression analysis. Furthermore, if the model is a nonlinear model, machine learning can be used to construct the model.

[0126] In this case, the biometric information determination unit 50 determines that an abnormality has occurred in the person when the virtual biometric information data acquired by the biometric information acquisition unit 49 exceeds the threshold value of the biometric information.

[0127] The body movement information acquisition unit 51 acquires, as body movement information data, body movement information detected by a body movement sensor that detects vibrations caused by a person's body movements while sleeping. Body movement information determination section 52 determines that an abnormality has occurred in the person when the body movement information data acquired by body movement information acquisition section 51 indicates a state in which there is no body movement for a predetermined period of time or longer. The body movement sensor is a sensor included in the living body / body movement sensor 3. The body movement information acquisition unit 51 acquires the body movement information of the person detected by the living body / body movement sensor 3 installed on the bed 2 as body movement information data. Body movement refers to the movement of the body. Body movement may occur when awakening occurs during light sleep, and body movement also occurs with breathing and swallowing during sleep. The body movement information determination unit 52 determines that an abnormality has occurred in the person when, for example, a state in which there is no body movement lasts for 10 minutes or more in body movement information data acquired by detecting body movement information using the biological / body movement sensor 3.

[0128] The body movement information acquisition unit 51 may use either or both of the scan data from the three-dimensional sensor 21 or thermal distribution data regarding the heat distribution inside the specified area 1 acquired by imaging the inside of the specified area 1 using the thermographic camera 23 installed in the specified area 1 as actual data, and acquire virtual body movement information data regarding vibrations caused by a person's body movements while sleeping based on the newly acquired actual data by inputting the newly acquired actual data into a virtual body movement sensor model that has previously learned the correspondence between vibrations caused by a person's body movements while sleeping and the actual data.

[0129] This is a method in which, instead of using an actual body motion sensor (living body motion sensor 3), the body motion sensor is used as a virtual sensor. Virtual body movement information data is body movement information data regarding vibrations caused by a person's body movements while sleeping, which is acquired by a virtual sensor, and refers to virtual body movement information data acquired by inputting newly acquired actual data into a virtual body movement sensor model to estimate body movement information regarding vibrations caused by a person's body movements while sleeping. In this embodiment, machine learning is used to construct a model for estimating a person's body movement information while sleeping, and a virtual body movement sensor model is constructed by previously learning the correspondence between a person's body movement information while sleeping and actual data by machine learning.

[0130] The virtual body movement sensor model uses machine learning to construct a correlation (correspondence) between the above-mentioned actual data and vibrations caused by a person's body movements while sleeping. The virtual body motion sensor model, for example, learns in advance a large number of data sets consisting of combinations of past scan data from the three-dimensional sensor 21 and vibration behavior such as vibration frequency and vibration strength at a given point in time in the past, and estimates the vibration behavior (frequency, vibration strength, etc.) based on newly acquired scan data from the three-dimensional sensor 21. In addition, the virtual body movement sensor model may be one that, for example, pre-learns a large number of data sets consisting of combinations of past thermal distribution data from the thermographic camera 23 and the vibration behavior (frequency, vibration strength, etc.) at a corresponding point in time in the past, and estimates the vibration behavior (frequency, vibration strength, etc.) based on newly acquired thermal distribution data from the thermographic camera 23. In addition, the virtual body movement sensor model may be a model that, for example, pre-learns a large number of data sets consisting of combinations of past scan data from the three-dimensional sensor 21 and thermal distribution data from the thermographic camera 23, and combinations of vibration behavior (frequency, vibration strength, etc.) at a given point in time in the past, and estimates the vibration behavior (frequency, vibration strength, etc.) based on newly acquired combinations of scan data from the three-dimensional sensor 21 and thermal distribution data from the thermographic camera 23.

[0131] Note that the model (virtual body movement sensor model) for estimating the body movement information of a person while sleeping is not limited to being constructed by machine learning, but can also be constructed by a statistical method. As an example of a statistical method, the virtual body movement sensor model may be constructed by multiple regression analysis with the vibration mode (frequency, vibration strength, etc.) as the objective variable. The explanatory variables are selected from, for example, the vibration frequency or vibration strength extracted from the past scan data of the three-dimensional sensor 21. In addition, the explanatory variables may be extracted from the past heat distribution data of the thermographic camera 23, or may be extracted from a combination of the past scan data of the three-dimensional sensor 21 and the past heat distribution data of the thermographic camera 23.

[0132] When constructing a virtual body motion sensor model, if the model is a linear model, or if the model is a nonlinear model but approximation to a linear model is effective, the model can be constructed using a linear regression method such as multiple regression analysis, just as in the case of a virtual biological sensor model. Furthermore, if the model is a nonlinear model, machine learning can be used to construct the model, just like in the case of the virtual biosensor model.

[0133] In this case, the body movement information determining section 52 determines that an abnormality has occurred in the person when the virtual body movement information data acquired by the body movement information acquiring section 51 indicates that there is no body movement for a predetermined period of time or longer.

[0134] As described above, when the biological / body motion sensor 3 is a virtual sensor, there is no need to install the biological / body motion sensor 3 on the bed of the person to be monitored, and therefore there is no risk of the biological / body motion sensor 3 being removed intentionally or inadvertently by the person. Furthermore, there is no need to prepare a separate biological / body motion sensor 3, which contributes to cost reduction. Furthermore, performance improvement of the biological / body motion sensor 3 can be achieved by simply improving the software by improving the virtual biological sensor model and the virtual body motion sensor model. Furthermore, by improving the software related to the virtual body motion sensor model, it is possible to reflect the improvement collectively in the rooms of multiple people to be monitored.

[0135] When the scan data determination unit 34 determines that an abnormality has occurred in a person in the predetermined area 1, the notification unit 53 notifies a supervisor of the predetermined area 1 that an abnormality has occurred in the person. The notification unit 53 notifies the monitor in the predetermined area 1 of the abnormality by notifying the monitor terminal 11 via the network 20 that an abnormality has occurred to a person. The notification method may be a push notification, a notification by email, or a push notification or an alert sound by a ringtone when an email arrives. The notification unit 53 also notifies the monitor terminal 11 via the network 20 that an abnormality has occurred in a person, as will be described below, thereby notifying the monitor in the predetermined area 1 of the abnormality.

[0136] When absence determination unit 33 determines that a person is absent, notification unit 53 notifies the monitor of the predetermined area 1 of the absence of a person in the predetermined area 1. The monitor can notice that a person is absent from the predetermined area 1, and can also notice that the person is absent during a time period when the person should be in the predetermined area 1, such as when the person is absent during bedtime, thereby enabling the monitor to notice something unusual about the person at an early stage. The notification unit 53 notifies the supervisor in the predetermined area 1 that an abnormality has occurred with a person, together with the access code issued by the code issuing unit 38. The supervisor who has received the notification can easily access the imaging data stored in the memory unit 10d by the memory control unit 37, by using the link of the access code.

[0137] When the body temperature determination unit 41 determines that there is an abnormality, the notification unit 53 notifies the supervisor in the predetermined area 1 that something abnormal has happened to the person. By receiving the notification, the supervisor can notice that something abnormal has happened to the body temperature of the person, and can rush to the person's side early to directly check the person's condition, etc.

[0138] When the temperature determination unit 43 determines that an abnormality has occurred, the notification unit 53 notifies a person watching the predetermined area 1 that an abnormality has occurred in the predetermined area 1. When the humidity determination section 45 determines that an abnormality has occurred, the notification section 53 notifies a person watching the predetermined area 1 that an abnormality has occurred in the predetermined area 1. When the sensation index determination unit 47 determines that an abnormality has occurred, the notification unit 53 notifies the person watching the predetermined area 1 that an abnormality has occurred in the predetermined area 1. When the temperature determination unit 43, the humidity determination unit 45, or the physical sensation index determination unit 47 determines an abnormality, the monitor is notified that an abnormality has occurred in the specified area 1, and is able to notice that an abnormality has occurred in the temperature, humidity, or physical sensation index in the specified area 1, and can contact someone in the specified area 1 or go to the specified area 1 to directly check for the abnormality, thereby preventing the abnormal situation from spreading.

[0139] When the biological information determination section 50 determines that there is an abnormality, the notification section 53 notifies the observer in the predetermined area 1 that an abnormality has occurred in the person. When the body movement information determination section 52 determines that there is an abnormality, the notification section 53 notifies the observer in the predetermined area 1 that an abnormality has occurred in the person. When the vital sign information determination unit 50 or the body movement information determination unit 52 determines an abnormality, the monitor is notified that something abnormal has occurred in the person, thereby enabling the monitor to notice that something abnormal has occurred in the vital signs or body movements of a person in the specified area 1, and to rush to the person's location early and directly check on their condition, etc.

[0140] The contents of the push notification and the text to be written in the e-mail when an abnormality is determined are prepared in advance according to the nature of the abnormality. For example, the contents of the text when the scan data determination unit 34 determines an abnormality may be "An abnormality has been detected in room XX by scanning with a three-dimensional sensor." Also, the contents of the document when the absence determination unit 33 determines an abnormality may be "A three-dimensional sensor scan has detected that no one is present in room XX."

[0141] Furthermore, when the body temperature determination unit 41 determines that there is an abnormality, the content of the sentence may be "An abnormality in the body temperature of a person in room XX has been detected." When the temperature determination unit 43 determines that there is an abnormality, the content of the sentence may be "An abnormality in the temperature of room XX has been detected." When the humidity determination unit 45 determines that there is an abnormality, the content of the sentence may be "An abnormality in the humidity of room XX has been detected."

[0142] Furthermore, when the body sensation index determination unit 47 determines that there is an abnormality, the content of the sentence may be "An abnormality has been detected in the body sensation index in room XX." When the biometric information determination unit 50 determines that there is an abnormality, the content of the sentence may be "An abnormality has been detected in the biometric information of a person in room XX." When the body movement information determination unit 52 determines that there is an abnormality, the content of the sentence may be "An abnormality has been detected in the body movement information of a person in room XX."

[0143] The notification unit 53 may have an automatic telephone call function, and when any of the scan data determination unit 34, absence determination unit 33, body temperature determination unit 41, temperature determination unit 43, humidity determination unit 45, body sensation index determination unit 47, vital information determination unit 50, and body movement information determination unit 52 determines an abnormality, may call a monitor in the predetermined area 1 to notify the abnormality. The call may be a regular phone or an IP phone. In addition, the notification unit 53 may notify the monitor of the abnormality by voice over the telephone, or, instead of by voice over the telephone, may notify the monitor of the abnormality by a social networking service (SNS) such as LINE (registered trademark) or a text message as a short mail.

[0144] The setting information acquisition unit 54 acquires setting information input to the smart remote control unit 26 by a person present in the predetermined area 1. The setting information acquisition unit 54 acquires, via the network 20, setting information of the smart remote control unit 26 that has already been input and is being used by a person in the predetermined area 1.

[0145] The smart remote control unit 26 can aggregate the operations of home electrical appliances that can be remotely controlled by infrared communication of a remote controller, and has a sensor function, a timer function, a macro function, a preset function, and the like. The smart remote control unit 26 can receive operations from the anomaly detection system 10 via the network 20, and can also receive operations from information processing devices such as pre-registered PCs, notebook PCs, tablet PCs, or smartphones via a connection via Wi-Fi or Bluetooth (registered trademark). Therefore, the anomaly detection system 10 and the information processing devices can operate household electrical appliances in the predetermined area 1 via the smart remote control unit 26.

[0146] The sensor function can control household electrical appliances when the temperature, humidity, and thermal index in the specified area 1 acquired by the temperature data acquisition unit 42, humidity data acquisition unit 44, and thermal index calculation unit 46 reach specified values. For example, the sensor function can be set to control the air conditioning equipment 5, such as starting operation of the air conditioning equipment 5, when the temperature in the specified area 1 reaches 20°C.

[0147] The timer function is a function that can be set to start or stop the operation of a household electrical appliance at a specified time. For example, the timer function can be set to turn on the lighting device 6 at 8:00 a.m. every day and to turn off the lighting device 6 at 11:00 p.m. every day.

[0148] The macro function sets sequence control, which is control that sequentially advances each stage of control of household electrical appliances according to a predetermined order, and can be set, for example, to start operation of air conditioners 5 at a specified time and turn on lighting devices 6 five minutes later. The macro function can also be set to turn off lighting devices 6 three minutes after operation of air conditioners 5 is stopped.

[0149] The preset function is a function used when performing various settings such as the sensor function, timer function, and macro function of the smart remote control unit 26, and is a function that allows the smart remote control unit 26 to be set based on various setting information registered in advance in the storage unit 10d of the anomaly detection system 10 by inputting the model number of the home electric appliance. As described above, the smart remote control unit 26 allows a wide variety of settings for utilizing the sensor function, timer function, and macro function, and the use of the preset function can eliminate the hassle of performing such a wide variety of settings. The model number of a home electrical appliance is a series of symbols, letters, numbers, etc. that identify the manufacturer and specifications of the home electrical appliance.

[0150] The setting information storage unit 55 separates the setting information acquired by the setting information acquisition unit 54 for each model number assigned to each household electric appliance, and stores the information in the storage unit 10d. The setting information acquired by the setting information acquisition unit 54 may be classified by model number of the household electrical appliance, and may be further subdivided from a viewpoint other than the model number. The model number classifies the household electrical appliance based on the manufacturer and the specifications. The setting information collected with the same model number may be further subdivided from a technical viewpoint such as function, purpose, and use. For example, the setting information acquired by the setting information acquisition unit 54 for one lighting device 6 may be further subdivided into setting information regarding lighting, setting information regarding turning off, setting information regarding brightness, setting information regarding energy saving, and setting information regarding sequence control in cooperation with other household electrical appliances.

[0151] Furthermore, the storage unit 10d that stores the setting information acquired by the setting information acquisition unit 54 may be constructed using a hierarchical database. A hierarchical database is a database that accumulates data associated in a tree shape. By storing the setting information in the hierarchical database, there is only one route from the upper layers to specific setting information, so that the time required to reach the desired setting information during a search is short, resulting in excellent search speed.

[0152] The search unit 56 searches for and identifies the setting information stored in the storage unit 10d based on the model number entered by the person in the other predetermined area 1. If the setting information stored in memory unit 10d is classified by model number and further subdivided from the technical viewpoints as described above, even if the number of setting information identified based on the model number is large, the desired setting information can be identified based on the subdivided technical viewpoints at a lower level, and the desired setting information can be reached in a short time. The setting information identified by the search unit 56 is used for setting the smart remote control unit 26 in another predetermined area 1. The setting information identified by the search unit 56 can be used to set the smart remote control units 26 in other predetermined areas 1, and the troublesome setting of the smart remote control units 26 can be reduced.

[0153] (Display screen of monitor terminal 11) The display screens 60, 80 of the monitor terminal 11 will be described with reference to Fig. 6 and Fig. 7. Fig. 6 and Fig. 7 are diagrams showing examples of the display screens 60, 80 of the monitor terminal 11 according to this embodiment. 6 shows a display screen 60 of the monitor's terminal 11. The display screen 60 is based on the biological information and body movement information detected by the biological information / body movement sensor 3. The monitor's terminal 11 is a notebook PC. The display screen 60 displays a list 61 of the status of each room to be monitored. Each of the multiple rooms to be monitored corresponds to a predetermined area 1.

[0154] As examples of the list 61 of the status of each room, rooms 101, 102, 103, and 305 will be described. The status display 62 of room 101 indicates that the person being monitored is Aaaa and is currently out of bed. The status display 63 of room 102 indicates that the person being monitored is Bbbb and is currently asleep, with a current heart rate 58 of 64 (beats / min) and a current respiratory rate 59 of 29 (beats / min). The status display 64 of room 103 indicates that the person being monitored is Cccc and is currently asleep, with a current heart rate 58 of 63 (beats / min) and a current respiratory rate 59 of 21 (beats / min). The status display 65 for Room 305 indicates that the person being monitored is Mmmm, is currently moving about as if asleep, and that his or her current heart rate 58 is 83 (beats per minute) and respiratory rate 59 is 18 (beats per minute).

[0155] Next, the personal status display window 71 on the right side of the display screen 60 will be described. The biological information and body movement information of the monitored person in the room selected from the list 61 of the status of each room is displayed.

[0156] The biological information and body movement information of Aaaa in room 101 are displayed in the individual status display window 71. In the status display window 71, the biological information and body movement information for a predetermined period are displayed in three sections, upper, middle and lower.

[0157] The upper part of the status display window 71 displays the status display 72 for the last hour. The status display 72 displays a sleeping status display bar 75, and graphs of a heart rate 77 and a respiratory rate 78. The sleeping status display bar 75 displays the statuses of getting out of bed 87, moving 88, and lying down 89 so that they can be identified for each time period.

[0158] In the middle of the status display window 71, a status display 73 for the current month is displayed. The state display 73 displays a sleeping state display graph 76, and graphs of a heart rate 77 and a respiratory rate 78. The sleeping state display graph 76 displays the states of getting out of bed 87, moving 88, and lying down 89 so that they can be identified for each time period.

[0159] At the bottom of the status display window 71, a status display 74 for the previous month is displayed. The state display 74 displays a sleeping state display graph 76, and graphs of a heart rate 77 and a respiratory rate 78. The sleeping state display graph 76 displays the states of getting out of bed 87, moving 88, and lying down 89 so that they can be identified for each time period.

[0160] The display of the list 61 of the status of each room can be switched by using the page selection tab 70. By using the page selection tab 70, for example, it is possible to display a list of only rooms for which an alert is currently being displayed. Moreover, by clicking a print button 68, the display screen 60 can be printed out.

[0161] FIG. 7 shows an image data display screen 90 for each room, which is displayed on the monitor terminal 11. The imaging data display screen 90 displays imaging data currently captured by the imaging unit 22 with night vision function for each room. The surveillance person can see the current state of each room by looking at the imaging data display screen 90. Imaging data displays 91 to 93 show imaging data for room 101, room 102, and room 103, respectively.

[0162] Next, an anomaly detection program according to an embodiment of the present invention will be described together with an anomaly detection method with reference to Fig. 8. Fig. 8 is a flowchart of an anomaly detection program according to an embodiment of the present invention. The anomaly detection method is executed by the CPU 10e of the anomaly detection system 10 based on the anomaly detection program.

[0163] The abnormality detection program includes a scan data acquisition step S31, an image data acquisition step S32, an absence determination step S33, a scan data determination step S34, an image data determination step S35, a determination time acquisition step S36, a memory control step S37, a code issuance step S38, a heat distribution acquisition step S39, a body temperature acquisition step S40, a body temperature determination step S41, a temperature data acquisition step S42, a temperature determination step S43, a humidity data acquisition step S44, a humidity determination step S45, a body sensation index calculation step S46, a body sensation index determination step S47, a remote control step S48, a biometric information acquisition step S49, a biometric information determination step S50, a body movement information acquisition step S51, a body movement information determination step S52, an alarm step S53, a setting information acquisition step S54, a setting information storage step S55, and a search step S56.

[0164] The anomaly detection program causes the CPU 10e of the anomaly detection system 10 to realize various functions such as a scan data acquisition function, an image data acquisition function, an absence determination function, a scan data determination function, an image data determination function, a determination time acquisition function, a memory control function, a code issuing function, a heat distribution acquisition function, a body temperature acquisition function, a body temperature determination function, a temperature data acquisition function, a temperature determination function, a humidity data acquisition function, a humidity determination function, a body sensation index calculation function, a body sensation index determination function, a remote control function, a biological information acquisition function, a biological information determination function, a body movement information acquisition function, a body movement information determination function, a notification function, a setting information acquisition function, a setting information storage function, and a search function. These functions are executed in the order shown in the flowchart of FIG. 8, but the order can be changed as appropriate. Note that each function overlaps with the description of the various functional units of the anomaly detection system 10 described above, and therefore detailed descriptions thereof will be omitted.

[0165] The scan data acquisition function acquires scan data of the inside of predetermined area 1 from three-dimensional sensor 21 (S31: scan data acquisition step). The imaging data acquisition function acquires imaging data from the imaging unit 22 that is installed in the predetermined area 1 and captures an image of the interior of the predetermined area 1 (S32: imaging data acquisition step). The absence determination function determines that there is no person in the specified area 1 when there is no person in the specified area 1 based on the difference between the scan data acquired by the scan data acquisition function and the scan data of the specified area 1 in an unoccupied state that was acquired in advance (S33: absence determination step). The scan data determination function determines whether or not an abnormality has occurred in a person in the specified area 1 based on the difference between the scan data acquired by the scan data acquisition function and the scan data of the specified area 1 acquired in advance when the person in the specified area 1 is normal (S34: scan data determination step).

[0166] The imaging data determination function inputs the imaging data acquired by the imaging data acquisition function into an imaging data learning model that has previously learned the correspondence between the occurrence of abnormalities in a person and the imaging data, and determines whether or not an abnormality has occurred in the person based on the imaging data acquired by the imaging data acquisition function (S35: imaging data determination step). When the scan data determination function determines that an abnormality has occurred in a person, the determination time acquisition function acquires the time at which the scan data determination function made the determination as the determination time (S36: determination time acquisition step). The storage control function stores the imaging data acquired by the imaging data acquisition function for a predetermined period from a predetermined time before the determination time acquired by the determination time acquisition function in the storage unit 10d (S37: storage control step). The code issuing function issues an access code for accessing the imaging data stored in the storage unit 10d in the storage control function (S38: code issuing step).

[0167] The heat distribution acquisition function acquires the heat distribution inside the specified area 1 as heat distribution data by capturing an image of the inside of the specified area 1 using a thermographic camera 23 installed in the specified area 1 (S39: heat distribution acquisition step). The body temperature acquisition function acquires the body temperature of a person in the predetermined area 1 based on the heat distribution data acquired by the heat distribution acquisition function (S40: body temperature acquisition step). The body temperature determination function determines that an abnormality has occurred in the person when the body temperature of the person acquired by the body temperature acquisition function exceeds a threshold value (S41: body temperature determination step). The temperature data acquisition function acquires the temperature inside the predetermined area 1 detected by the temperature sensor 24 that detects the temperature inside the predetermined area 1 as temperature data (S42: temperature data acquisition step). The temperature determination function determines that an abnormality has occurred in the predetermined area 1 when the temperature data acquired by the temperature data acquisition function exceeds a threshold value (S43: temperature determination step).

[0168] The humidity data acquisition function acquires the humidity inside the predetermined area 1 detected by the humidity sensor 25 that detects the humidity inside the predetermined area 1 as humidity data (S44: humidity data acquisition step). The humidity determination function determines that an abnormality has occurred in the predetermined area 1 when the humidity data acquired by the humidity data acquisition function exceeds a threshold value (S45: humidity determination step). The physical sensation index calculation function calculates a physical sensation index of a person in the predetermined area 1 based on the temperature data acquired by the temperature data acquisition function and the humidity data acquired by the humidity data acquisition function (S46: physical sensation index calculation step). The body sensation index determining function determines that an abnormality has occurred in the predetermined area 1 when the body sensation index of the person calculated by the body sensation index calculating function exceeds a threshold value (S47: body sensation index determining step).

[0169] The remote control function controls the smart remote control unit 26 that operates and monitors home electrical appliances within a predetermined area (S48: remote control step). The biometric information acquisition function acquires, as biometric information data, the biometric information of a person detected by a biometric sensor (biometric / body movement sensor 3) that detects biometric information of a person while sleeping (S49: biometric information acquisition step). The biometric information determination function determines that an abnormality has occurred in the person when the biometric information data acquired by the biometric information acquisition function exceeds a threshold value of the biometric information (S50: biometric information determination step). The body movement information acquisition function acquires body movement information detected by a body movement sensor (biometric body movement sensor 3) that detects vibrations caused by body movements of a person while sleeping as body movement information data (S51: body movement information acquisition step). The body movement information determination function determines that an abnormality has occurred in the person when the body movement information data acquired by the body movement information acquisition function indicates a state in which there is no body movement for a predetermined period of time or longer (S52: body movement information determination step). When the scan data determination function determines that an abnormality has occurred in a person in the predetermined area 1, the notification function notifies a supervisor in the predetermined area 1 that an abnormality has occurred in the person (S53: notification step).

[0170] The setting information acquisition function acquires setting information input to the smart remote control unit 26 by a person present in the predetermined area 1 (S54: setting information acquisition step). The setting information storage function separates the setting information acquired by the setting information acquisition function for each model number assigned to each household electric appliance and stores the same in the storage unit 10d (S55: setting information storage step). The search function searches and identifies the setting information stored in the memory unit 10d based on the model number entered by the person in the other predetermined area 1 (S56: Search step).

[0171] According to the anomaly detection system 10 of the above-described embodiment, the presence or absence of an abnormality is determined based on the difference between the scan data obtained by scanning the inside of the specified area 1 using the three-dimensional ultrasonic sensor and the three-dimensional LiDAR sensor 21 and the scan data obtained in advance when a person in the specified area 1 is normal, so that it is possible to detect a person's fall or falling off the bed 2.

[0172] Furthermore, according to the anomaly detection system 10 of the above-described embodiment, the scan data determination unit 34 determines whether or not an anomaly has occurred using a scan data learning model that has previously learned the correspondence between the occurrence of an abnormality in a person and the scan data, thereby enabling the occurrence of an abnormality to be determined more accurately.

[0173] Furthermore, according to the anomaly detection system 10 of the above-mentioned embodiment, the presence or absence of an abnormality in a person within the specified area 1 is determined based on the image data of the imaging unit 22 with night vision function in addition to the scan data of the three-dimensional sensor 21, so that the presence or absence of an abnormality can be determined more accurately than when only one of the three-dimensional sensor 21 and the imaging unit 22 with night vision function is used.

[0174] Furthermore, according to the anomaly detection system 10 of the above-described embodiment, an access code is issued to access the imaging data acquired by the imaging data acquisition unit 32 for a specified period starting from a specified time before the determination time at which the scan data determination unit 34 and the imaging data determination unit 35 determined that an abnormality has occurred, so that a monitor can easily view the imaging data and confirm whether or not an abnormality has occurred within the specified area 1.

[0175] Furthermore, according to the anomaly detection system 10 of the above-described embodiment, the body temperature of a person is measured based on the thermal distribution image captured and acquired by the thermographic camera 23, so that the body temperature of a person can be confirmed from a remote location without contact.

[0176] Furthermore, according to the anomaly detection system 10 of the above-described embodiment, the determination of abnormalities in the temperature and humidity inside the specified area 1 is made based on the discomfort index and the sensible temperature, so that the determination can be made based on criteria close to the sensations of people present in the specified area 1.

[0177] The present invention is not limited to the anomaly detection system 10, the anomaly detection method, and the anomaly detection program according to the above-described embodiments, and can be implemented by various other modified examples or application examples without departing from the gist of the present invention as described in the claims. [Explanation of symbols]

[0178] 1 Predetermined area 2 beds 3. Biological and body movement sensors 3a cable 4 pillars 4a Installation stand 5 Air conditioning equipment 6 Lighting equipment 10 Anomaly Detection System 10a Communication Interface 10b ROM 10c RAM 10d Recording section 10e CPU 10f Input / Output Interface 10g Input Device 10h output device 11 Monitor terminal 12 Monitored Person's Terminal 12a Monitored person terminal A 12b Monitored person terminal B 12c Communication Interface 12d ROM 12e RAM 12f Recording section 12g CPU 12h Input / Output Interface 20 Network 21 Three-dimensional sensor 22 Imaging unit with night vision function 23 Thermography Camera 24 Temperature Sensor 25 Humidity Sensor 26 Smart remote control unit 31 Scan data acquisition unit 32 Imaging data acquisition unit 33 Absence determination section 34 Scan data judgment unit 35 Imaging data judgment unit 36 Judgment time acquisition unit 37 Memory control unit 38 Code Issuance Department 39 Heat distribution acquisition section 40 Body temperature acquisition section 41 Body temperature determination section 42 Temperature data acquisition section 43 Temperature judgment section 44 Humidity data acquisition section 45 Humidity determination section 46 Physical sensation index calculation section 47 Experience Index Judgment Department 48 Remote control unit 49 Biometric information acquisition unit 50 Biometric information determination unit 51 Body movement information acquisition unit 52 Body movement information determination unit 53 Notification Department 54 Setting information acquisition unit 55 Setting information storage unit 56 Search Section 58 Heart Rate 59 Breathing rate 60 Administrator terminal display screen 61 List of room conditions 62 Room 101 status display 63 Status of Room 102 64 Status of Room 103 65 Status of Room 305 67 Status display 68 Print button 69 Room Number 70 Page Selection Tab 71 Personal status window 72 Status display for the last hour 73 Current month status display 74 Last month's status display 75 Sleeping status indicator bar 76 Sleep status display graph 77 Heart Rate Graph 78 Respiration rate graph 80 Administrator terminal display screen 81 Screen switching button 82 Heart Rate 83 breathing rate 84 Graph horizontal axis scale selection button 85 Sleeping status display graph 86 Heart Rate Graph 87 Getting out of bed 88 Moving 89 Lying down 90 Display screen of image data from each room 91 Display of image data for rooms 101 92 Display of image data from Room 102 93 Display of image data for Room 103

Claims

1. An anomaly detection system that detects an occurrence of an anomaly in a person in a predetermined area by scanning the inside of the predetermined area using a three-dimensional sensor installed in the predetermined area, a scan data acquisition unit that acquires scan data of the interior of the predetermined area from the three-dimensional sensor; a scan data determination unit that determines whether or not an abnormality has occurred in the person in the specified area based on a difference between the scan data acquired by the scan data acquisition unit and scan data of the specified area acquired in advance when the person in the specified area is normal; a notification unit that notifies a monitor of the predetermined area that an abnormality has occurred in the person when the scan data determination unit determines that an abnormality has occurred in the person in the predetermined area; An anomaly detection system comprising:

2. 2. The anomaly detection system according to claim 1, wherein the anomaly is that the person falls within the predetermined area.

3. The scan data determination unit The anomaly detection system according to claim 1, characterized in that the scan data acquired by the scan data acquisition unit is input into a scan data learning model that has previously learned the correspondence between the occurrence of an abnormality in the person and the scan data, thereby determining whether or not an abnormality has occurred in the person based on the scan data acquired by the scan data acquisition unit.

4. The present invention further includes an absence determination unit that determines that a person is absent in the specified area when there is no person in the specified area based on a difference between the scan data acquired by the scan data acquisition unit and scan data of the specified area in an unmanned state that has been acquired in advance, The notification unit is 2. The anomaly detection system according to claim 1, wherein when the absence determination unit determines that the person is absent, the absence of the person in the predetermined area is notified to a person watching the predetermined area.

5. an imaging data acquisition unit that is installed in the predetermined area and acquires imaging data of an imaging unit that captures an image of an interior of the predetermined area; a determination time acquisition unit that acquires, when the scan data determination unit determines that the abnormality has occurred in the person, a time when the scan data determination unit makes the determination as a determination time; a storage control unit configured to store in a storage unit the imaging data acquired by the imaging data acquisition unit for a predetermined period from a predetermined time before the determination time acquired by the determination time acquisition unit; The anomaly detection system according to claim 1 , further comprising:

6. a code issuing unit that issues an access code for the storage control unit to access the imaging data stored in the storage unit, Further preparation, The notification unit is 6. The anomaly detection system according to claim 5, further comprising a notification to a monitor in the predetermined area that the abnormality has occurred in the person, together with the access code issued by the code issuing unit.

7. an imaging data determination unit that inputs the imaging data acquired by the imaging data acquisition unit into an imaging data learning model that has previously learned a correspondence relationship between the occurrence of an abnormality in the person and the imaging data, and determines the occurrence of an abnormality in the person based on the imaging data acquired by the imaging data acquisition unit; Further preparation, The determination time acquisition unit 6. The anomaly detection system according to claim 5, wherein, when the imaging data determination unit determines that the anomaly has occurred, the time at which the imaging data determination unit made the determination is acquired as a determination time.

8. a thermal distribution acquisition unit that acquires, as thermal distribution data, a thermal distribution inside the predetermined region by capturing an image of the inside of the predetermined region using a thermographic camera installed in the predetermined region; a body temperature acquisition unit that acquires a body temperature of a person in the predetermined area based on the heat distribution data acquired by the heat distribution acquisition unit; a body temperature determination unit that determines that an abnormality has occurred in the person when the body temperature of the person acquired by the body temperature acquisition unit exceeds a threshold value; Further comprising: The notification unit is 2. The anomaly detection system according to claim 1, wherein when the body temperature determination unit determines that an abnormality has occurred, a supervisor in the specified area is notified that an abnormality has occurred in the person.

9. a temperature data acquisition unit that acquires, as temperature data, the air temperature inside the specified area detected by a temperature sensor that detects the air temperature inside the specified area; a temperature determination unit that determines that an abnormality has occurred in the predetermined region when the temperature data acquired by the temperature data acquisition unit exceeds a threshold value; Further comprising: The notification unit is 2. The anomaly detection system according to claim 1, wherein, when the temperature determination unit determines that an abnormality has occurred, a person monitoring the specified area is notified that an abnormality has occurred in the specified area.

10. a humidity data acquisition unit that acquires, as humidity data, the humidity inside the predetermined area detected by a humidity sensor that detects the humidity inside the predetermined area; a humidity determination unit that determines that an abnormality has occurred in the predetermined region when the humidity data acquired by the humidity data acquisition unit exceeds a threshold value; Further comprising: The notification unit is The anomaly detection system according to claim 9 , wherein when the humidity determination unit determines that an abnormality has occurred, a person monitoring the specified area is notified that an abnormality has occurred in the specified area.

11. a body sensation index calculation unit that calculates a body sensation index of the person in the predetermined area based on the temperature data acquired by the temperature data acquisition unit and the humidity data acquired by the humidity data acquisition unit; a body sensation index determination unit that determines that an abnormality has occurred in the predetermined area when the body sensation index of the person calculated by the body sensation index calculation unit exceeds a threshold value; Further comprising: The notification unit is The anomaly detection system according to claim 10, characterized in that, when the physical sensation index determination unit determines that an abnormality has occurred, a monitor of the specified area is notified that an abnormality has occurred in the specified area.

12. A remote control unit that controls a smart remote control unit that operates and monitors the home appliances in the specified area, Further preparation, The household electrical appliance is an air conditioning appliance that conditions air in the specified area, The anomaly detection system according to claim 11, characterized in that when at least one of the temperature determination unit, the humidity determination unit, and the physical sensation index determination unit determines that an abnormality has occurred, the remote control unit controls the air conditioning equipment so that the temperature and humidity inside the specified area become preset temperature and humidity.

13. 12. The anomaly detection system according to claim 11, wherein the thermal sensation index is at least one of a discomfort index and a thermal sensation temperature.

14. a biometric information acquiring unit that acquires, as biometric information data, biometric information of the person detected by a biometric sensor that detects biometric information of the person while sleeping; a biometric information determination unit that determines that an abnormality has occurred in the person when the biometric information data acquired by the biometric information acquisition unit exceeds a threshold value of the biometric information; Further comprising: The notification unit is 2. The anomaly detection system according to claim 1, wherein, when the biological information determination unit determines that an abnormality has occurred, a monitor in the specified area is notified that an abnormality has occurred in the person.

15. The biological information acquisition unit is Either or both of the scan data of the three-dimensional sensor or thermal distribution data related to the thermal distribution inside the specified area obtained by imaging the inside of the specified area using a thermography camera installed in the specified area is used as real data, and the newly acquired real data is input into a virtual biometric sensor model that has previously learned a correspondence relationship between the biometric information of the person while sleeping and the real data, thereby acquiring virtual biometric information data related to the biometric information of the person while sleeping based on the newly acquired real data; The biological information determination unit The anomaly detection system according to claim 14 , wherein it is determined that an abnormality has occurred in the person when the virtual biometric information data acquired by the biometric information acquisition unit exceeds a threshold value of biometric information.

16. 15. The anomaly detection system according to claim 14, wherein the biological information is at least one of a heart rate and a respiratory rate of the person while sleeping.

17. a body movement information acquisition unit that acquires body movement information detected by a body movement sensor that detects vibrations caused by the body movement of the person while sleeping as body movement information data; a body movement information determination unit that determines that an abnormality has occurred in the person when the body movement information data acquired by the body movement information acquisition unit indicates that the person has no body movement for a predetermined period of time or longer; Further comprising: The notification unit is 2. The abnormality detection system according to claim 1, wherein when the body movement information determination unit determines that an abnormality has occurred, a monitor in the specified area is notified that an abnormality has occurred in the person.

18. The body movement information acquisition unit is Either or both of the scan data of the three-dimensional sensor or thermal distribution data relating to the thermal distribution inside the specified area obtained by imaging the inside of the specified area using a thermography camera installed in the specified area is used as actual data, and the newly acquired actual data is input into a virtual body movement sensor model that has previously learned a correspondence relationship between vibrations caused by the person's body movements while sleeping and the actual data, thereby acquiring virtual body movement information data relating to vibrations caused by the person's body movements while sleeping based on the newly acquired actual data; The body movement information determination unit The anomaly detection system according to claim 17, characterized in that the virtual body movement information data acquired by the body movement information acquisition unit determines that an abnormality has occurred in the person if the virtual body movement information data acquired by the body movement information acquisition unit indicates a state in which there is no body movement for a predetermined period of time or longer.

19. A setting information acquisition unit that acquires setting information input by a person in the predetermined area to the smart remote control unit; a setting information storage unit that separates the setting information acquired by the setting information acquisition unit for each model number assigned to each of the household electrical appliances and stores the setting information in a storage unit; a search unit that searches for and identifies the setting information stored in the storage unit based on the model number input by a person in another predetermined area; Further comprising: The anomaly detection system according to claim 12, characterized in that the setting information identified by the search unit is used for setting the smart remote control unit in another predetermined area.

20. An anomaly detection method for detecting an occurrence of an anomaly in a person in a predetermined area by scanning the inside of the predetermined area using a three-dimensional sensor installed in the predetermined area, On the computer, a scan data acquisition step of acquiring scan data of the interior of the predetermined area from the three-dimensional sensor; a scan data determination step of determining whether or not an abnormality has occurred in the person in the specified area based on a difference between the scan data acquired in the scan data acquisition step and scan data of the specified area previously acquired when the person in the specified area is normal; a notification step of notifying a supervisor of the predetermined area that an abnormality has occurred in the person when it is determined in the scan data determination step that an abnormality has occurred in the person in the predetermined area; The anomaly detection method according to the present invention comprises the steps of:

21. An anomaly detection program that detects an occurrence of an anomaly in a person in a predetermined area by scanning the inside of the predetermined area using a three-dimensional sensor installed in the predetermined area, On the computer, a scan data acquisition function for acquiring scan data of the interior of the predetermined area from the three-dimensional sensor; a scan data determination function that determines whether or not an abnormality has occurred in the person in the specified area based on a difference between the scan data acquired by the scan data acquisition function and scan data of the specified area previously acquired when the person in the specified area is normal; a notification function that notifies a person watching the predetermined area that an abnormality has occurred in the person when the scan data determination function determines that an abnormality has occurred in the person in the predetermined area; An anomaly detection program characterized by:

Images