HEAT ABNORMALITY NOTIFICATION DEVICE, HEAT ABNORMALITY NOTIFICATION SYSTEM, HEAT ABNORMALITY NOTIFICATION METHOD, AND PROGRAM

Abstract

A thermal image acquisition unit (11) acquires thermal images of the inside of a residential building, which are captured in time series. An estimation unit (14) inputs the thermal images which have been acquired by the thermal image acquisition unit (11) into a trained model stored in a storage unit (13) and estimates an urgency degree and a notification range that correspond to a thermal anomaly. According to the urgency degree and the notification range estimated by the estimation unit (14), a notification means determination unit (17) determines a user to be notified and a method for notification to the user. A communication unit (18) uses the determined method for notification to notify the thermal anomaly to a mobile terminal (30) which is possessed by the user who has been determined by the notification means determination unit (17).

Description

[Technical field]

[0001] The present disclosure relates to a thermal anomaly notification device, a thermal anomaly notification system, a thermal anomaly notification method, and a program. [Background technology]

[0002] In recent years, for example, in order to prevent the outbreak of fires in a residence, a monitoring system has been known that uses various sensors to sense the inside of the residence and notifies the user of any abnormal condition if any abnormality is detected. For example, Patent Document 1 discloses a technology for a monitoring system that can appropriately monitor a resident who uses a stove without placing an excessive burden on the resident's family, guardian, etc.

[0003] The monitoring system described in Patent Document 1 includes a heat sensor that detects heat near the stove, a human presence sensor that detects the location of the resident in the house, and a control unit that determines an abnormal state of the stove and notifies a mobile terminal. When this monitoring system detects that the stove is on using the heat sensor, it starts measuring the elapsed time that the ignition state continues, and switches the elapsed time for notification to be compared depending on the location of the resident detected by the human presence sensor, appropriately changing the timing of the notification of the abnormal state.

[0004] In addition, in the monitoring system described in Patent Document 1, as the measured elapsed time of the ignition state becomes longer, the notification level progresses from a lower level to a higher level, and the content of the notification to the mobile terminal is changed according to the notification level. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2016-103083 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology described in the above-mentioned Patent Document 1 could only respond to a predetermined abnormal state of a predetermined device, that is, whether or not the elapsed time measured in the ignition state of the stove reaches the elapsed time for notification to be compared. In addition, as for the notification level, every time the elapsed time in the ignition state reaches the elapsed time for notification specified for the current notification level, the elapsed time for notification specified for the higher notification level is used, and in the end, it can be said that the abnormal state is notified only by determining whether the measured elapsed time reaches the elapsed time for notification. Therefore, the technology described in Patent Document 1 could not realize a different notification level depending on which abnormal state is detected from among many types of abnormal states.

[0007] For this reason, there has been a demand for technology that is capable of detecting a wider variety of thermal anomalies and notifying appropriate users of detected thermal anomalies in an appropriate notification method.

[0008] The present disclosure has been made to solve the problems described above, and aims to provide a thermal anomaly notification device, a thermal anomaly notification system, a thermal anomaly notification method, and a program that are capable of detecting a wider variety of thermal anomalies and notifying appropriate users of the detected thermal anomalies in an appropriate notification method. [Means for solving the problem]

[0009] In order to achieve the above object, the heat anomaly notification device according to the present disclosure includes: A thermal image acquisition means for acquiring thermal images taken in time series inside the residence; The thermal image acquired by the thermal image acquisition means Chronological An inference means for inferring an urgency level and a notification range corresponding to the thermal anomaly by inputting the thermal image into a trained model that has been trained to infer a thermal anomaly; a determination means for determining a user to be notified and a method of notifying the user in accordance with the urgency and the notification range inferred by the inference means; a notification means for notifying the user determined by the determination means of the thermal anomaly using the notification method; Equipped with. Effect of the Invention

[0010] In the thermal anomaly notification device according to the present disclosure, the thermal image acquisition means acquires thermal images taken in chronological order within the residence. The inference means inputs the thermal images acquired by the thermal image acquisition means into a trained model that has been trained to infer thermal anomalies, and infers the urgency and notification range corresponding to the thermal anomaly. The determination means determines the user to be notified and the notification method for the user according to the urgency and notification range inferred by the inference means. The notification means then notifies the user determined by the determination means of the thermal anomaly using the notification method. As a result, it is possible to detect a wider variety of thermal anomalies, and to notify the appropriate user of the detected thermal anomaly using an appropriate notification method. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a heat anomaly notification system according to an embodiment of the present disclosure. [Diagram 2] A diagram to explain the situation when a user is outside a residence where a heat anomaly notification system is installed. [Diagram 3] A diagram to explain the situation when a user is in a residence where a heat anomaly notification system is installed. [Figure 4A] FIG. 1 is a diagram showing an example of a thermal image. [Figure 4B] FIG. 13 is a diagram showing another example of a thermal image. [Diagram 5] FIG. 1 is a diagram illustrating an example of a functional block of a heat anomaly notification device according to a first embodiment of the present disclosure. [Figure 6] FIG. 1 is a diagram for explaining a detailed configuration of a learning unit and a learning phase executed by the learning unit; [Figure 7] FIG. 1 is a diagram showing an example of risk level information according to the first embodiment of the present disclosure. [Figure 8] Diagram to explain neural networks [Figure 9]FIG. 1 is a diagram for explaining a utilization phase executed by a heat anomaly notification device according to the first embodiment of the present disclosure; [Figure 10] A flowchart showing an example of a learning process according to the first embodiment of the present disclosure. [Figure 11] A flowchart showing an example of a thermal anomaly notification process according to the first embodiment of the present disclosure. [Figure 12] FIG. 13 is a diagram showing an example of a functional block of a heat anomaly notification device according to a second embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram showing an example of risk level information according to the second embodiment of the present disclosure. [Figure 14] FIG. 13 is a diagram for explaining a utilization phase executed by a heat anomaly notification device according to a second embodiment of the present disclosure. [Figure 15] A flowchart showing an example of a thermal anomaly notification process according to the second embodiment of the present disclosure. [Figure 16] FIG. 13 is a diagram showing an example of a functional block of a heat anomaly notification device according to a third embodiment of the present disclosure. [Figure 17] FIG. 13 is a diagram for explaining a utilization phase executed by a heat anomaly notification device according to a third embodiment of the present disclosure. [Figure 18] A flowchart showing an example of a thermal anomaly notification process according to the third embodiment of the present disclosure. [Figure 19] FIG. 13 is a diagram showing an example of the overall configuration of a heat anomaly notification system according to another embodiment of the present disclosure in which a heat anomaly notification device is installed in a residence. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the drawings. In addition, the same or corresponding parts in the drawings are given the same reference numerals. Below, a case where a heat anomaly notification system is installed in a residence where a user lives will be described, but as described later, the present disclosure can also be applied in the same way when a heat anomaly notification system is installed in a building used by a user for work, such as an office building. That is, the embodiments described below are for explanation purposes and do not limit the scope of the present disclosure. Therefore, a person skilled in the art can adopt an embodiment in which each or all of these elements are replaced with an equivalent, and these embodiments are also included in the scope of the present disclosure.

[0013] (Embodiment 1) 1 is a schematic diagram showing an example of the overall configuration of a heat anomaly notification system 1 according to a first embodiment of the present disclosure. The heat anomaly notification system 1 includes a heat anomaly notification device 10, a thermal image sensor 20, a mobile terminal 30 carried by a user U, and a linked device 40. The heat anomaly notification device 10, the thermal image sensor 20, the mobile terminal 30, and the linked device 40 are connected to each other so as to be able to communicate with each other via a network 80, such as the Internet, for example.

[0014] In such a heat anomaly notification system 1, for example, as shown in Fig. 2, a thermal image sensor 20 and a linked device 40 are installed in a residence H where a user U resides. The heat anomaly notification device 10 is realized, for example, by a server device constructed on a network 80. Fig. 2 also shows a case where a user U carrying a mobile terminal 30 is outside the residence H. In this case, the mobile terminal 30 is connected to the network 80 using, for example, a mobile communication unit, and is able to communicate with the heat anomaly notification device 10.

[0015] 3 also illustrates a case where a user U who owns a mobile terminal 30 is in a residence H. In this case, the mobile terminal 30 is capable of communicating with a linked device 40 using a short-range wireless communication unit represented by, for example, Wi-Fi, Bluetooth (registered trademark) Low Energy (BLE), or near field communication (NFC).

[0016] The thermal anomaly notification device 10 is a device that detects thermal anomalies in the residence H from thermal images captured by the thermal image sensor 20 and notifies the user. Examples of thermal anomalies include forgetting to turn off the burner, forgetting to turn off the stove, forgetting to turn off the iron, abnormal heat generation from an electrical outlet, leaving the electric kettle on and heating it without water, and fever from a child or elderly person.

[0017] The thermal image sensor 20 is installed, for example, in the residence H, facing electrical appliances, cooking appliances, etc. where a thermal anomaly is expected to occur, and captures a thermal image in the residence H. For example, as shown in Figs. 2 and 3, when some thermal anomaly TA occurs in a display placed in the residence H, the thermal image sensor 20 captures a thermal image as shown in Fig. 4A. The thermal image shown in Fig. 4A shows, as an example, a state in which a part of a socket connected to the display is heated and appears white. In addition, the thermal image sensor 20 may capture a thermal image including a person. For example, when an elderly person has collapsed in the residence H, the thermal image sensor 20 captures a thermal image as shown in Fig. 4B. In Fig. 4B, as an example, a thermal image is shown of an elderly person who has collapsed with a fever in the residence H where the room temperature is high. In this case, the person's head, arms, etc. are also shown to be heated and appear white.

[0018] The thermal image sensor 20 captures these thermal images in time series and transmits the captured thermal images to the thermal anomaly notification device 10 in sequence.

[0019] Returning to Fig. 2, the mobile terminal 30 is a smartphone, tablet, or the like, owned by the user U. The mobile terminal 30 is equipped with various communication functions, and even if the mobile terminal 30 is carried outside the residence H when the user U goes out, it can connect to the network 80 from the outside and communicate with the heat anomaly notification device 10.

[0020] The linked device 40 is, for example, a network home appliance or an IoT (Internet of Things) device installed in the residence H. When a user U carrying a mobile terminal 30 is in the residence H, the linked device 40 connects to the mobile terminal 30 using, for example, a short-range wireless unit, and transmits a connection history including the connection time to the heat anomaly notification device 10. In addition, the linked device 40 can receive manual operation from the user U, and when the linked device 40 actually receives a manual operation from the user U, it transmits, for example, an operation history including the operation time to the heat anomaly notification device 10.

[0021] Note that the above-described thermal image sensor 20 may also be connectable to the mobile terminal 30 using, for example, a short-range wireless unit, like the linked device 40. In this case, when a user U carrying the mobile terminal 30 is in the residence H, the thermal image sensor 20 also connects to the mobile terminal 30 using the short-range wireless unit, and transmits, for example, a connection history including the connection time to the heat anomaly notification device 10.

[0022] Therefore, for example, assuming a case in which a heat abnormality occurring in an electrical appliance, cooking appliance, etc. within residence H is detected, if no connection history is sent from the thermal image sensor 20 or the linked device 40, and if no operation history is sent from the linked device 40, the heat abnormality notification device 10 can determine that the user U is not within residence H, and will send a heat abnormality notification to the mobile terminal 30, for example, when attempting to notify user U of a heat abnormality.

[0023] Conversely, when a connection history is sent from the thermal image sensor 20 or the linked device 40, or when an operation history is sent from the linked device 40, the heat anomaly notification device 10 can determine that the user U is within the residence H, and it is also effective to send a heat anomaly notification to the linked device 40, for example, when attempting to notify the user U of a heat anomaly.

[0024] Hereinafter, the functional blocks of the heat anomaly notification device 10 according to the first embodiment of the present disclosure will be described with reference to Fig. 5. As shown in Fig. 5, the heat anomaly notification device 10 includes a thermal image acquisition unit 11, which is an example of a thermal image acquisition means, a learning unit 12, a storage unit 13, an inference unit 14, which is an example of an inference means, an in-home information acquisition unit 15, which is an example of a history acquisition means, a position estimation unit 16, which is an example of a position estimation means, a notification means determination unit 17, which is an example of a determination means, a communication unit 18, which is an example of a notification means and a notification unit, and a control unit 19. The functional blocks shown in Fig. 5 are realized, for example, by a central processing unit (CPU) using a random access memory (RAM) as a work memory and appropriately executing a program stored in a ROM.

[0025] The thermal image acquisition unit 11 sequentially acquires thermal images captured by the thermal image sensor 20 in time series.

[0026] The learning unit 12 performs learning by adding various data to a model that has been learned in advance. For example, as shown in Fig. 6, the learning unit 12 includes a data acquisition unit 121 that is an example of a data acquisition means, and a model generation unit 122 that is an example of a model generation means. The lower part of Fig. 6 is a diagram for explaining the learning phase executed by the learning unit 12.

[0027] In the learning phase, the data acquisition unit 121 acquires learning data DT. Here, the learning data DT is, for example, data for supervised learning, and a set of input and result data is input to the neural network NN for learning. More specifically, in the present embodiment 1, the learning data DT is data in which a thermal image and risk level information are associated with each other.

[0028] As a specific example, the risk level information is information in which the situation in the house is associated with the urgency level and the notification range as shown in Fig. 7. The urgency level is the degree of urgency at the time of notification, and the notification range is the target range at the time of notification.

[0029] Returning to Fig. 6, the model generation unit 122 performs learning using the learning data DT in which such thermal images and risk level information are associated with each other. The learning algorithm used by the model generation unit 122 may be a known algorithm such as supervised learning, unsupervised learning, reinforcement learning, etc. Below, a case where a neural network NN is applied will be described as an example.

[0030] The model generation unit 122 uses a neural network NN to generate a trained model LM by so-called supervised learning. The neural network NN is formed by an input layer made up of a plurality of neurons, an intermediate layer made up of a plurality of neurons, and an output layer made up of a plurality of neurons. The intermediate layer is also called a hidden layer, and may be one layer or two or more layers.

[0031] For example, in a three-layered neural network NN as shown in Figure 8, when multiple inputs are input to the input layer (X1-X3), the values ​​are multiplied by weights W1 (w11-w16) and input to the intermediate layer (Y1-Y2), and the results are further multiplied by weights W2 (w21-w26) and output from the output layer (Z1-Z3). This output result changes depending on the respective values ​​of weights W1 and W2.

[0032] In the present disclosure, the neural network NN is trained by changing the weights of the neural network NN through so-called supervised learning according to the learning data DT which associates the thermal images acquired by the data acquisition unit 121 with the risk level information as shown in FIG. 7 described above.

[0033] 6 generates and outputs a trained model LM by processing the above-mentioned learning phase. The trained model LM thus output by the model generation unit 122 is stored in the storage unit 13 and is used by the inference unit 14 to infer thermal anomalies by inputting a thermal image.

[0034] Returning to FIG. 5, the memory unit 13 stores various data such as thermal images, trained models, and device operation data.

[0035] The inference unit 14 infers a thermal anomaly from the time series thermal images and the trained model. That is, the inference unit 14 infers the urgency and notification range corresponding to the thermal anomaly obtained by using the trained model. That is, by inputting the thermal image acquired by the thermal image acquisition unit 11 to the trained model generated by the above-mentioned learning unit 12, more specifically, the model generation unit 122, it is possible to output the urgency and notification range inferred from the thermal image.

[0036] In this disclosure, we have described the case where the urgency and notification range are output using a trained model trained by the model generation unit 122 of the learning unit 12, but it is also possible to obtain a trained model generated by an external device, such as another heat anomaly notification device 10, and output the urgency and notification range based on this trained model.

[0037] The home information acquisition unit 15 acquires from the linked device 40 at least one of a connection history of the linked device 40 connecting to the mobile terminal 30 and an operation history of a manual operation from the user U being accepted.

[0038] The position estimation unit 16 estimates the position of the user U from at least one of the connection history and the operation history. In other words, the heat anomaly notification device 10 determines whether the user U is in the residence H or not based on at least one of whether a connection history is sent from the thermal image sensor 20 or the linked device 40 and whether an operation history of accepting a manual operation is sent from the linked device 40.

[0039] The notification means determination unit 17 determines the notification target and the notification method based on the urgency and notification range corresponding to the thermal anomaly, which are inferred by the inference unit 14 from the time-series thermal images and the trained model. For example, when the urgency is inferred to be "low" and the notification range is "narrow," the notification means determination unit 17 determines the notification target to be within the same household and the notification method to be email or SNS (social networking service). When the urgency is inferred to be "high" and the notification range is "wide," the notification means determination unit 17 determines the notification target to be hospitals and private security companies and the notification method to be pop-up display, push notification, alarm, or phone call.

[0040] The communication unit 18 is, for example, various communication modules, and notifies, for example, the notification target determined by the notification means determination unit 17 of a thermal anomaly using the communication method determined by the notification means determination unit 17 described above.

[0041] The control unit 19 controls the entire heat anomaly notification device 10. For example, when the learning unit 12 processes the learning phase, the control unit 19 controls the data acquisition unit 121 and the model generation unit 122 to execute learning using the learning data DT and generate a learned model LM, as shown in FIG.

[0042] Below, with reference to FIG. 9, we will explain the utilization phase in which the thermal anomaly notification device 10 of embodiment 1 having such a block configuration utilizes time-series thermal images and a trained model to notify appropriate notification targets of thermal anomalies in an appropriate notification method.

[0043] The thermal image acquisition unit 11 sequentially acquires thermal images captured in time series by the thermal image sensor 20. Next, the inference unit 14 performs calculations on the time series thermal images acquired by the thermal image acquisition unit 11 using the trained model stored in the storage unit 13, thereby inferring the urgency level and notification range corresponding to the thermal anomaly.

[0044] Furthermore, the notification means determination unit 17 determines the notification target and the notification method based on the urgency and notification range corresponding to the heat anomaly inferred by the inference unit 14. Next, the communication unit 18 notifies the notification target determined by the notification means determination unit 17 of the heat anomaly using the communication method determined by the notification means determination unit 17. For example, the communication unit 18 transmits a heat anomaly notification to the mobile terminal 30 or the linked device 40 to notify the notification targets including the user U of the heat anomaly.

[0045] Hereinafter, the operation of the heat anomaly notification device 10 configured as above will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a flowchart showing an example of a learning process according to the first embodiment of the present disclosure. Fig. 11 is a flowchart showing an example of a heat anomaly notification process according to the first embodiment of the present disclosure.

[0046] First, a description will be given of the learning process in Fig. 10. This learning process is executed by the learning unit 12 as shown in Fig. 6 described above when generating a trained model.

[0047] First, the learning unit 12 acquires learning data DT (step S11). That is, the data acquisition unit 121 acquires learning data DT in which a thermal image and risk level information are associated with each other. This risk level information is, for example, information in which a situation in a residence is associated with an urgency level and a notification range as shown in FIG.

[0048] In step S11, the data acquisition unit 121 simultaneously acquires the thermal image and the risk level information, but as long as the thermal image and the risk level information are input in association with each other, the thermal image and the risk level information may be acquired at different times. In addition, in step S11, the location information of the user U who owns the mobile terminal 30 may be acquired.

[0049] The learning unit 12 executes a learning process (step S12). That is, the model generation unit 122 trains the neural network NN by so-called supervised learning according to the learning data DT in which the thermal image and the risk level information acquired by the data acquisition unit 121 are associated with each other, and generates a trained model LM.

[0050] The learning unit 12 stores the trained model LM (step S13). That is, the model generation unit 122 stores the trained model LM generated in the above step S12 in the storage unit 13.

[0051] Next, the heat anomaly notification process of Fig. 11 will be described. This heat anomaly notification process is executed by the heat anomaly notification device 10 using the configuration shown in Fig. 9 after the trained model is stored in the storage unit 13.

[0052] First, the thermal image acquiring unit 11 acquires a thermal image (step S21). That is, the thermal image acquiring unit 11 sequentially acquires thermal images photographed in time series by the thermal image sensor 20. Note that this step S21 is an example of a thermal image acquiring step.

[0053] The inference unit 14 inputs the thermal image into the trained model stored in the storage unit 13 to obtain the urgency and notification range (step S22). That is, the inference unit 14 infers the urgency and notification range corresponding to the thermal anomaly by calculating the time-series thermal image acquired in the above step S21 using the trained model stored in the storage unit 13. Note that this step S22 is an example of an inference step.

[0054] The inference unit 14 outputs the inference result (step S23). That is, the inference unit 14 outputs the urgency and notification range obtained by the inference to the notification means determination unit 17.

[0055] The notification means determination unit 17 determines the notification target and notification method based on the urgency and the notification range (step S24). That is, the notification means determination unit 17 determines an appropriate notification target and notification method based on the urgency and the notification range output from the inference unit 14 in the above step S23. Note that this step S24 is an example of a determination step.

[0056] The communication unit 18 notifies the thermal abnormality (step S25). That is, the communication unit 18 notifies the notification target determined by the notification means determination unit 17 in the above step S24 of the thermal abnormality by the communication method determined by the notification means determination unit 17. Note that this step S25 is an example of a notification step.

[0057] As a result, by utilizing thermal images and a trained model, it becomes possible to detect a wider variety of thermal anomalies, and to notify appropriate users of detected thermal anomalies using appropriate notification methods.

[0058] (Modification of the first embodiment) In the thermal anomaly notification device 10 of the above embodiment 1, an example has been described in which a thermal image is input into a trained model to infer the urgency and notification range, but position estimation information indicating the position of user U estimated by the position estimation unit 16 may also be further input into the trained model to infer the urgency and notification range.

[0059] Hereinafter, a heat anomaly notification device 10 according to a modified example of the first embodiment will be described. The overall configuration of the heat anomaly notification system 1 is the same as that of FIG. 1. The functional block configuration of the heat anomaly notification device 10 is the same as that of FIG. 5. The inference unit 14 infers the urgency level and notification range corresponding to the heat anomaly by calculating the time-series thermal images acquired by the thermal image acquisition unit 11 and the position estimation information indicating the position of the user U estimated by the position estimation unit 16 using the learned model stored in the storage unit 13. By further using the position estimation information of the user U, it is possible to improve the estimation accuracy of whether or not there is a person in the residence H. This makes it possible to improve the estimation accuracy of the urgency level and notification range corresponding to the heat anomaly.

[0060] (Embodiment 2) In the thermal anomaly notification device 10 of the above embodiment 1, a case has been described in which only a thermal image is input into the trained model to infer the urgency and notification range, but the thermal image and equipment status information may also be input into the trained model to infer the urgency and notification range.

[0061] Hereinafter, a heat anomaly notification device 10 according to a second embodiment of the present disclosure will be described, which is characterized in that a thermal image and equipment status information are input to a trained model to infer risk information including urgency and notification range. The overall configuration of the heat anomaly notification system 1 is the same as that shown in FIG.

[0062] FIG. 12 is a diagram showing functional blocks of a heat anomaly notification device 10 according to the second embodiment of the present disclosure. The heat anomaly notification device 10 shown in FIG. 12 includes a thermal image acquisition unit 11, which is an example of a thermal image acquisition means, a learning unit 12, a storage unit 13, an inference unit 14, which is an example of an inference means, an appliance status information acquisition unit 51, which is an example of an appliance status information acquisition means, a position estimation unit 16, which is an example of a position estimation means, a notification means determination unit 17, which is an example of a determination means, a communication unit 18, which is an example of a notification means, and a control unit 19. That is, the heat anomaly notification device 10 shown in FIG. 12 includes an appliance status information acquisition unit 51 instead of the home information acquisition unit 15 of the heat anomaly notification device 10 according to the first embodiment shown in FIG. 5 described above. In addition, in the heat anomaly notification device 10 shown in FIG. 12, the home information acquisition unit 15 may be left, and the home information acquisition unit 15 and the appliance status information acquisition unit 51 may coexist.

[0063] The device status information acquisition unit 51 acquires device status information from the linked device 40 installed in the residence H. Note that the device status information is device status information of the linked device 40, for example.

[0064] In the heat anomaly notification device 10 shown in FIG. 12, the learning unit 12 performs learning using the learning data DT in which the thermal image, the device status information, and the hazard level information are associated with each other. At that time, the hazard level information shown in FIG. 13 may be used. The hazard level information shown in FIG. 13 specifies the device in the "situation in the residence" more specifically than the hazard level information in FIG. 7 described above. In this case, as shown in FIG. 6 described above, the model generation unit 122 performs learning using the learning data DT in which the thermal image, the device status information, and the hazard level information are associated with each other, and generates and outputs the learned model LM. The learned model LM output by the model generation unit 122 is stored in the storage unit 13 and is used by the inference unit 14 to infer a heat anomaly by inputting the thermal image and the device status information.

[0065] Below, with reference to FIG. 14, we will explain the utilization phase in which the thermal anomaly notification device 10 of embodiment 2 having such a block configuration utilizes time-series thermal images, equipment status information, and a trained model to notify appropriate notification targets of thermal anomalies in an appropriate notification method.

[0066] The thermal image acquisition unit 11 sequentially acquires thermal images captured in time series by the thermal image sensor 20. Meanwhile, the device status information acquisition unit 51 acquires device status information from the linked device 40 installed in the residence H. Note that the device status information is, for example, the device status information of the linked device 40. Next, the inference unit 14 performs calculations on the time-series thermal images acquired by the thermal image acquisition unit 11 and the device status information acquired by the device status information acquisition unit 51 using the trained model stored in the storage unit 13, thereby inferring the urgency level and notification range corresponding to the thermal anomaly.

[0067] Furthermore, the notification means determination unit 17 determines the notification target and the notification method based on the urgency and notification range corresponding to the heat anomaly inferred by the inference unit 14. Next, the communication unit 18 notifies the notification target determined by the notification means determination unit 17 of the heat anomaly using the communication method determined by the notification means determination unit 17. For example, the communication unit 18 transmits a heat anomaly notification to the mobile terminal 30 or the linked device 40 to notify the notification targets including the user U of the heat anomaly.

[0068] Hereinafter, the operation of the heat anomaly notification device 10 configured as above will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of a heat anomaly notification process according to the second embodiment of the present disclosure.

[0069] First, the thermal image acquiring unit 11 acquires a thermal image (step S21). That is, the thermal image acquiring unit 11 sequentially acquires thermal images captured by the thermal image sensor 20 in time series.

[0070] The device status information acquisition unit 51 acquires device status information (step S31). That is, the device status information acquisition unit 51 acquires device status information from the linked device 40 installed in the residence H. Note that the device status information is, for example, the device status information of the linked device 40.

[0071] The inference unit 14 inputs the thermal image and the device status information into the trained model stored in the storage unit 13 to obtain the urgency and notification range (step S32). That is, the inference unit 14 infers the urgency and notification range corresponding to the thermal anomaly by calculating the time-series thermal image acquired in the above step S21 and the device status information acquired in the above step S31 using the trained model stored in the storage unit 13. For example, whether the heat source of the device in a high temperature state is a specific device such as an iron or a stove, or a heat source that does not depend on the state of the device, can be reflected in the inference process of the urgency and notification range.

[0072] The inference unit 14 outputs the inference result (step S23). That is, the inference unit 14 outputs the urgency and notification range obtained by the inference to the notification means determination unit 17.

[0073] The notification means determination unit 17 determines the notification target and notification method based on the urgency and the notification range (step S24). That is, the notification means determination unit 17 determines an appropriate notification target and notification method based on the urgency and the notification range output from the inference unit 14 in the above step S23.

[0074] The communication unit 18 notifies the thermal abnormality (step S25). That is, the communication unit 18 notifies the notification target determined by the notification means determination unit 17 in the above step S24 of the thermal abnormality by the communication method determined by the notification means determination unit 17.

[0075] As a result, by using device status information of linked devices 40 that affect thermal anomalies in addition to thermal images for inference, the accuracy of estimating the urgency and notification range corresponding to thermal anomalies can be improved, and a wider variety of thermal anomalies can be notified to appropriate users using appropriate notification methods.

[0076] (Embodiment 3) In the thermal anomaly notification device 10 according to the above-mentioned embodiments 1 and 2, the case where the thermal image acquired from the thermal image sensor 20 is directly input to the trained model has been described, but the thermal image may also be normalized and then input to the trained model.

[0077] Hereinafter, a description will be given of a heat anomaly notification device 10 according to a third embodiment of the present disclosure, which is characterized in that a thermal image acquired from a thermal image sensor 20 is normalized and then input to a trained model. The overall configuration of the heat anomaly notification system 1 is the same as that shown in FIG.

[0078] FIG. 16 is a diagram showing functional blocks of a heat anomaly notification device 10 according to the third embodiment of the present disclosure. The heat anomaly notification device 10 shown in FIG. 16 includes a thermal image acquisition unit 11, which is an example of a thermal image acquisition means, a learning unit 12, a storage unit 13, an inference unit 14, which is an example of an inference means, a normalization unit 61, which is an example of a normalization means, a position estimation unit 16, which is an example of a position estimation means, a notification means determination unit 17, which is an example of a determination means, a communication unit 18, which is an example of a notification means, and a control unit 19. That is, the heat anomaly notification device 10 shown in FIG. 16 includes a normalization unit 61 instead of the home information acquisition unit 15 of the heat anomaly notification device 10 according to the first embodiment shown in FIG. 5 described above. In addition, in the heat anomaly notification device 10 shown in FIG. 16, the home information acquisition unit 15 may be left, and the home information acquisition unit 15 and the normalization unit 61 may coexist.

[0079] The normalization unit 61 normalizes the thermal image acquired from the thermal image sensor 20 by appropriately pre-processing it. For example, the normalization unit 61 normalizes the thermal image by widening or narrowing the range of the thermal image sensor 20. Specifically, the normalization unit 61 makes the accuracy finer for a temperature range in which the human body temperature can be, such as 35°C to 40°C, and makes the accuracy coarser for other temperature ranges. Alternatively, for a temperature range of 60°C or higher, which is clearly a thermal abnormality and there is no room for determining that the urgency is not high, the accuracy within the temperature range is made coarse to ensure notification accuracy.

[0080] Hereinafter, with reference to FIG. 17, we will explain the utilization phase in which the thermal anomaly notification device 10 of embodiment 3 having such a block configuration utilizes time-series thermal images and a trained model to notify appropriate notification targets of thermal anomalies in an appropriate notification method.

[0081] The thermal image acquisition unit 11 sequentially acquires thermal images captured in time series by the thermal image sensor 20. Then, the normalization unit 61 properly pre-processes and normalizes the thermal images acquired from the thermal image sensor 20. For example, the normalization unit 61 normalizes the thermal image by widening or narrowing the range of the thermal image sensor 20. Next, the inference unit 14 calculates the thermal image normalized by the normalization unit 61 using the trained model stored in the storage unit 13, thereby inferring the urgency level and notification range corresponding to the thermal anomaly.

[0082] Furthermore, the notification means determination unit 17 determines the notification target and the notification method based on the urgency and notification range corresponding to the heat anomaly inferred by the inference unit 14. Next, the communication unit 18 notifies the notification target determined by the notification means determination unit 17 of the heat anomaly using the communication method determined by the notification means determination unit 17. For example, the communication unit 18 transmits a heat anomaly notification to the mobile terminal 30 or the linked device 40 to notify the notification targets including the user U of the heat anomaly.

[0083] Hereinafter, the operation of the heat anomaly notification device 10 configured as above will be described with reference to Fig. 18. Fig. 18 is a flowchart showing an example of a heat anomaly notification process according to the third embodiment of the present disclosure.

[0084] First, the thermal image acquiring unit 11 acquires a thermal image (step S21). That is, the thermal image acquiring unit 11 sequentially acquires thermal images captured by the thermal image sensor 20 in time series.

[0085] The normalization unit 61 normalizes the thermal image (step S41). That is, the normalization unit 61 normalizes the thermal image acquired from the thermal image sensor 20 by appropriately pre-processing it. For example, the normalization unit 61 normalizes the thermal image by widening or narrowing the range of the thermal image sensor 20.

[0086] The inference unit 14 inputs the thermal image into the trained model stored in the storage unit 13 to obtain the urgency and notification range (step S22). That is, the inference unit 14 calculates the thermal image normalized in the above step S41 using the trained model stored in the storage unit 13, thereby inferring the urgency and notification range corresponding to the thermal anomaly.

[0087] The inference unit 14 outputs the inference result (step S23). That is, the inference unit 14 outputs the urgency and notification range obtained by the inference to the notification means determination unit 17.

[0088] The notification means determination unit 17 determines the notification target and notification method based on the urgency and the notification range (step S24). That is, the notification means determination unit 17 determines an appropriate notification target and notification method based on the urgency and the notification range output from the inference unit 14 in the above step S23.

[0089] The communication unit 18 notifies the thermal abnormality (step S25). That is, the communication unit 18 notifies the notification target determined by the notification means determination unit 17 in the above step S24 of the thermal abnormality by the communication method determined by the notification means determination unit 17.

[0090] As a result, by utilizing thermal images that have been normalized to increase or decrease the accuracy in specific temperature ranges, the accuracy of estimating the urgency and notification range corresponding to thermal anomalies can be improved, and thermal anomalies can be notified to appropriate users using appropriate notification methods.

[0091] (Other embodiments) In the above embodiments 1 to 3, as shown in Figures 2 and 3, the heat anomaly notification device 10 is realized by a server device constructed on a network 80, but the heat anomaly notification device 10 may also be installed within the residence H.

[0092] Hereinafter, a heat anomaly notification system 1 according to another embodiment, characterized in that the heat anomaly notification device 10 is also installed in a residence H, will be described with reference to Fig. 19. As shown in Fig. 19, a heat anomaly notification device 10, a thermal image sensor 20, and a linked device 40 are installed in a residence H where a user U resides. Note that within the residence H, the heat anomaly notification device 10, the thermal image sensor 20, the mobile terminal 30, and the linked device 40 are connected to each other so as to be able to communicate with each other via a network 90, which is typified by a wireless LAN, as an example.

[0093] In this case as well, it becomes possible to detect a wider variety of thermal anomalies, and to notify the appropriate user of the detected thermal anomaly in an appropriate notification method.

[0094] In the above embodiment, the program executed by the heat anomaly notification device 10 can be stored and distributed in a computer-readable recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), a MO (Magneto-Optical Disk), a USB memory, a memory card, etc. By installing such a program in a specific or general-purpose computer, the computer can be made to function as the heat anomaly notification device 10 in the above embodiment.

[0095] The above-mentioned program may also be stored in a disk device of a server device on a communication network such as the Internet, and downloaded to a computer by, for example, superimposing the program on a carrier wave. The above-mentioned processing can also be achieved by starting and executing the program while transferring it via the communication network. Furthermore, the above-mentioned processing can also be achieved by executing all or a part of the program on a server device, and executing the program while the computer transmits and receives information related to the processing via the communication network.

[0096] In addition, when the above-mentioned functions are shared and realized by the OS (Operating System) or by the OS working together with an application, etc., only the parts other than the OS may be stored on the above-mentioned recording medium and distributed, or may be downloaded to a computer.

[0097] Various embodiments and modifications of the present disclosure are possible without departing from the broad spirit and scope of the present disclosure. The above-described embodiments are for explaining the present disclosure and do not limit the scope of the present disclosure. In other words, the scope of the present disclosure is indicated by the claims, not the embodiments. Various modifications made within the scope of the claims and the scope of the disclosure equivalent thereto are considered to be within the scope of the present disclosure. [Industrial Applicability]

[0098] The present disclosure can be suitably adopted in a thermal anomaly notification device, a thermal anomaly notification system, a thermal anomaly notification method, and a program that are capable of detecting a wider variety of thermal anomalies and notifying appropriate users of the detected thermal anomalies in an appropriate notification method. [Explanation of symbols]

[0099] 1 Thermal anomaly notification system, 10 Thermal anomaly notification device, 11 Thermal image acquisition unit, 12 Learning unit, 121 Data acquisition unit, 122 Model generation unit, 13 Memory unit, 14 Inference unit, 15 Home information acquisition unit, 16 Position estimation unit, 17 Notification means determination unit, 18 Communication unit, 19 Control unit, 51 Device status information acquisition unit, 61 Normalization unit, 20 Thermal image sensor, 30 Mobile terminal, 40 Linked device, 80, 90 Network

Claims

1. A thermal image acquisition means for acquiring thermal images taken in time series inside the residence; an inference means for inferring an urgency level and a notification range corresponding to the thermal anomaly by inputting the thermal images of the time series acquired by the thermal image acquisition means into a trained model that has been trained to infer a thermal anomaly; a determination means for determining a user to be notified and a method of notifying the user in accordance with the urgency and the notification range inferred by the inference means; a notification means for notifying the user determined by the determination means of the thermal anomaly using the notification method; A thermal abnormality notification device comprising:

2. a history acquisition means for acquiring a connection history between a linked device installed in the residence and a mobile terminal carried by the user, or an operation history of the linked device by the user; and a location estimation means for estimating whether the user is located inside or outside the residence based on the connection history or the operation history acquired by the history acquisition means. The thermal anomaly notification device according to claim 1 .

3. a device status information acquiring unit configured to acquire device status information of a linked device from the linked device installed in the home, The inference means inputs the equipment status information acquired by the equipment status information acquisition means into the trained model together with the time-series thermal images acquired by the thermal image acquisition means, and infers the urgency and notification range corresponding to the thermal anomaly. The thermal anomaly notification device according to claim 1 .

4. The thermal image acquisition device further includes a normalization device for normalizing the thermal images in time series acquired by the thermal image acquisition device, The inference means inputs the thermal images of the time series normalized by the normalization means into the trained model to infer an urgency level and a notification range corresponding to the thermal anomaly. The thermal abnormality notification device according to any one of claims 1 to 3.

5. A data acquisition means for acquiring learning data including thermal images taken in a time series within a residence and a level of urgency and a notification range corresponding to the thermal images; A model generation means for generating a trained model for inferring thermal anomalies using the training data, The thermal abnormality notification device according to any one of claims 1 to 3.

6. A heat anomaly notification system in which a mobile terminal carried by a user, a thermal image sensor installed in a residence, and a heat anomaly notification device are communicatively connected via a network, The thermal anomaly notification device is a thermal image acquisition means for acquiring thermal images taken in time series by the thermal image sensor inside the residence; an inference means for inferring an urgency level and a notification range corresponding to the thermal anomaly by inputting the thermal images of the time series acquired by the thermal image acquisition means into a trained model that has been trained to infer a thermal anomaly; a determination means for determining the user to be notified and a method of notifying the user according to the urgency and the notification range inferred by the inference means; a notification means for notifying the portable terminal of the user determined by the determination means of the thermal anomaly by using the notification method, Thermal anomaly notification system.

7. A thermal anomaly notification performed by a thermal anomaly notification device, A thermal image acquisition step for acquiring thermal images taken in time series inside the residence; an inference step of inputting the thermal images of the time series acquired in the thermal image acquisition step into a trained model that has been trained to infer a thermal anomaly, and inferring an urgency and a notification range corresponding to the thermal anomaly; a determination step of determining a user to be notified and a method of notifying the user according to the urgency and the notification range inferred in the inference step; a notification step of notifying the user determined in the determination step of the thermal anomaly using the notification method; A thermal anomaly notification method comprising:

8. Computer, a thermal image acquisition unit that acquires thermal images taken in time series inside the residence; an inference unit that inputs the thermal images of the time series acquired by the thermal image acquisition unit into a trained model that has been trained to infer a thermal anomaly, and infers an urgency level and a notification range corresponding to the thermal anomaly; a determination unit that determines a user to be notified and a method of notifying the user in accordance with the urgency and the notification range inferred by the inference unit; a notification unit that notifies the user determined by the determination unit of the thermal anomaly using the notification method; A program that functions as a

Images