State estimation method, state estimation device, and program

Abstract

This state estimation method comprises: an imaging step (step S1) for capturing an image of one or more photographic subjects, each being a person or animal, by using a thermal imaging camera (20); an acquisition step (step S2) for acquiring identification information for identifying the one or more photographic subjects in the thermal image captured in the imaging step; and an estimation step (step S3) for estimating the respective states of one or more estimation subjects from among the one or more photographic subjects on the basis of the thermal image captured in the imaging step and the identification information acquired in the acquisition step.

Description

[Technical Field]

[0001] The present disclosure relates to a state estimation method, a state estimation device, and a program. [Background technology]

[0002] Conventionally, there are known state estimation methods for estimating the state of at least one of a human and an animal. As an example of the state estimation method, Patent Document 1 discloses a method for distinguishing between a human and an animal using a signal obtained from a human presence sensor and estimating the state of the human. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-242687 Summary of the Invention [Problem to be solved by the invention]

[0004] Here, when one or more objects are photographed using a thermal imaging camera and the state of one or more estimated objects among the one or more objects is estimated based on the captured thermal image, there is a problem in that it is difficult to identify the one or more objects based on the temperature shown in the thermal image alone, and it is difficult to estimate the state of the one or more estimated objects.

[0005] The present disclosure has been made to solve such problems, and aims to provide a state estimation method and the like that can easily estimate the state of one or more estimation targets. [Means for solving the problem]

[0006] A state estimation method according to one aspect of the present disclosure includes a photographing step of photographing one or more subjects, each of which is a person or an animal, using a thermal imaging camera; an acquisition step of acquiring identification information for identifying the one or more subjects in the thermal image photographed in the photographing step; and an estimation step of estimating the state of one or more estimation targets among the one or more subjects based on the thermal image photographed in the photographing step and the identification information acquired in the acquisition step.

[0007] Furthermore, a state estimation device according to one embodiment of the present disclosure includes a photographing unit that photographs one or more photographic subjects, each of which is a person or an animal, using a thermal imaging camera; an acquisition unit that acquires identification information for identifying the one or more photographic subjects in the thermal image photographed by the photographing unit; and an estimation unit that estimates the state of one or more estimation targets among the one or more photographic subjects based on the thermal image photographed by the photographing unit and the identification information acquired by the acquisition unit.

[0008] A program according to one aspect of the present disclosure is a program for causing a computer to execute the above-described state estimation method. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a state estimation method and the like that can easily estimate the state of one or more estimation targets. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating a state estimating device and the like according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the state estimating device of FIG. [Figure 3] FIG. 3 is a flowchart showing an example of the operation of the state estimating device of FIG. [Figure 4] FIG. 4 is a table showing a first example of the identification information. [Figure 5] FIG. 5 is a diagram showing a first example of a thermal image. [Figure 6] FIG. 6 is a table showing a second example of the identification information. [Figure 7] FIG. 7 is a diagram showing a second example of the thermal image. [Figure 8] FIG. 8 is a table showing a third example of the identification information. [Figure 9] FIG. 9 is a diagram showing a third example of a thermal image. [Figure 10] FIG. 10 is a table showing a fourth example of the identification information. [Figure 11] FIG. 11 is a graph for explaining another example of the operation of the state estimating device of FIG. [Figure 12] FIG. 12 is a diagram illustrating a state estimating device and the like according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating a state estimating device and the like according to the third embodiment. [Figure 14] FIG. 14 is a diagram illustrating a state estimating device and the like according to the fourth embodiment. [Figure 15] FIG. 15 is a diagram illustrating a state management system according to the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described. Note that each of the embodiments described below represents a specific example of the present disclosure. Therefore, the numerical values, components, the arrangement and connection of the components, the steps and the order of the steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Therefore, among the components in the following embodiments, components that are not recited in the independent claims will be described as optional components.

[0012] Furthermore, each drawing is a schematic diagram and is not necessarily an exact illustration. In each drawing, the same reference numerals are used to denote substantially the same components, and redundant explanations will be omitted or simplified.

[0013] (First embodiment) 1 is a diagram showing a state estimating device 10 according to the first embodiment, etc. The state estimating device 10 according to the first embodiment will be described with reference to FIG.

[0014] As shown in FIG. 1, the state estimation device 10 is a device that estimates the state of one or more estimation targets. As will be described in detail later, each of the one or more estimation targets is a photographed target of which state is to be estimated among one or more photographed targets photographed using a thermal imaging camera 20. For example, each of the one or more estimation targets is a person or an animal. Here, the one or more estimation targets are person 1, and the state estimation device 10 estimates the state of person 1. For example, the state of each of the one or more estimation targets is the sleep state of each of the one or more estimation targets. The sleep state specifically refers to the depth of sleep, such as REM sleep, non-REM sleep, mid-sleep awakening, and wakefulness, and may also be, but is not limited to, cessation of breathing during sleep, as seen in sleep apnea syndrome. Furthermore, the state of each of the one or more estimation targets may also be, but is not limited to, the activity state (amount of body movement) of a person in a room, a health-related state such as body temperature, etc., other than the sleep state.

[0015] The state estimation device 10 estimates the state of each of one or more estimation targets using a thermal imaging camera 20. The thermal imaging camera 20 is a camera for capturing thermal images. For example, a thermal image is an image that represents the distribution of heat. The thermal imaging camera 20 is placed in a position where it can capture an image of one or more capture targets. Here, the thermal imaging camera 20 is provided above a bed 2 on which a person 1 sleeps, and captures the person 1 lying on the bed 2. For example, the bed 2 is a bed. The state estimation device 10 is communicatively connected to the thermal imaging camera 20, and estimates the state of each of one or more estimation targets among the one or more capture targets based on the thermal image captured using the thermal imaging camera 20.

[0016] For example, the state estimation device 10 is realized by a processor, a memory, and the like.

[0017] The state estimating device 10 and the like have been described above.

[0018] Fig. 2 is a block diagram showing the functional configuration of the state estimating device 10 in Fig. 1. The functional configuration of the state estimating device 10 will be described with reference to Fig. 2.

[0019] As shown in FIG. 2, the state estimation device 10 includes an imaging unit 11, an acquisition unit 12, an estimation unit 13, and an output unit 14.

[0020] The photographing unit 11 photographs one or more subjects, each of which is a person or an animal, using the thermal imaging camera 20. That is, each of the one or more subjects is a person or an animal and is photographed using the thermal imaging camera 20. For example, when each of the one or more subjects is sleeping, the photographing unit 11 photographs the one or more subjects while they are sleeping using the thermal imaging camera 20. For example, the photographing unit 11 photographs the one or more subjects using the thermal imaging camera 20 at a predetermined time interval.

[0021] The acquisition unit 12 acquires identification information for identifying one or more imaging targets in the thermal image captured by the imaging unit 11. For example, the identification information includes information for estimating which of the one or more imaging targets are in the thermal image captured by the imaging unit 11. Furthermore, for example, the identification information includes information for estimating which of the one or more imaging targets are in the thermal image captured by the imaging unit 11 are one or more estimation targets whose state is to be estimated.

[0022] For example, the identification information includes information indicating the number of one or more subjects to be photographed. For example, when one subject is photographed using the thermal imaging camera 20, the identification information includes information indicating that the number of one or more subjects to be photographed is 1. Furthermore, for example, when two subjects are photographed using the thermal imaging camera 20, the identification information includes information indicating that the number of one or more subjects to be photographed is 2.

[0023] Furthermore, for example, the identification information includes information indicating the number of one or more people included in one or more shooting objects, and information indicating the number of one or more animals included in one or more shooting objects. For example, when one person and one dog are photographed using the thermal imaging camera 20, the identification information includes information indicating that the number of one or more people included in one or more shooting objects is one, and information indicating that the number of one or more animals included in one or more shooting objects is one. For example, when one person, one dog, and one cat are photographed using the thermal imaging camera 20, the identification information includes information indicating that the number of one or more people included in one or more shooting objects is one, and information indicating that the number of one or more animals included in one or more shooting objects is two.

[0024] Furthermore, for example, the identification information includes information indicating the position where each of the one or more estimated targets sleeps. For example, the identification information includes information indicating the absolute position where each of the one or more estimated targets sleeps. For example, the identification information includes information indicating the relative position where each of the one or more estimated targets sleeps.

[0025] Furthermore, for example, the identification information includes information indicating the number of one or more estimation targets. For example, if the one or more shooting targets are one person and the state of the person is to be estimated, the identification information includes information indicating that the number of one or more estimation targets is 1. For example, if the one or more shooting targets are one person and one dog and the state of only the person of the person and the dog is to be estimated, the identification information includes information indicating that the number of one or more estimation targets is 1.

[0026] Furthermore, for example, the identification information includes information indicating the number of one or more people included in one or more estimation targets, and information indicating the number of one or more animals included in one or more estimation targets. For example, when the state of one person and one dog is estimated, the identification information includes information indicating that the number of one or more people included in one or more estimation targets is 1, and information indicating that the number of one or more animals included in one or more estimation targets is 1. For example, when the state of one person, one dog, and one cat is estimated, the identification information includes information indicating that the number of one or more people included in one or more estimation targets is 1, and information indicating that the number of one or more animals included in one or more estimation targets is 2.

[0027] For example, the identification information is input in advance by a user or the like and stored in the state estimation device 10, and the acquisition unit 12 acquires the identification information stored in the state estimation device 10. Note that, for example, the acquisition unit 12 may acquire the identification information stored in a device or the like external to the state estimation device 10.

[0028] The estimation unit 13 estimates the state of each of one or more estimation targets among the one or more captured targets based on the thermal image captured by the imaging unit 11 and the identification information acquired by the acquisition unit 12. For example, the estimation unit 13 identifies which targets in the thermal image are the one or more estimation targets using the identification information, and estimates the state of each of the one or more estimation targets based on the temperature of each of the one or more estimation targets in the thermal image.

[0029] For example, when the number of one or more locations with a temperature above a predetermined temperature in the thermal image captured by the photographing unit 11 is greater than the number of one or more subjects to be photographed, the estimation unit 13 preferentially identifies the locations with higher temperatures among the one or more locations as the one or more subjects to be photographed. For example, when the number of one or more locations with a temperature above a predetermined temperature in the thermal image captured by the photographing unit 11 is three and the number of one or more subjects to be photographed is two, the estimation unit 13 identifies the location with the highest temperature and the location with the second highest temperature among the one or more locations with a temperature above the predetermined temperature in the thermal image as the one or more subjects to be photographed. For example, each of the one or more locations with a temperature above a predetermined temperature in the thermal image is a cluster of temperature regions above the predetermined temperature in the thermal image.

[0030] Furthermore, for example, the estimation unit 13 preferentially identifies as a person an object that was located at a predetermined position earliest among one or more objects in a thermal image captured by the imaging unit 11. For example, if the one or more objects include two people and one animal, the estimation unit 13 will identify as a person an object that was located at a predetermined position earliest and an object that was located next earliest among one or more objects in a thermal image captured by the imaging unit 11.

[0031] Furthermore, for example, the estimation unit 13 identifies one or more estimation targets based on the positions of one or more imaging targets in the thermal image captured by the imaging unit 11. For example, if the identification information includes information indicating absolute positions of one or more estimation targets when they lie down, the estimation unit 13 identifies which one or more estimation targets are in the thermal image based on the absolute positions of the one or more imaging targets in the thermal image captured by the imaging unit 11. For example, if the identification information includes information indicating relative positions of one or more estimation targets when they lie down, the estimation unit 13 identifies which one or more estimation targets are in the thermal image based on the positional relationship of the one or more imaging targets in the thermal image captured by the imaging unit 11.

[0032] The output unit 14 outputs the estimation result estimated by the estimation unit 13. For example, the output unit 14 outputs the estimation result to an analysis device that performs analysis using the estimation result estimated by the estimation unit 13.

[0033] The functional configuration of the state estimating device 10 has been described above.

[0034] Fig. 3 is a flowchart showing an example of the operation of the state estimating device 10 in Fig. 1. With reference to Fig. 3, an example of the operation of the state estimating device 10 will be described.

[0035] As shown in FIG. 3, first, the photographing unit 11 photographs one or more subjects, each of which is a person or an animal, using the thermal imaging camera 20 (photographing step) (step S1).

[0036] When the photographing unit 11 photographs one or more photographing targets, the acquiring unit 12 acquires identification information for identifying the one or more photographing targets in the thermal image photographed in the photographing step (acquisition step) (step S2).

[0037] When the acquisition unit 12 acquires the identification information, the estimation unit 13 estimates the state of each of one or more estimation targets among the one or more imaging targets based on the thermal image captured in the imaging step and the identification information acquired in the acquisition step (estimation step) (step S3).

[0038] Here, Fig. 4 is a table showing a first example of the identification information. Fig. 5 is a diagram showing a first example of a thermal image. A first example of the state estimation by the estimation unit 13 will be described with reference to Figs. 4 and 5.

[0039] As shown in Figure 4, here, the identification information includes information indicating that the subject of photography is a person, the number of people being photographed is 1, and whether or not state estimation is required for the person being photographed, i.e., whether or not the person being photographed is an estimation subject.

[0040] In this way, the identification information includes information indicating the number of one or more subjects to be photographed, information indicating the number of one or more people included in one or more subjects to be photographed, information indicating the number of one or more estimated subjects, and information indicating the number of one or more people included in one or more estimated subjects. Here, the identification information indicates that the number of one or more subjects to be photographed is 1, the number of one or more people included in one or more subjects to be photographed is 1, the number of one or more estimated subjects is 1, and the number of one or more people included in one or more estimated subjects is 1.

[0041] As shown in Fig. 5, here, there are two locations (see locations A and B surrounded by dashed lines in Fig. 5) where the temperature is above the predetermined temperature in the thermal image captured in the capturing step. In other words, here, the number of locations where the temperature is above the predetermined temperature in the thermal image captured in the capturing step is two.

[0042] For example, if the subject wakes up in the middle of the night and goes to sleep in a different place than where they were originally sleeping, the place where the subject was originally sleeping and the place where the subject is currently sleeping in the thermal image taken in the shooting step may be at or above a predetermined temperature, and the number of one or more places at or above the predetermined temperature in the thermal image taken in the shooting step may be greater than the number of one or more subjects being photographed.

[0043] For example, in the estimation step, if the number of one or more locations in the thermal image captured in the photographing step that are at or above a predetermined temperature is greater than the number of one or more subjects to be photographed, the estimation unit 13 preferentially identifies the location with the higher temperature among the one or more locations as the one or more subjects to be photographed.

[0044] In this case, the number of one or more locations in the thermal image captured in the capturing step that are at or above a predetermined temperature is two, the number of one or more subjects to be captured is one, and the number of one or more locations in the thermal image captured in the capturing step that are at or above a predetermined temperature is greater than the number of one or more subjects to be captured, so the estimation unit 13 identifies the location with the higher temperature among the one or more locations (see location A surrounded by dashed lines in Figure 5) as the one or more subjects to be captured.

[0045] Here, since the number of one or more shooting targets and the number of one or more estimation targets are the same, the estimation unit 13 identifies the one or more shooting targets as one or more estimation targets.

[0046] The estimation unit 13 estimates the state of one or more estimation targets based on the temperature of the one or more estimation targets in the thermal image. For example, the estimation unit 13 estimates the sleep depth of the one or more estimation targets.

[0047] The first example of state estimation by the estimation unit 13 has been described above.

[0048] Fig. 6 is a table showing a second example of the identification information. Fig. 7 is a diagram showing a second example of the thermal image. A second example of the state estimation by the estimation unit 13 will be described with reference to Figs. 6 and 7.

[0049] As shown in Figure 6, here, the identification information includes information indicating that the subjects to be photographed are people and animals, the number of people to be photographed is 1, the number of animals to be photographed is 1, whether state estimation is required for the people to be photographed, i.e., whether the people to be photographed are the subject of estimation, and whether state estimation is required for the animals to be photographed, i.e., whether the animals to be photographed are the subject of estimation.

[0050] In this way, the identification information includes information indicating the number of one or more subjects to be photographed, information indicating the number of one or more people included in one or more subjects to be photographed, information indicating the number of one or more animals included in one or more subjects to be photographed, information indicating the number of one or more estimated subjects, and information indicating the number of one or more people included in one or more estimated subjects. Here, the identification information indicates that the number of one or more subjects to be photographed is two, the number of one or more people included in one or more subjects to be photographed is one, the number of one or more animals included in one or more subjects to be photographed is one, and the number of one or more people included in one or more estimated subjects is one.

[0051] As shown in Figure 7, here, there are two locations in the thermal image captured in the imaging step where the temperature is above the predetermined temperature (see locations C and D surrounded by dashed lines in Figure 7). In other words, here, the number of locations in the thermal image captured in the imaging step where the temperature is above the predetermined temperature is two.

[0052] For example, one of the one or more locations represents a person, and the other of the one or more locations represents an animal.

[0053] For example, in the estimation step, the estimation unit 13 preferentially identifies, as a person, one or more objects in the thermal image captured in the capturing step that is located at a predetermined position earlier than the other objects. For example, the predetermined position is a position that can be captured by the thermal imaging camera 20.

[0054] Here, in the thermal image captured in the photographing step, the subject indicated at location D among the one or more subjects was located at a position where it could be photographed by the thermal imaging camera 20 earlier than the subject indicated at location C among the one or more subjects, and therefore the estimation unit 13 identifies the subject indicated at location D among the one or more subjects as a person, and identifies the subject indicated at location C as an animal.

[0055] Here, the number of one or more estimated objects is one, and the number of one or more people included in the one or more estimated objects is one, so the estimation unit 13 identifies the object indicated at location D among the one or more photographed objects as one or more estimated objects.

[0056] The estimation unit 13 estimates the state of one or more estimation targets based on the temperature of the one or more estimation targets in the thermal image. For example, the estimation unit 13 estimates the sleep depth of the one or more estimation targets.

[0057] For example, in the estimation step, the estimation unit 13 may prioritize identifying as an animal an object that was located at a predetermined position earlier among one or more objects in the thermal image captured in the capturing step. Furthermore, the estimation unit 13 may prioritize recognizing as an animal an object whose maximum or average temperature is higher. This is because the body temperature of dogs and cats is higher than that of humans, and because humans wear clothing, the temperature of the surface of the clothing captured in the image is lower than the temperature of the skin surface. Furthermore, for example, the state of an animal may be estimated, and the identification information may include information indicating the number of one or more animals included in one or more estimation objects.

[0058] The second example of state estimation by the estimation unit 13 has been described above.

[0059] Fig. 8 is a table showing a third example of identification information. Fig. 9 is a diagram showing a third example of a thermal image. A third example of state estimation by the estimation unit 13 will be described with reference to Figs. 8 and 9.

[0060] As shown in Figure 8, the identification information here includes information indicating that the subjects to be photographed are people and animals, the number of people to be photographed is 1, the number of animals to be photographed is 1, whether or not state estimation is required for the people to be photographed, i.e., whether or not the people to be photographed are the subjects to be estimated, whether or not state estimation is required for the animals to be photographed, i.e., whether or not the animals to be photographed are the subjects to be estimated, whether or not fever detection is required for the people to be photographed, whether or not fever detection is required for the animals to be photographed, and the sleeping position of the subjects to be photographed.

[0061] In this way, the identification information includes information indicating the number of one or more subjects to be photographed, information indicating the number of one or more people included in one or more subjects to be photographed, information indicating the number of one or more animals included in one or more subjects to be photographed, information indicating the number of one or more estimated subjects, information indicating the number of one or more people included in one or more estimated subjects, and information indicating the sleeping position of each of the one or more estimated subjects. Here, the identification information indicates that the number of one or more subjects to be photographed is 2, the number of one or more people included in one or more subjects to be photographed is 1, the number of one or more animals included in one or more subjects to be photographed is 1, the number of one or more estimated subjects is 1, the number of one or more people included in one or more estimated subjects is 1, and that the person included in one or more estimated subjects sleeps on the right side.

[0062] As shown in Fig. 9, here, there are two locations (see locations E and F surrounded by dashed lines in Fig. 9) where the temperature is above the predetermined temperature in the thermal image captured in the capturing step. In other words, here, the number of locations where the temperature is above the predetermined temperature in the thermal image captured in the capturing step is two.

[0063] For example, one of the one or more locations represents a person, and the other of the one or more locations represents an animal.

[0064] For example, in the estimation step, the estimation unit 13 identifies one or more estimation targets based on the positions of one or more imaging targets in the thermal image captured in the imaging step.

[0065] Here, in the thermal image captured in the capturing step, the object indicated at location F among the one or more objects to be captured is located to the right of the object indicated at location E among the one or more objects to be captured, and the estimation unit 13 identifies the object indicated at location F among the one or more objects to be captured as one or more estimated objects.

[0066] The estimation unit 13 estimates the state of one or more estimation targets based on the temperature of the one or more estimation targets in the thermal image. For example, the estimation unit 13 estimates the sleep depth of the one or more estimation targets.

[0067] For example, the output unit 14 outputs an alert based on the temperature of one or more objects in a thermal image. For example, the normal temperature of one or more objects is measured, and if the temperature of one or more objects in a thermal image is higher than the normal temperature of the one or more objects, the output unit 14 outputs an alert. Specifically, for example, if the average surface temperature of the objects over the last 10 days is 33°C and the root mean square error (RSME) is 0.5°C, and the average surface temperature of the objects becomes 35°C, the output unit 14 outputs an alert.

[0068] The third example of the state estimation by the estimation unit 13 has been described above.

[0069] Returning to FIG. 3, once the estimation unit 13 estimates the state of each of the one or more estimation targets, the output unit 14 outputs the estimation result by the estimation unit 13 (output step) (step S4).

[0070] An example of the operation of the state estimating device 10 has been described above.

[0071] Fig. 10 is a table showing a fourth example of the identification information. Fig. 11 is a graph for explaining another example of the operation of the state estimation device 10 of Fig. 1. Another example of the operation of the state estimation device 10 will be described with reference to Figs. 10 and 11.

[0072] 10, here, the one or more subjects to be photographed are person A and person B. Person B is ill, and an illness mode is set for person B.

[0073] For example, the estimation unit 13 estimates the depth of sleep of person B based on the thermal image captured in the capturing step and the identification information acquired in the acquiring step, and the output unit 14 notifies person A in real time of the estimation result by the estimation unit 13. This allows person A to know that person B is sleeping, and can take care not to wake person B, for example.

[0074] Furthermore, for example, the output unit 14 notifies person A of the temperature of person B. This allows person A to recognize whether person B's temperature is rising or falling.

[0075] Furthermore, for example, the estimation unit 13 determines whether or not person B has woken up based on the sleep depth of person B, and when person B has woken up, the output unit 14 notifies person A that person B has woken up. For example, as shown in FIG. 11 , the estimation unit 13 can determine that person A has woken up if the sleep depth becomes shallow when body temperature is low. By notifying person A that person B has woken up, person A can be prevented from disturbing person B's sleep, and person A can inquire about person B's physical condition and take care of person B's diet when person B wakes up.

[0076] The state estimating device 10 and the like according to the first embodiment have been described above.

[0077] The state estimation method according to the first embodiment includes a photographing step (step S1) of photographing one or more subjects, each of which is a person or an animal, using a thermal imaging camera 20, an acquisition step (step S2) of acquiring identification information for identifying the one or more subjects in the thermal image photographed in the photographing step, and an estimation step (step S3) of estimating the state of one or more estimation targets among the one or more subjects based on the thermal image photographed in the photographing step and the identification information acquired in the acquisition step.

[0078] This makes it easier to identify one or more shooting targets in the thermal image using the identification information, and therefore makes it easier to identify one or more estimation targets included in one or more shooting targets in the thermal image, and makes it possible to estimate the state of the one or more estimation targets easily and with high accuracy.

[0079] In the state estimation method according to the first embodiment, the estimation step estimates the sleep state of each of one or more estimation targets.

[0080] This allows one or more sleep states to be estimated easily and with high accuracy.

[0081] In the state estimating method according to the first embodiment, the identification information includes information indicating the number of subjects to be photographed, which is one or more.

[0082] This allows the number of one or more objects to be photographed to be used, making it easier to identify one or more objects to be photographed in the thermal image, and therefore making it easier to identify one or more estimated objects contained in one or more objects to be photographed, and making it easier and more accurate to estimate the state of one or more estimated objects.

[0083] In addition, in the state estimation method according to the first embodiment, in the estimation step, if the number of one or more locations in the thermal image captured in the photographing step that are at or above a predetermined temperature is greater than the number of one or more subjects to be photographed, the location with the highest temperature among the one or more locations is preferentially identified as the one or more subjects to be photographed.

[0084] This prevents one or more locations in the thermal image that are not the subject of one or more imaging from being mistakenly assumed to be the subject of one or more imaging, making it easier to identify the subject of one or more imaging in the thermal image, and therefore making it easier to identify one or more estimated subjects contained in the subject of one or more imaging, and making it easier and more accurate to estimate the state of the one or more estimated subjects.

[0085] In addition, in the state estimation method according to the first embodiment, the identification information includes information indicating the number of one or more people included in one or more subjects, and information indicating the number of one or more animals included in one or more subjects.

[0086] This allows the number of one or more people and the number of one or more animals included in one or more subjects to be photographed to be used, making it easier to identify one or more subjects to be photographed in the thermal image, and therefore making it easier to identify one or more estimated subjects included in one or more subjects to be photographed, and making it easier and more accurate to estimate the state of one or more estimated subjects.

[0087] In addition, in the state estimation method according to the first embodiment, in the estimation step, the object that was positioned at a predetermined position earlier than the other object in the thermal image captured in the photographing step is preferentially identified as a person.

[0088] This makes it easy to estimate which objects in the thermal image are people, and makes it easier to identify one or more captured objects in the thermal image, making it easier to identify one or more estimated objects contained in one or more captured objects, and making it easier and more accurate to estimate the state of one or more estimated objects.

[0089] In the state estimation method according to the first embodiment, the identification information includes information indicating the lying position of each of the one or more estimation targets.

[0090] This allows the position in which each of the one or more estimation targets lies to be used, making it easier to identify the one or more imaging targets in the thermal image, making it easier to identify the one or more estimation targets contained in the one or more imaging targets, and making it possible to estimate the state of the one or more estimation targets more easily and with higher accuracy.

[0091] In the state estimating method according to the first embodiment, the estimating step identifies one or more estimation targets based on the positions of the one or more imaging targets in the thermal image captured in the imaging step.

[0092] This makes it easier to identify one or more estimation targets included in one or more imaging targets based on the positions of the one or more imaging targets in the thermal image, and allows the state of the one or more estimation targets to be estimated more easily and with higher accuracy.

[0093] In the state estimation method according to the first embodiment, the identification information includes information indicating the number of estimation targets, which is one or more.

[0094] This allows the number of one or more estimation targets to be used, making it easier to identify one or more photographed targets in a thermal image, making it easier to identify one or more estimation targets contained in one or more photographed targets, and making it possible to estimate the state of one or more estimation targets more easily and with higher accuracy.

[0095] In addition, in the state estimation method according to the first embodiment, the identification information includes information indicating the number of one or more people included in one or more estimation targets, and information indicating the number of one or more animals included in one or more estimation targets.

[0096] This allows the number of one or more people and the number of one or more animals included in one or more estimation targets to be used, making it easier to identify one or more estimation targets included in one or more shooting targets, and making it possible to estimate the state of one or more estimation targets more easily and with higher accuracy.

[0097] In addition, the state estimation device according to the first embodiment includes an imaging unit 11 that uses a thermal imaging camera 20 to capture images of one or more subjects, each of which is a person or an animal; an acquisition unit 12 that acquires identification information for identifying the one or more subjects in the thermal image captured by the imaging unit 11; and an estimation unit 13 that estimates the state of one or more estimation targets among the one or more subjects based on the thermal image captured by the imaging unit 11 and the identification information acquired by the acquisition unit 12.

[0098] This provides the same effects as the above-described state estimation method.

[0099] (Second embodiment) 12 is a diagram showing a state estimating device 10 etc. according to the second embodiment. The state estimating device 10 etc. according to the second embodiment will be described with reference to FIG.

[0100] As shown in FIG. 12, the state estimating device 10 according to the second embodiment is mainly different from the state estimating device 10 according to the first embodiment in that the state estimating device 10 according to the second embodiment further acquires a detection result from a fire detection sensor 30.

[0101] The estimation unit 13 estimates the state of one or more estimation targets based on the detection result by the fire detection sensor 30. For example, when the fire detection sensor 30 detects smoke, the estimation unit 13 estimates that one or more estimation targets are awake. This prevents the estimation unit 13 from estimating that one or more estimation targets are sleeping, for example, when one or more estimation targets are not sleeping but are smoking.

[0102] The state estimating device 10 and the like according to the second embodiment have been described above.

[0103] (Third embodiment) 13 is a diagram illustrating a state estimating device 10 etc. according to the third embodiment. The state estimating device 10 etc. according to the third embodiment will be described with reference to FIG.

[0104] As shown in FIG. 13, the state estimating device 10 according to the third embodiment is mainly different from the state estimating device 10 according to the first embodiment in that the state estimating device 10 according to the third embodiment further acquires a detection result from an illuminance sensor 40.

[0105] The estimation unit 13 estimates the state of one or more estimation targets based on the detection result by the illuminance sensor 40.

[0106] For example, when the illuminance detected by the illuminance sensor 40 is equal to or lower than a predetermined illuminance, the estimation unit 13 estimates that one or more estimation targets are sleeping. This allows the estimation unit 13 to easily estimate that one or more estimation targets are sleeping, and therefore to easily estimate the sleeping states of the one or more estimation targets.

[0107] Furthermore, for example, when the illuminance detected by the illuminance sensor 40 is higher than a predetermined illuminance, the estimation unit 13 estimates that one or more estimation targets are awake. This allows the estimation unit 13 to easily estimate that one or more estimation targets are awake, and therefore to easily estimate the state when one or more estimation targets are awake.

[0108] For example, the estimation unit 13 may acquire a signal indicating the operating state of a lighting device, or a signal indicating the on / off state of a switch of a lighting device, and estimate whether one or more estimation targets are asleep or awake based on these signals.

[0109] The state estimating device 10 and the like according to the third embodiment have been described above.

[0110] (Fourth embodiment) 14 is a diagram showing a state estimating device 10 etc. according to the fourth embodiment. With reference to FIG. 14, the state estimating device 10 etc. according to the fourth embodiment will be described.

[0111] As shown in FIG. 14, the state estimating device 10 according to the fourth embodiment mainly differs from the state estimating device 10 according to the first embodiment in that the state estimating device 10 controls a lighting device 50.

[0112] The state estimation device 10 estimates the body movements of one or more estimation targets from a thermal image, and controls the lighting device 50 based on the body movements of the one or more estimation targets.

[0113] For example, when the state estimation device 10 estimates that one or more estimation subjects are about to fall asleep, it controls the lighting device 50 to dim the lights, thereby encouraging the one or more estimation subjects to fall asleep.

[0114] Furthermore, for example, the state estimation device 10 controls the lighting device 50 based on the time of day.

[0115] For example, when it is time for one or more estimation subjects to wake up, the state estimation device 10 controls the lighting device 50 to brighten the lights, thereby encouraging the one or more estimation subjects to wake up.

[0116] Furthermore, for example, if one or more estimation subjects are about to fall asleep during the daytime, the state estimation device 10 dims the lights and then brightens them 10 to 30 minutes later, thereby suppressing disruption of the circadian rhythm and enabling more comfortable sleep.

[0117] Furthermore, for example, when the state estimation device 10 estimates that one or more estimation targets have fallen off the bed, it controls the lighting device 50 to brighten the lights. For example, when one or more estimation targets remain at the bedside for a predetermined time without turning on the lights, the state estimation device 10 determines that one or more estimation targets have fallen off the bed.

[0118] The state estimating device 10 and the like according to the fourth embodiment have been described above.

[0119] (Fifth embodiment) 15 is a diagram showing a state management system 100 etc. according to the fifth embodiment. The state management system 100 etc. will be described with reference to FIG.

[0120] As shown in FIG. 15, a state management system 100 according to the fifth embodiment includes a plurality of state estimation devices 10 (see FIG. 1, etc.) each provided in a respective one of a plurality of residences 3, and a server 101.

[0121] Each of the plurality of state estimation devices 10 transmits the estimation result by the estimation unit 13 to the server 101.

[0122] The server 101 generates and manages statistical information from estimation results transmitted from each of the multiple state estimation devices 10. For example, the server 101 generates statistical information on sleep for each region. Also, for example, the server 101 generates statistical information on sleep for each age. Also, for example, the server 101 generates statistical information on sleep for each event.

[0123] The server 101 transmits the generated statistical information to each of the plurality of state estimation devices 10.

[0124] This allows, for example, people living in each of a plurality of residences 3 to recognize the sleep-onset state of people living near their own residence 3, and to fall asleep in accordance with the sleep-onset state of those people so as not to cause any inconvenience to those people with noise, etc., thereby improving sleep efficiency in each area.

[0125] This also makes it possible to close the gates to an area when the sleep onset rate in that area exceeds a predetermined threshold, thereby increasing crime prevention and improving sleep efficiency in that area.

[0126] This also allows, for example, for the convenience store lights and street lights in a certain area to be turned off when the sleep rate in that area exceeds a predetermined threshold, thereby promoting energy conservation.

[0127] This also makes it possible to close roads leading to a certain area when the sleep onset rate in that area exceeds a predetermined threshold, thereby increasing crime prevention and improving sleep efficiency in that area.

[0128] This also makes it possible to carry out maintenance of public systems in a certain area if the sleep onset rate in that area exceeds a predetermined threshold, thereby enabling maintenance to be carried out while preventing it from interfering with the lives of people living in that area.

[0129] The state management system 100 and the like according to the fifth embodiment have been described above.

[0130] (Other embodiments, etc.) While the state estimation device and the like according to one or more aspects have been described above based on the embodiments, the present disclosure is not limited to these embodiments. As long as the modifications do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art may also be included within the scope of the present disclosure.

[0131] For example, the estimated bedtime and wake-up time for each set subject may be notified to the user (e.g., the user himself / herself) the next morning or later. By using the bedtime and wake-up time, it becomes easier to distinguish between a person and a pet (animal), so if the identification of a person or a pet is incorrect, it can be corrected or the identification can be improved the next day or later.

[0132] Furthermore, for example, the thermal sensations of a person and a pet may be estimated from a thermal image, and the air conditioning in the room may be controlled based on this. Specifically, for example, the air conditioning may be controlled based on the thermal sensation of the person, not the pet, based on the result of identifying whether the person is a person or a pet. This can promote comfortable sleep for people. For example, the thermal sensation can be estimated from the temperature difference between the area of ​​the person (animal) in the thermal image and the temperature around that area, but the method of estimating the thermal sensation is not limited to this, and any method may be used. Note that, for example, the air conditioning may be controlled based on the thermal sensation of a pet. For example, when multiple people are sleeping, it may be possible to set which of the two people's thermal sensations the air conditioning should be controlled based on, or the air conditioning may be controlled based on a thermal sensation somewhere between the two people's thermal sensations.

[0133] Furthermore, for example, the one or more subjects may be a person, an animal, or two people. For example, if the one or more subjects are two people, the one or more subjects may be represented as person A and person B. Furthermore, for example, if two people are close to each other in a thermal image and cannot be separated, that part may be omitted from the state detection.

[0134] In the above-described embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU (Central Processing Unit) or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that realizes the devices and the like of the above-described embodiments is a program that causes a computer to execute each step included in the flowchart shown in FIG.

[0135] The following cases are also included in this disclosure:

[0136] (1) The at least one device is specifically a computer system comprising a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The at least one device achieves its function when the microprocessor operates in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate instructions to the computer to achieve a predetermined function.

[0137] (2) Some or all of the components constituting at least one of the above devices may be configured as a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured to include a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program.

[0138] (3) Some or all of the components constituting at least one of the above devices may be configured as an IC card or a standalone module that can be attached to the device. The IC card or module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates according to a computer program. This IC card or module may be tamper-resistant.

[0139] (4) The present disclosure may be embodied as the methods described above, a computer program for implementing these methods on a computer, or a digital signal comprising the computer program.

[0140] The present disclosure may also be a computer program or a digital signal recorded on a computer-readable recording medium, such as a flexible disk, a hard disk, a CD (Compact Disc)-ROM, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc), a semiconductor memory, etc. Alternatively, the present disclosure may be a digital signal recorded on such a recording medium.

[0141] The present disclosure may also be applied to transmitting a computer program or digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, or the like.

[0142] Furthermore, the program or digital signal may be recorded on a recording medium and transferred, or the program or digital signal may be transferred via a network or the like, so that the program or digital signal may be implemented by another independent computer system. [Industrial Applicability]

[0143] The state estimation device and the like according to the present disclosure can be used in devices and the like that estimate the state of one or more estimation targets. [Explanation of symbols]

[0144] 10 State Estimation Device 11 Filming Department 12 Acquisition Department 13 Estimation part 14 Output section 100 State Management System 101 Server

Claims

1. A state estimation method executed by a state estimation device, comprising: An imaging step of imaging one or more objects, each of which is a person or an animal, using a thermal imaging camera; an acquiring step of acquiring identification information for identifying the one or more photographed objects in the thermal image photographed in the photographing step; an estimation step of estimating the state of one or more estimation targets among the one or more photographed targets based on the thermal image photographed in the photographing step and the identification information acquired in the acquisition step, the identification information includes information indicating the number of the one or more subjects to be photographed, In the estimation step, if the number of one or more locations having a temperature equal to or higher than a predetermined temperature in the thermal image captured in the photographing step is greater than the number of the one or more photographed objects, a location with a higher temperature among the one or more locations is preferentially identified as the one or more photographed objects. State estimation methods.

2. In the estimation step, a sleep state of each of the one or more estimation targets is estimated. The state estimation method according to claim 1 .

3. the identification information includes information indicating the number of one or more people included in the one or more shooting subjects, and information indicating the number of one or more animals included in the one or more shooting subjects; In the estimating step, a target that is located at a predetermined position earlier than the other target in the thermal image captured in the capturing step is preferentially identified as a person. The state estimation method according to claim 1 or 2.

4. the identification information includes information indicating a position where each of the one or more estimation targets lies down, In the estimating step, the one or more estimation targets are identified based on positions of the one or more imaging targets in the thermal image captured in the imaging step. The state estimation method according to claim 1 or 2.

5. an imaging unit that uses a thermal imaging camera to capture images of one or more subjects, each of which is a person or an animal; an acquisition unit that acquires identification information for identifying the one or more photographed objects in the thermal image photographed by the photographing unit; an estimation unit that estimates the state of one or more estimation targets among the one or more imaging targets based on the thermal image captured by the imaging unit and the identification information acquired by the acquisition unit, the identification information includes information indicating the number of the one or more subjects to be photographed, When the number of one or more locations having a temperature equal to or higher than a predetermined temperature in the thermal image captured by the image capturing unit is greater than the number of the one or more subjects to be captured, the estimation unit preferentially identifies a location with a higher temperature among the one or more locations as the one or more subjects to be captured. State estimator.

6. A program for causing a computer to execute the state estimation method according to claim 1 or 2.

Images