Image monitoring device, image monitoring method, and control system

Abstract

To prevent a chair or the like warmed by a body temperature of a person from being erroneously detected as a person even after the person leaves the chair in a monitoring target space.SOLUTION: An image monitoring device is provided with: a detection image generation unit (201) configured to generate a detection image including a change region having a prescribed or higher temperature difference in a monitoring target space; an analysis unit (202) configured to analyze a plurality of detection images to temporally track the change region; a detection target exclusion unit (203) configured to set the change region as a detection target when a position of the change region extracted for the first time in the plurality of detection images is within an entrance / exit region of the monitoring target space, and exclude the change region from the detection target when the position of the change region is within an indoor region of the monitoring target space; and a determination unit (204) configured to determine the existence of a constant-temperature moving object in the monitoring target space on the basis of a change region other than the change region excluded from the detection target by the detection target exclusion unit, out of the change region included in the detection image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image monitoring device, an image monitoring method, and a control system. [Background technology]

[0002] There is a known technology for detecting the presence or absence of people and the number of people in a room from a thermal image that shows the surface temperature distribution in the room. For example, Patent Document 1 discloses a human detection device that captures thermal images of the room in chronological order, detects stationary people based on the most recently captured thermal image, and tracks moving people based on the difference between two thermal images captured in succession, thereby identifying the positions of people present in the room.

[0003] When detecting a constant temperature object such as a person in a monitored space based on its thermal image, a background object such as a chair that is warmed by the body heat of the person in the monitored space may continue to be erroneously detected as a person even after the person has left. Therefore, if the lighting is turned on / off based on the person detection result, the lighting will remain on until the temperature of the chair drops after the person has left the seat.

[0004] In response to this, it has been disclosed that an image monitoring device sequentially acquires thermal images of the monitored space, compares each thermal image with a reference image of the monitored space to extract areas of change, and tracks these areas of change over time.The device applies threshold processing to temperature changes that cause the temperature of the above-mentioned areas of change to approach the background temperature over time, thereby determining whether or not the areas of change represent a person (Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-217823 A [Patent Document 2] Patent No. 6000762 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the technology disclosed in Patent Document 2, when noise is included in the thermal image, it is necessary to increase the threshold value in order to distinguish between the temperature change in the change area and the noise. In particular, since noise is large in a low-resolution infrared image sensor, if threshold processing is performed using a large threshold value for the temperature change in the change area, it may not be possible to determine that the change area is not a person even if there is a temperature change, or it may take a long time to determine that the change area is not a person. In addition, the temperature change in the change area tends to change differently depending on the material of the chair, which is heated by human body heat, and the surrounding environment, and there is a possibility that the appropriate threshold value will differ depending on the situation.

[0007] An object of the present invention is to provide an image monitoring device and a control system that do not erroneously detect a chair that has been warmed by the body heat of a person as a person even after the person has left the seat. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, an image monitoring device according to one embodiment of the present invention comprises a detection image creation unit configured to extract a change area having a temperature difference of a predetermined value or more by comparing a thermal image representing a surface temperature distribution in a monitored space with a reference image representing a reference surface temperature distribution in the monitored space, and create a detection image including the extracted change area as a detection target in the monitored space; an analysis unit configured to analyze a plurality of detection images created in time series by the detection image creation unit and track the change area over time; a detection target exclusion unit configured to exclude a change area extracted for the first time in the plurality of detection images from the detection target if the position of the change area extracted for the first time in the plurality of detection images is within a first region of the monitored space, and to make the change area the detection target if the position of the change area extracted for the first time in the plurality of detection images is within a second region of the monitored space; and a determination unit configured to determine the presence of a constant-temperature moving object in the monitored space based on change areas included in the detection image other than the change areas excluded from the detection targets by the detection target exclusion unit.

[0009] In an image monitoring device according to one embodiment of the present invention, the device may further include a detection target revival unit configured to make the changed area a detection target when the changed area excluded from the detection target by the detection target exclusion unit moves over time within the detection image within a predetermined period of time.

[0010] In an image monitoring device according to one embodiment of the present invention, the first area may be an indoor area of ​​the monitored space, and the second area may be an entrance / exit area of ​​the monitored space.

[0011] In this case, a region setting unit may be further provided that sets the range of an indoor region and an entrance / exit region for the monitored space.

[0012] In an image monitoring device according to one embodiment of the present invention, the first area may be a localized location where objects that are not the detection target but have a surface temperature similar to that of the detection target frequently appear.

[0013] In an image monitoring device according to one embodiment of the present invention, the determination unit may be configured to determine that a number of constant-temperature moving objects exist within the monitored space which is the number of change areas included in the detection image minus the number of change areas excluded from the detection target by the detection target exclusion unit.

[0014] In an image monitoring device according to one embodiment of the present invention, the device may further include an entry / exit monitoring unit that monitors changes in the number of constant-temperature moving objects in the monitored space based on the number of constant-temperature moving objects in the monitored space determined by the determination unit.

[0015] Moreover, an image monitoring method according to one embodiment of the present invention includes an imaging step of imaging a monitored space and sequentially generating thermal images representing a surface temperature distribution within the monitored space; a detection image creation step of comparing each of the sequentially generated thermal images with a reference image representing a reference surface temperature distribution of the monitored space to extract changed areas having a temperature difference of a predetermined value or more, and creating a detection image including the extracted changed areas as detection targets in the monitored space; an analysis step configured to analyze the multiple detection images created in time series and track the changed areas included in the multiple detection images over time; a detection target exclusion step of excluding the changed area from the detection target when a position of a changed area extracted for the first time in the multiple detection images created in time series is within a first area of ​​the monitored space, and setting the changed area as the detection target when a position of a changed area extracted for the first time in the multiple detection images is within a second area of ​​the monitored space; and a determination step of determining the presence of a constant-temperature moving object in the monitored space based on the changed areas included in the detection images excluding the changed areas excluded from the detection targets.

[0016] Furthermore, a control system according to one embodiment of the present invention includes any one of the image monitoring devices described above, and a lighting control unit that controls the on / off state of lighting equipment in the monitored space based on the number of constant-temperature moving objects determined by the determination unit to be present in the monitored space and the positions of the constant-temperature moving objects.

[0017] In addition, a control system according to another embodiment of the present invention includes any of the image monitoring devices described above, and an air conditioning control unit that controls the air conditioning capacity within the monitored space based on the number of constant-temperature moving objects determined by the determination unit to be present in the monitored space and the positions of the constant-temperature moving objects. Effect of the Invention

[0018] According to the present invention, it is possible to prevent a chair that has been warmed by the body heat of a person from being erroneously detected as a person even after the person has left the monitored space. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a block diagram showing the configuration of an image monitoring device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of a detection image in the first embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram for explaining the detection operation of the image monitoring device according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram for explaining an example of the hardware configuration of the image monitoring device according to the first embodiment of the present invention. [Diagram 5] FIG. 5 is a flowchart for explaining the operation of the image monitoring device according to the first embodiment of the present invention. [Figure 6] FIG. 6 is a diagram for explaining the detection operation of the image monitoring device according to the first embodiment of the present invention. [Figure 7] FIG. 7 is a diagram for explaining the detection operation of the image monitoring device according to the first embodiment of the present invention. [Figure 8] FIG. 8 is a block diagram showing the configuration of an image monitoring device according to the second embodiment of the present invention. [Figure 9] FIG. 9 is a diagram for explaining the operation of the image monitoring device according to the second embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart for explaining the operation of the image monitoring device according to the second embodiment of the present invention. [Figure 11] FIG. 11 is a block diagram showing the configuration of an image monitoring device according to the third embodiment of the present invention. [Figure 12] FIG. 12 is a flowchart for explaining the operation of the image monitoring device according to the third embodiment of the present invention. [Figure 13] FIG. 13 is a block diagram showing the configuration of an image monitoring device according to the fourth embodiment of the present invention. [Figure 14] FIG. 14 is a flowchart for explaining the operation of the image monitoring device according to the fourth embodiment of the present invention. [Figure 15] FIG. 15 is a block diagram showing a lighting control system including an image monitoring device according to an embodiment of the present invention. [Figure 16] FIG. 16 is a block diagram showing an air conditioning control system including an image monitoring device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0021] [First embodiment] The image monitoring device according to the first embodiment of the present invention detects constant-temperature moving objects in a monitored space, sequentially acquires thermal images of the monitored space, and processes these thermal images to determine the presence or absence of constant-temperature moving objects in the monitored space. As shown in Fig. 1, the image monitoring device 1 according to the present embodiment includes an imaging unit 10, an image processing device 20, a storage device 30, an output unit 40 that outputs the determination result by the image processing device 20, and an input / output device 50.

[0022] The imaging unit 10 is an imaging device that captures an image of a monitored space and generates a thermal image that represents a surface temperature distribution within the monitored space. For example, an infrared image sensor or an infrared sensor array can be used as the imaging device that constitutes the imaging unit 10.

[0023] The image processing device 20 is a device that determines whether or not a constant-temperature moving object exists in the monitored space, based on the thermal images sequentially acquired from the imaging unit 10. The specific configuration and operation of the image processing device 20 will be described later.

[0024] The storage device 30 is a device capable of storing various information, and can be configured from a main storage device, an auxiliary storage device such as an HDD (hard disk drive) or an SSD (solid state drive). In this embodiment, the storage device 30 includes a reference image storage unit 301, a detection image storage unit 302, a tracking information storage unit 303, and an area information storage unit 304. Among these, the reference image storage unit 301 stores a reference image representing a surface temperature distribution serving as a reference for the monitored space, that is, an image representing the surface temperature distribution of the monitored space in a state in which a constant-temperature moving object such as a person does not exist. The detection image storage unit 302 stores detection images created by the detection image creation unit 201 described later in chronological order. The tracking information storage unit 303 stores information on a change area to be tracked in a plurality of detection images created in chronological order, for example, an ID that identifies the change area to be tracked and the coordinates of the change area. The area information storage unit 304 stores information for dividing the monitored space into a plurality of areas. The storage device 30 may also be provided with a set value storage unit 305 for storing various set values ​​for operating the image processing device 20 and a program storage unit 306 for storing programs.

[0025] The output unit 40 is a display device that displays the result of the determination by the image processing device 20 .

[0026] Next, the image processing device 20 will be described with reference to FIGS. As shown in FIG. 1, the image processing device 20 includes a detection image creation unit 201, an analysis unit 202a, a detection target exclusion unit 203, a determination unit 204, and an entry / exit monitoring unit 205.

[0027] The detection image creation unit 201 is configured to compare the thermal image Hn acquired from the imaging unit 10 with the reference image Hs read from the reference image storage unit 301 of the storage device 30, extract a change area having a temperature difference of a predetermined value or more, and create a detection image Dn including the extracted change area as a detection target in the monitored space. As described above, the reference image Hs is a thermal image representing a surface temperature distribution serving as a reference in the monitored space, and is a thermal image representing the surface temperature distribution in the monitored space when no constant-temperature moving object exists. The process of creating the detection image Dn, for example, takes the difference in brightness for each pixel between the thermal image Hn and the reference image Hs, and extracts pixels whose difference exceeds a predetermined threshold value. By such a process, if the thermal image Hn includes an image of a constant-temperature moving object or the like, the image is extracted as a change area having a higher temperature than the reference image, and a detection image Dn including the change area as a detection target is created.

[0028] The detection image creation unit 201 creates detection images Dn-2, Dn-1, Dn each time multiple thermal images Hn-2, Hn-1, Hn are acquired in chronological order from the imaging unit 10, stores them in chronological order in the detection image memory unit 302, and inputs them to the analysis unit 202a.

[0029] An example of the detection image Dn is shown in Fig. 2. The detection image Dn includes a boundary 60 of the monitored space, and if an object with a certain temperature or higher exists in the monitored space, that part appears as a changed area 61. Therefore, if there is a person in the monitored space, that person will appear as a changed area in the detection image, and a chair warmed by the person's body heat will also appear as a changed area as long as the temperature remains above a certain temperature even after the person leaves the seat.

[0030] In this embodiment, as shown in FIG. 2, the monitored space is divided into an entrance / exit area 62 and an indoor area 63. The entrance / exit area 62 is, for example, an area near an entrance to a floor that is the monitored space, and is an area where people may enter and exit the monitored space. The indoor area 63 is an area of ​​the detected image excluding the entrance / exit area 62. The ranges of the entrance / exit area 62 and the indoor area 63 can be set via the input / output device 50, and information about the set areas may be stored in the area information storage unit 304 of the storage device 30. A method of image detection based on such areas will be described later.

[0031] The analysis unit 202a is configured to analyze a plurality of detection images created in time series by the detection image creation unit 201, and to track the changed area included in the plurality of detection images over time. More specifically, the analysis unit 202a detects a changed area included in the detection image Dn every time a detection image Dn created by the detection image creation unit 201 is input, and assigns identification information to each changed area and sequentially stores the coordinates An in the tracking information storage unit 303 of the storage device 30.

[0032] In addition, the analysis unit 202a reads out the most recently created detection image Dn-1 and the identification information and coordinates An-1 of the changed area contained in the detection image Dn-1 from the detection image memory unit 302 and the tracking information memory unit 303, respectively, and compares the currently created detection image Dn with the most recently created detection image Dn-1 to determine whether a certain changed area is present in both the currently created detection image Dn and the previously created detection image Dn-1, or whether the changed area is present in only one of the two detection images Dn, Dn-1 created in succession, and whether the changed area is trackable.

[0033] For example, by comparing two detection images Dn and Dn-1, if the change areas detected in the respective detection images at least partially overlap each other, or if the center of gravity of each change area is calculated and the distance between the two centers of gravity is equal to or less than a predetermined value, it is determined that the change areas are common to the currently created detection image Dn and the previously created detection image Dn-1, and that the movement of the change area in the monitored space, i.e., the change over time in the position of the change area, is traceable. A change area determined to be traceable in this way can be considered to represent a constant temperature moving object such as a person.

[0034] On the other hand, a changed region that exists in only one of the two detection images Dn, Dn-1 created in succession is judged as follows. First, for any changed area that was included in the previously created detection image Dn-1 but is not included in the currently created detection image Dn, it is assumed that the area has disappeared from the monitored space, and it is determined that the area is untraceable, and tracking of the area is terminated. In contrast, a changed area that was not included in the previously created detection image Dn-1 but is included in the currently created detection image Dn, i.e., a changed area that is newly detected within the monitored space, is determined to be trackable in principle and becomes the target for subsequent tracking.

[0035] The information thus obtained regarding whether or not the changed area can be tracked is stored in the tracking information storage unit 303 together with the identification information of the changed area and the coordinates An, which are linked to each other.

[0036] The detection target exclusion unit 203 is configured to exclude from the detection targets those changed areas that have been determined to be trackable by the analysis unit 202a and that have been newly detected within the monitored space and that have been detected in the indoor area 63 (see Figure 2).

[0037] As described above, in this embodiment, the monitored space is divided into an entrance / exit area 62 and an indoor area 63 as shown in FIG. 2. The indoor area 63 is generally an area that cannot be accessed without passing through the entrance / exit area 62. Such an indoor area 63 can basically be an area in which a person who has not been captured in a temperature image until now does not suddenly appear. In this case, a newly detected changed area in the indoor area 63 of the monitored space may not be a person who has entered the monitored space, but may be furniture such as a chair that has been warmed by the body heat of a person who has been in the monitored space for some time. Therefore, a newly detected changed area in the monitored space that is detected in the indoor area 63 is excluded from the detection target.

[0038] On the other hand, the entrance / exit area 62 is the remaining area excluding the indoor area 63 from the monitored space, is the area near the entrance to the floor which is the monitored space, and is an area where people may enter and leave the monitored space, so in the entrance / exit area 62, when a person who has not been captured in the thermal image so far enters the monitored space, this person will appear as a new changed area in the detection image, and when a person who has been captured in the thermal image so far leaves the monitored space, the changed area will disappear from the detection image. Therefore, it is appropriate to make the newly appeared changed area in the entrance / exit area 62 the detection target. The indoor area 63 may be limited to a localized area where objects that are not the detection target but have a surface temperature similar to that of the detection target frequently appear.

[0039] Figure 3 is a diagram to explain the interpretation when a new change area appears or disappears. This is a summary of the positions and meanings of change areas that exist in only one of two detection images Dn and Dn-1, which were created in succession.

[0040] Regarding a changed area that was included in the previously created detection image Dn-1 ("present") but not included in the currently created detection image Dn ("not present"), if the changed area in the previously created detection image Dn-1 is within the entrance / exit area 62, it can be considered that a person has "left the room." On the other hand, if the changed area that was included in the previously created detection image Dn-1 ("present") is within the indoor area 63 but not included in the currently created detection image Dn ("not present"), it must be interpreted as a "sudden disappearance," and in this case, it is suspected that the changed area was not detected ("lost") in the currently created detection image Dn.

[0041] In contrast, for a change area that was not included in the previously created detection image Dn-1 ("not there") but is included in the currently created detection image Dn ("present"), if the change area is located within the entrance / exit area 62 of the monitored space (if the coordinates of the change area in the currently created detection image Dn are within the entrance / exit area 62), it can be assumed that a person who has not been captured in the thermal image until now has suddenly appeared, and that a person has "entered the room." On the other hand, if the newly detected change area is located within the indoor area 63, such as the center of the floor, it must be interpreted as a "sudden appearance," and in this case, since an indoor area is a place where it is difficult for a person to suddenly appear, there is a high possibility that an object other than a person will be erroneously detected as a person ("ghost") in the currently created detection image Dn. For example, if a new changed area appears within an indoor area of ​​a monitored space, it can be assumed that the changed area does not represent a person, but rather that a non-constant temperature moving object such as a chair that has been warmed by the person's body heat has been recognized as a changed area because the person has left the chair.

[0042] Therefore, in the image monitoring device 1 of this embodiment, for a newly detected changed area, the detection target exclusion unit 203 checks the coordinates of the changed area, and if the position of the changed area is within the entrance / exit area 62 of the monitored space, it determines that the changed area is the detection target, i.e., that it is a person, whereas if it is within the indoor area 63, it determines that the changed area is not a person.

[0043] Information regarding the area of ​​the monitored space in the detection image may be stored in the area information storage unit 304 of the storage device 30, and the detection target exclusion unit 203 may refer to or read out this information.

[0044] The determination unit 204 is configured to determine the presence of constant-temperature moving objects in the monitored space based on, among the changed areas included in the detection image Dn, changed areas other than the changed areas excluded from the detection target by the detection target exclusion unit 203. More specifically, in this embodiment, the determination unit 204 is configured to count the number of changed areas included in the detection image Dn minus the number of changed areas excluded from the detection target by the detection target exclusion unit 203, that is, the number of changed areas determined to be people, and determine that the number of constant-temperature moving objects exists in the monitored space.

[0045] The entrance / exit monitoring unit 205 is configured to monitor the changes in the number of people staying in the monitored space based on the number of constant-temperature moving objects, i.e., people, in the monitored space determined by the determining unit 204.

[0046] The image processing device 20 as described above can be realized, for example, by installing an image monitoring program in a computer equipped with a CPU (Central Processing Unit), memory, and an input / output interface as shown in FIG.

[0047] Next, the operation of the image monitoring device 1 according to this embodiment will be described with reference to Fig. 2, Fig. 5 to Fig. 7. In the image monitoring device 1 according to this embodiment, it is assumed that thermal images Hn-2, Hn-1, Hn, ... are input in chronological order from the imaging unit 10 to the image processing device 20, and detection images Dn-2, Dn-1 are created in chronological order from the thermal images Hn-2, Hn-1.

[0048] First, when a thermal image Hn is acquired from the imaging unit 10 (S10 in FIG. 5), the image processing device 20 compares the thermal image Hn with a reference image Hs stored in advance in the reference image storage unit 301 of the storage device 30 in the detection image creation unit 201 to create a detection image Dn (S11 in FIG. 5). The created detection image Dn is input to the analysis unit 202a and is stored in the detection image storage unit 302 of the storage device 30.

[0049] When detection images Dn created in chronological order are input, the analysis unit 202a detects changed areas included in the detection image Dn, specifies coordinates An for each changed area, and stores them in the tracking information storage unit 303 of the storage device 30 (S12 in FIG. 5). In addition, the analysis unit 202a reads out coordinates An-1 of the changed area in the immediately preceding detection image Dn-1 from the tracking information storage unit 303 of the storage device 30, and compares them with coordinates An of the changed area in the detection image Dn (S13 in FIG. 5), to determine whether tracking is possible (S14 in FIG. 5).

[0050] Any changed area that was included in the immediately preceding detection image Dn-1 but not included in the current detection image Dn is determined to be untraceable (S14:N in FIG. 5), and such changed area is excluded from the detection targets for determining whether or not it is a person (S17a in FIG. 5).

[0051] On the other hand, for any changed areas that are determined to be trackable, including any changed areas that have been newly detected in the current detection image Dn (S14:Y in FIG. 5), it is confirmed whether or not the changed areas are newly detected in the detection image Dn (S15 in FIG. 5).

[0052] If the changed area is tracked from a changed area contained in the immediately preceding detection image Dn-1, that is, if the changed area is not a changed area newly detected in the detection image Dn (S15:N in FIG. 5), the judgment unit 204 judges that the changed area is a person (S17b in FIG. 5).

[0053] On the other hand, if the changed area is a changed area that has been newly detected in the detection image Dn (S15: Y in FIG. 5), the position of the changed area is confirmed (S16 in FIG. 5). If the newly detected changed area in the detection image Dn is detected in the indoor area 63 (S16:N in FIG. 5), the changed area is excluded from the detection target for determining whether or not it is a person (S17a in FIG. 5), and if the newly detected changed area in the detection image Dn is detected in the entrance / exit area 62 (S16:Y in FIG. 5), the changed area is determined to be a person (S17b in FIG. 5).

[0054] The above-mentioned determination results are tallied by the determination unit 204 (S18 in FIG. 5). The tallied result is the number obtained by subtracting the number of changed areas excluded from detection targets by the detection target exclusion unit 203 from among the changed areas included in the detection image Dn, that is, the number of changed areas determined to be people (S17b in FIG. 5). The tallied result is output to the output unit 40 as a value representing the number of constant-temperature moving objects present in the monitored space (S19 in FIG. 5). Then, if the next thermal image is input from the imaging unit 10 (S20:Y in FIG. 5), the process from S10 is repeated, and if the next thermal image is not input within a predetermined time (S20:N in FIG. 5), the series of processes is terminated.

[0055] An example of a detection image when a person enters the monitored space is shown in Fig. 6. Fig. 6 shows how a changed area detected in an entrance / exit area 62 at time N is tracked at time N+1. If a new changed area 61 appears in a detection image Dn at time N and is located in the entrance / exit area 62, the changed area 61 is determined to be a person. By making such a new changed area 61 a tracking target, it is determined to be a person even if it subsequently moves to an indoor area 63, as in detection image Dn+1.

[0056] On the other hand, an example in which a person appears to have suddenly appeared in the indoor region 63 is shown in Fig. 7. Fig. 7 shows that a changed region that did not appear in detection images Dn-1 and Dn at times N-1 and N appears in the indoor region 63 in detection image Dn+1 at time N+1. For example, a chair placed in the indoor region 63 may be detected as a changed region after a person who was sitting on the chair leaves the chair. In this case, when a changed region is detected for the first time at time N+1, it appears as if a person has suddenly appeared, but since the person is in the indoor region 63, which is an area that cannot be entered without passing through the entrance / exit region 62, such a changed region is excluded from detection targets and determined not to be a person.

[0057] According to the image monitoring device 1 of this embodiment, when a new change area is detected, it is confirmed whether the location of the change area is an entrance / exit area or an indoor area of ​​the monitored space, and if it is an indoor area, the change area is excluded from the detection target. Therefore, it is possible to prevent a chair warmed by the person's body heat from being mistakenly detected as a person even after the person has left the monitored space.

[0058] [Second embodiment] In the first embodiment described above, the most recently created detection image Dn-1 and the identification information and coordinates An-1 of the changed area contained in that detection image Dn-1 are read out, and the currently created detection image Dn is compared with the most recently created detection image Dn-1 to determine whether a certain changed area is trackable. In contrast, the image monitoring device 1a according to the second embodiment of the present invention determines whether the changed area can be tracked using only the currently created detection image Dn, the identification information of the changed area, and the coordinates An-1. This differs from the image monitoring device 1 according to the first embodiment in that it does not have the detection image storage unit 302.

[0059] The configuration of an image monitoring device 1a according to a second embodiment of the present invention is shown in Fig. 8. Components common to the image monitoring device 1 according to the first embodiment are given the same reference numerals and detailed explanations are omitted. The operation of the image monitoring device 1a according to this embodiment is shown in the flowchart of Fig. 10. However, since the flowchart of Fig. 10 is similar to the operation flowchart of Fig. 5 according to the first embodiment, detailed explanations are omitted.

[0060] As shown in Fig. 8, the image monitoring device 1a according to this embodiment includes an imaging section 10, an image processing device 20a, a storage device 30a, and an output section 40 that outputs the determination result by the image processing device 20a. Here, the image processing device 20a includes a detection image creation section 201, an analysis section 202b, a detection target exclusion section 203, a determination section 204, and an entry / exit monitoring section 205. In addition, the storage device 30a includes a reference image storage section 301, a tracking information storage section 303, and an area information storage section 304.

[0061] The detection image creation unit 201 creates detection images Dn-2, Dn-1, Dn each time multiple thermal images Hn-2, Hn-1, Hn are acquired in chronological order from the imaging unit 10 (S10 in FIG. 10) and inputs them to the analysis unit 202b (S11 in FIG. 10).

[0062] The analysis unit 202b reads out the identification information and coordinates An-1 of the changed area created immediately before from the tracking information storage unit 303 (S12 in FIG. 10), and compares the identification information and coordinates An-1 of the changed area created immediately before with the detection image Dn created this time and the detection image created immediately before (S13 in FIG. 10), and determines whether a certain changed area is traceable depending on whether it exists in both the detection image Dn created this time and the identification information and coordinates An-1 of the changed area created immediately before, or whether it exists in only one of the detection image Dn or the identification information and coordinates An-1 of the changed area (S14 in FIG. 10).

[0063] For example, by comparing the detection image Dn with the identification information of the changed area and the coordinate An-1, if the changed area detected in the detection image Dn overlaps at least partially with the identification information of the changed area and the coordinate An-1, or by calculating the center of gravity of the changed area of ​​the detection image Dn and the distance between the center of gravity and the identification information of the changed area and the coordinate An-1 is less than a predetermined value, the identification information of the changed area created immediately before and the current coordinate An-1 are determined to be a changed area present in the detection image Dn created this time, and the movement of the changed area within the monitored space, i.e., the temporal change in the position of the changed area, is determined to be traceable. A changed area determined to be traceable in this way can be considered to represent a constant temperature moving object such as a person.

[0064] On the other hand, a changed area that exists in only one of the detection image Dn, the identification information of the changed area, and the coordinates An-1 is determined as follows. First, for any changed area that was included in the identification information and coordinates An-1 of the changed area that was the target of tracking until the previous time but is not included in the detection image Dn created this time, it is assumed that the changed area has disappeared from the monitored space, and it is determined that the area is untraceable, and tracking of that area is terminated. In contrast to this, a changed area that was not included in the identification information and coordinates An-1 of the changed area that was the target of tracking previously but is included in the detection image Dn created this time, that is, a changed area that has been newly detected within the monitored space, is determined in principle to be trackable and becomes the target of tracking thereafter (S15 in FIG. 10).

[0065] The information thus obtained regarding whether or not the changed area can be tracked is stored in the tracking information storage unit 303 in association with the identification information of the changed area and the coordinates An.

[0066] FIG. 9 is a diagram for explaining the interpretation of when a changed area newly appears or disappears, and summarizes the positions and meanings within the monitored space of changed areas that exist only in either the detected image Dn or the detected image with the identification information of the changed area and the coordinates An-1.

[0067] For a changed area that was included in the previous changed area identification information and coordinates An-1 ("present") but is not included in the currently created detection image Dn ("not present"), if the coordinates in the previous changed area identification information and coordinates An-1 are within the entrance / exit area 62, it can be considered that a person has "left the room." On the other hand, if a changed area that was included in the previous changed area identification information and coordinates An-1 ("present") is within the indoor area 63 but is not included in the currently created detection image Dn ("not present"), it must be interpreted as a "sudden disappearance," and in this case, it is suspected that a changed area has been overlooked ("lost") in the currently created detection image Dn.

[0068] In contrast, for a changed area that was not included in the previous change area identification information and coordinates An-1 ("not there") but is included in the currently created detection image Dn ("present"), if the position is within the entrance / exit area 62 of the monitored space (if the coordinates of the changed area in the currently created detection image Dn are within the entrance / exit area 62), it can be assumed that a person who has not been captured in the thermal image so far suddenly appears, and it can be considered that a person has "entered" (S16:Y in FIG. 10). On the other hand, if the position of the newly detected changed area is within the indoor area 63, such as the center of the floor, it must be interpreted as "suddenly appearing." In this case, since the indoor area is a place where it is difficult for a person to suddenly appear, there is a high possibility that an object other than a person will be erroneously detected as a person ("ghost") in the currently created detection image Dn (S16:N in FIG. 10). For example, if a new changed area appears within an indoor area of ​​a monitored space, it can be assumed that the changed area does not represent a person, but rather that a non-constant temperature moving object such as a chair that has been warmed by the person's body heat has been recognized as a changed area because the person has left the chair.

[0069] Therefore, in the image monitoring device 1a of this embodiment, for a newly detected changed area, the detection target exclusion unit 203 checks the coordinates of the changed area, and if the position of the changed area is within the entrance / exit area 62 of the monitored space, it determines that the changed area is the detection target, i.e., that it is a person (S17b in FIG. 10), whereas if the position is within the indoor area 63, it determines that the changed area is not a person (17a in FIG. 10).

[0070] [Third embodiment] In the first embodiment described above, when a newly detected changed area in a detection image generated in time series is detected within the indoor area 63 of the monitored space, the changed area is determined to not be a person, and the changed area is excluded from the detection targets. In contrast, an image monitoring device 1b according to a third embodiment of the present invention is similar to the image monitoring device 1 according to the first embodiment in that it excludes from the detection target a changed area newly detected within the indoor area 63 of the monitored space, but differs from the image monitoring device 1 according to the first embodiment in that it tracks a changed area that has been excluded from the detection target, and if the changed area moves within the detection image within a specified period of time, it includes the changed area as a detection target.

[0071] The configuration of an image monitoring device 1b according to a third embodiment of the present invention is shown in Fig. 11. Note that components common to the image monitoring device 1 according to the first embodiment are given the same reference numerals and detailed explanations are omitted. Furthermore, the operation of the image monitoring device 1b according to this embodiment will be explained based on the flowchart in Fig. 12.

[0072] 11, an image monitoring device 1b according to this embodiment includes an imaging unit 10, an image processing device 20b, a storage device 30, and an output unit 40 that outputs a determination result by the image processing device 20b. Here, the image processing device 20b includes a detection image creation unit 201, an analysis unit 202a, a detection target exclusion unit 203, a detection target restoration unit 206, a determination unit 204, and an entry / exit monitoring unit 205.

[0073] The detection target revival unit 206, which characterizes the image monitoring device 1b in the third embodiment, is configured to track the changed area that has been excluded from the detection targets by the detection target exclusion unit 203, and if the changed area moves within a specified period of time, to make the changed area a detection target.

[0074] As shown in FIG. 12, image monitoring device 1b of the third embodiment performs processes from S10 to S16 in the same way as image monitoring device 1 of the first embodiment, but tracks the changed area excluded from the detection targets by detection target exclusion unit 203 (S16a in FIG. 12), and if the changed area moves within a specified period of time (S16a:Y in FIG. 12), it restores the changed area as a detection target and judges it to be a person (S17b in FIG. 12), and if it does not move within the specified period of time (S16a:N in FIG. 12), it judges that the changed area is not a person and excludes it from the detection targets (S17b in FIG. 12).

[0075] Applying this to the example of Fig. 7, if a changed area that did not appear in detection images Dn-1, Dn at times N-1, N appears in indoor area 63 in detection image Dn+1 at time N+1, such a changed area is initially determined to not be a person and is excluded from detection targets (S16:N in Fig. 12). However, if the changed area is tracked for a certain period of time and characteristics such as the changed area moving are observed (S16a:Y in Fig. 12), it is determined to be a person (S17b in Fig. 12).

[0076] Thus, according to the image monitoring device 1b of the third embodiment, even if a changed area is not detected in the immediately preceding detection image, it can be treated as a detection target by being equipped with the detection target revival unit 206, and the number of constant-temperature moving objects present in the monitored space can be accurately grasped.

[0077] [Fourth embodiment] In the third embodiment described above, the most recently created detection image Dn-1 and the identification information and coordinates An-1 of the changed area contained in that detection image Dn-1 are read out, and the currently created detection image Dn is compared with the most recently created detection image Dn-1 to determine whether a certain changed area is trackable. In contrast, the image monitoring device 1c according to the fourth embodiment of the present invention determines whether the changed area can be traced using only the currently created detection image Dn, the identification information of the changed area, and the coordinates An-1. It differs from the image monitoring device 1a according to the second embodiment in that it has a detection target restoration unit 206, and it differs from the image monitoring device 1b according to the third embodiment in that it does not have a detection image storage unit 302.

[0078] The configuration of an image monitoring device 1c according to a fourth embodiment of the present invention is shown in FIG. 13. Components common to the image monitoring device 1b according to the third embodiment are given the same reference numerals and detailed description thereof will be omitted. Components not common to the image monitoring device 1b according to the third embodiment are the same as the differences between the image monitoring device 1 according to the first embodiment and the image monitoring device 1a according to the second embodiment, so are given the same reference numerals and detailed description thereof will be omitted. The operation of the image monitoring device 1c according to this embodiment is shown in the flowchart of FIG. 14. However, since the flowchart of FIG. 14 is similar to the operation flowchart of FIG. 12 according to the third embodiment, detailed description thereof will be omitted.

[0079] 13, an image monitoring device 1c according to the present embodiment includes an imaging unit 10, an image processing device 20c, a storage device 30a, and an output unit 40 that outputs a determination result by the image processing device 20c. Here, the image processing device 20c includes a detection image creation unit 201, an analysis unit 202b, a detection target exclusion unit 203, a detection target restoration unit 206, a determination unit 204, and an entry / exit monitoring unit 205.

[0080] According to the image monitoring device 1c of this embodiment, as in the second embodiment, there is no need to provide a detection image storage unit for storing detection images in the storage device 30, and the storage capacity for storing images can be reduced. Also, as in the third embodiment, even if a changed area is not detected in the immediately preceding detection image, it can be made a detection target by providing the detection target revival unit 206, and the number of constant-temperature moving objects present in the monitored space can be accurately grasped.

[0081] <Application Examples> The image monitoring device according to an embodiment of the present invention can be applied to various control systems having different control targets. For example, as shown in FIG. 15, a lighting control system 7 can be configured by combining an image monitoring device 1, 1a, 1b, 1c according to an embodiment of the present invention with a lighting control unit 70 that controls the on / off state of lighting equipment 71 in the monitored space based on the number of constant-temperature moving objects determined by a determination unit 204 to be present in the monitored space and the positions of those constant-temperature moving objects. Furthermore, as shown in FIG. 16, an air conditioning control system 8 can be configured by combining an image monitoring device 1, 1a, 1b, 1c relating to one embodiment of the present invention with an air conditioning control unit 80 that controls air conditioning equipment 81 in the monitored space based on the number of constant-temperature moving objects determined by the determination unit 204 to be present in the monitored space and the positions of those constant-temperature moving objects. In this case, if the monitored space is divided into multiple zones and the image monitoring devices 1, 1a, 1b, and 1c are configured to grasp the number of temperature-controlled moving objects present in each zone, it becomes possible to perform detailed lighting and air conditioning control according to the presence or absence of people in each zone.

[0082] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Industrial Applicability]

[0083] The present invention can be used in image monitoring devices and control systems. [Explanation of symbols]

[0084] 1, 1a, 1b, 1c...image monitoring device, 10...imaging unit, 20, 20a...image processing device, 30, 30a...storage device, 40...output unit, 201...detection image creation unit, 202a, 202b...analysis unit, 203...detection target exclusion unit, 204...determination unit, 205...entrance / exit monitoring unit, 206...detection target restoration unit, 61...changed area, 62...entrance / exit area, 63...indoor area, 7...lighting control system, 70...lighting control unit, 71...lighting equipment, 8...air conditioning control system, 80...air conditioning control unit, 81...air conditioning equipment.

Claims

1. a detection image creation unit configured to compare a thermal image showing a surface temperature distribution in a monitored space with a reference image showing a reference surface temperature distribution in the monitored space, extract a change area having a temperature difference of a predetermined value or more, and create a detection image including the extracted change area as a detection target in the monitored space; an analysis unit configured to analyze a plurality of detection images created in time series by the detection image creation unit to track the change area over time; a detection target exclusion unit configured to exclude a changed region that is extracted for the first time in the plurality of detection images from a detection target when the position of the changed region that is extracted for the first time is within a first region of the monitored space, and to include the changed region in the detection target when the position of the changed region that is extracted for the first time in the plurality of detection images is within a second region of the monitored space; a determination unit configured to determine the presence of a constant-temperature moving object in the monitored space based on a change area other than the change area excluded from the detection target by the detection target exclusion unit, among change areas included in the detection image; An image monitoring device comprising:

2. 2. The image monitoring device according to claim 1, The image monitoring device further includes a detection target revival unit configured to make the change area a detection target when the change area excluded from the detection target by the detection target exclusion unit moves over time in the detection image within a predetermined period of time.

3. 3. The image monitoring device according to claim 1, An image monitoring device, wherein the first area is an indoor area of ​​the monitored space, and the second area is an entrance / exit area of ​​the monitored space.

4. 4. The image monitoring device according to claim 3, The image monitoring device further comprises an area setting unit that sets the range of an indoor area and an entrance / exit area for the monitored space.

5. 3. The image monitoring device according to claim 1, An image monitoring device, wherein the first region is a localized location where objects having a surface temperature similar to that of the detection target frequently appear even though they are not the detection target.

6. 3. The image monitoring device according to claim 1, An image monitoring device, wherein the determination unit is configured to determine that a number of constant-temperature moving objects exists within the monitored space, the number of change areas being calculated by subtracting the number of change areas excluded from the detection target by the detection target exclusion unit from the number of change areas included in the detection image.

7. 3. The image monitoring device according to claim 1, The image monitoring device further includes an entry / exit monitoring unit that monitors changes in the number of constant-temperature moving objects in the monitored space based on the number of constant-temperature moving objects in the monitored space determined by the determination unit.

8. an imaging step of imaging a monitored space and sequentially generating thermal images representing a surface temperature distribution within the monitored space; a detection image creation step of comparing each of the sequentially generated thermal images with a reference image that represents a reference surface temperature distribution in the monitored space, extracting a change area having a temperature difference of a predetermined value or more, and creating a detection image that includes the extracted change area as a detection target in the monitored space; an analysis step configured to analyze a plurality of detection images created in time series and to track the change region included in the plurality of detection images over time; a detection target excluding step of excluding a changed region extracted for the first time from the plurality of detection images created in time series when the position of the changed region is within a first region of the monitored space, and setting the changed region as the detection target when the position of the changed region extracted for the first time from the plurality of detection images is within a second region of the monitored space; a determination step of determining the presence of a constant-temperature moving object in the monitored space based on a change area excluding a change area excluded from a detection target from among change areas included in the detection image; An image monitoring method comprising:

9. An image monitoring device according to claim 1 or 2; a lighting control unit that controls a lighting state of lighting equipment in the monitored space based on the number of constant-temperature moving objects determined to be present in the monitored space by the determination unit and the positions of the constant-temperature moving objects; A control system having

10. An image monitoring device according to claim 1 or 2; an air conditioning control unit that controls an air conditioning capacity in the monitored space based on the number of constant-temperature moving objects determined to be present in the monitored space by the determination unit and the position of the constant-temperature moving object n; A control system having

Images