Image monitoring apparatus, image monitoring method, and control system

Abstract

The invention provides an image monitoring apparatus, an image monitoring method, and a control system. The present invention is provided with: a detection image creation unit configured to create a detection image including a change region having a temperature difference equal to or greater than a predetermined temperature difference in a space to be monitored; an analysis unit configured to detect the plurality of detection images and track a change region over time; and a detection object excluding unit configured to, when the position of the change region extracted for the first time in the plurality of detection images is within the entrance / exit region of the space to be monitored, use the change region as a detection object, and, when the position of the change region is within the indoor region of the space to be monitored, use the change region as a detection object. Excluding the change area from a detection object; and a determination unit configured to determine the presence of a constant-temperature moving object in the monitoring object space on the basis of a change region other than the change region excluded from the detection object by the detection object excluding unit among the change regions included in the detection object.

Description

Technical Field

[0001] The invention relates to an image monitoring device, an image monitoring method and a control system. Background Art

[0002] There is a known technology for detecting whether there are people or the number of people in a room based on a thermal image showing the surface temperature distribution in the room. For example, Patent Document 1 discloses a human detection device that takes thermal images of a room in a time series, detects stationary people based on the most recently taken thermal image, and tracks moving people based on the difference between two thermal images taken before and after, thereby determining the position of the people in the room.

[0003] When a constant-temperature object such as a person in the monitored space is detected based on the thermal image, a background object such as a chair heated by the body temperature of a person in the monitored space may continue to be mistakenly detected as a person even after the person leaves. Therefore, if the lighting is turned on / off based on the person detection result, the lighting will continue to be turned on during the period after the person leaves the seat and before the temperature of the chair drops.

[0004] In contrast, an image monitoring device is disclosed which sequentially acquires thermal images taken of a monitored object space, compares each thermal image with a reference image of the monitored object space to extract a changed area, and tracks the changed area over time. By performing threshold processing on the temperature change of the changed area so that the temperature approaches the background temperature over time, it is determined whether the changed area is a person (Patent Document 2).

[0005] [Prior art literature]

[0006] [Patent Document]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2016-217823

[0008] [Patent Document 2] Japanese Patent No. 6000762 Summary of the invention

[0009] [Problems to be solved by the invention]

[0010] However, in the technology disclosed in Patent Document 2, when the thermal image contains noise, it is necessary to increase the threshold value in order to distinguish the temperature change of the change area from the noise. In particular, in a low-resolution infrared image sensor, since the noise is large, when threshold processing is performed on the temperature change of the change area using a large threshold value, there is a possibility that even if there is a temperature change, it cannot be determined that the change area is not a person, or it takes time to make the determination.

[0011] In addition, the temperature change in the changing area has different tendency depending on the material of the chair heated by human body temperature or the surrounding environment, and the appropriate threshold value may differ depending on the scene.

[0012] An object of the present invention is to provide an image monitoring device and a control system that prevents a chair or the like heated by a person's body temperature from being erroneously detected as a person even after the person leaves the seat.

[0013] [Technical means to solve the problem]

[0014] In order to achieve the above-mentioned purpose, an image monitoring device according to one embodiment of the present invention includes: a detection image production unit, configured to compare a thermal image representing a surface temperature distribution of a monitoring object space with a reference image representing a reference surface temperature distribution of the monitoring object space, extract a change region having a temperature difference greater than a specified value, and thereby produce a detection image including the extracted change region as a detection object in the monitoring object space; an analysis unit, configured to analyze a plurality of detection images produced in a time series by the detection image production unit, and track the change region over time; a detection object exclusion unit, configured to exclude the change region from the detection object when the position of the change region extracted first in the plurality of detection images is within a first region of the monitoring object space, and to treat the change region as a detection object when the position of the change region extracted first in the plurality of detection images is within a second region of the monitoring object space; and a determination unit, configured to determine the presence of a constant-temperature moving object in the monitoring object space based on a change region other than the change region excluded from the detection object by the detection object exclusion unit among the change regions included in the detection image.

[0015] In an image monitoring device according to one embodiment of the present invention, a detection object recovery unit may also be included, wherein when a change area excluded from the detection object by the detection object exclusion unit moves over time in the detection image within a specified period, the change area is treated as a detection object.

[0016] In the image monitoring device according to one embodiment of the present invention, the first area may be an indoor area of ​​the monitoring target space, and the second area may be an entrance area of ​​the monitoring target space.

[0017] In this case, an area setting unit may be further provided to set the range of an indoor area and an entrance and exit area for the monitored space.

[0018] In the image monitoring device according to one embodiment of the present invention, the first area may be set to a local area where objects that are not detection targets but have a surface temperature similar to that of the detection target frequently appear.

[0019] In an image monitoring device according to one embodiment of the present invention, the determination unit may be configured to determine that there are constant-temperature moving objects in the monitored object space, the number of which is the number obtained by subtracting the number of change areas excluded from the detection object by the detection object exclusion unit from the number of change areas included in the detection image.

[0020] The image monitoring device according to one embodiment of the present invention may further include an entry and exit monitoring unit configured to monitor changes in the number of constant temperature moving objects in the monitoring space based on the number of constant temperature moving objects in the monitoring space determined by the determination unit.

[0021] In addition, an image monitoring method according to an embodiment of the present invention includes: a shooting step of shooting a monitored object space and sequentially generating thermal images representing the surface temperature distribution in the monitored object space; a detection image production step of comparing each of the plurality of thermal images generated sequentially with a reference image representing the reference surface temperature distribution of the monitored object space, extracting a change region having a temperature difference greater than a specified value, and thereby producing a detection image including the extracted change region as a detection object in the monitored object space; an analysis step of analyzing the plurality of detection images produced in a time series and tracking the change region included in the plurality of detection images over time; a detection object exclusion step of excluding the change region from the detection object when the position of the change region extracted first in the plurality of detection images produced in a time series is within the first region of the monitored object space, and treating the change region as the detection object when the position of the change region extracted first in the plurality of detection images is within the second region of the monitored object space; and a determination step of determining the presence of a constant-temperature moving object in the monitored object space based on the change region included in the detection image excluding the change region excluded from the detection object.

[0022] In addition, a control system of one embodiment of the present invention comprises: any one of the image monitoring devices; and a lighting control unit, which controls the lighting / extinguishing state of the lighting equipment in the monitored object space based on the number of constant temperature moving objects determined by the determination unit to exist in the monitored object space and the positions of the constant temperature moving objects.

[0023] In addition, a control system of another embodiment of the present invention comprises: any one of the image monitoring devices; and an air conditioning control unit, which controls the air conditioning capacity in the monitored space based on the number of constant temperature moving objects determined by the determination unit to be present in the monitored space and the positions of the constant temperature moving objects.

[0024] [Effects of the Invention]

[0025] According to the present invention, even if a person leaves his / her seat in the monitored space, it is possible to prevent a chair or the like heated by the person's body temperature from being erroneously detected as a person. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a block diagram showing the structure of the image monitoring device according to the first embodiment of the present invention.

[0027] Figure 2 This is a diagram for explaining an example of a detection image according to the first embodiment of the present invention.

[0028] Figure 3 It is a diagram for explaining the detection operation of the image monitoring device according to the first embodiment of the present invention.

[0029] Figure 4 This is a diagram for explaining a hardware configuration example of the image monitoring device according to the first embodiment of the present invention.

[0030] Figure 5 This is a flowchart for explaining the operation of the image monitoring device according to the first embodiment of the present invention.

[0031] Figure 6 It is a diagram for explaining the detection operation of the image monitoring device according to the first embodiment of the present invention.

[0032] Figure 7 It is a diagram for explaining the detection operation of the image monitoring device according to the first embodiment of the present invention.

[0033] Figure 8 It is a block diagram showing the structure of an image monitoring device according to a second embodiment of the present invention.

[0034] Fig. 9 It is a diagram for explaining the operation of the image monitoring device according to the second embodiment of the present invention.

[0035] Fig.10 This is a flowchart for explaining the operation of the image monitoring device according to the second embodiment of the present invention.

[0036] Fig.11 This is a block diagram showing the structure of an image monitoring device according to a third embodiment of the present invention.

[0037] Fig.12 This is a flowchart for explaining the operation of the image monitoring device according to the third embodiment of the present invention.

[0038] Fig.13 This is a block diagram showing the structure of an image monitoring device according to a fourth embodiment of the present invention.

[0039] Fig.14 This is a flowchart for explaining the operation of the image monitoring device according to the fourth embodiment of the present invention.

[0040] Fig.15 This is a block diagram showing a lighting control system including an image monitoring device according to an embodiment of the present invention.

[0041] Fig.16 This is a block diagram showing an air-conditioning control system including the image monitoring device according to the embodiment of the present invention.

[0042] [Explanation of Symbols]

[0043] 1. 1a, 1b, 1c: Image monitoring device

[0044] 10: Photography Department

[0045] 20, 20a: Image processing device

[0046] 30, 30a: Storage device

[0047] 40: Output section

[0048] 201: Detection Image Production Department

[0049] 202a, 202b: Analysis Department

[0050] 203: Detection object exclusion department

[0051] 204: Judgment Department

[0052] 205: Entry and exit monitoring department

[0053] 206: Detection object recovery unit

[0054] 61: Change area

[0055] 62: Entrance and exit area

[0056] 63: Indoor area

[0057] 7: Lighting control system

[0058] 70: Lighting Control Department

[0059] 71: Lighting equipment

[0060] 8: Air conditioning control system

[0061] 80: Air conditioning control department

[0062] 81: Air conditioning equipment DETAILED DESCRIPTION

[0063] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0064] [First embodiment]

[0065] The image monitoring device of the first embodiment of the present invention takes a constant temperature moving object in a monitoring target space as a detection target, sequentially acquires thermal images obtained by photographing the monitoring target space, and performs image processing on the thermal images to determine whether there is a constant temperature moving object in the monitoring target space. Figure 1 As shown, the image monitoring device 1 of this embodiment includes an imaging unit 10 , an image processing device 20 , a storage device 30 , an output unit 40 that outputs a determination result obtained by the image processing device 20 , and an input / output device 50 .

[0066] Here, the imaging unit 10 is an imaging device that captures an image of a monitoring target space and generates a thermal image showing the surface temperature distribution in the monitoring target space. As an imaging device constituting the imaging unit 10, for example, an infrared image sensor or an infrared sensor array can be used.

[0067] The image processing device 20 is a device that determines whether there is a constant-temperature moving object in the monitoring target space based on the thermal images sequentially acquired from the imaging unit 10. The specific structure and operation of the image processing device 20 will be described later.

[0068] The storage device 30 is a device that can store various information, and may include a main storage device, a hard disk drive (HDD) or a solid state drive (SSD) and other auxiliary storage devices. In this embodiment, the storage device 30 includes: a reference image storage unit 301, a detection image storage unit 302, a tracking information storage unit 303 and an area information storage unit 304. Among them, the reference image storage unit 301 stores a reference image representing the reference surface temperature distribution of the monitored object space, that is, an image representing the surface temperature distribution of the monitored object space in a state where there is no constant temperature moving object such as a person. The detection image storage unit 302 stores the detection images produced by the detection image production unit 201 described later in a time series. The tracking information storage unit 303 stores information related to the change area as the tracking object in multiple detection images produced in a time series, such as an identification (ID) used to determine the change area as the tracking object and the coordinates of the change area. The area information storage unit 304 stores information for dividing the monitored object space into multiple areas. Furthermore, the storage device 30 may be provided with a setting value storage unit 305 for storing various setting values ​​for operating the image processing device 20 or a program storage unit 306 for storing programs.

[0069] The output unit 40 is a display device that displays the determination result obtained by the image processing device 20 .

[0070] Next, refer to Figures 1 to 6 The image processing device 20 will be described.

[0071] like Figure 1 As shown, the image processing device 20 includes a detection image creating unit 201 , an analyzing unit 202 a , a detection target excluding unit 203 , a determining unit 204 , and a room entry and exit monitoring unit 205 .

[0072] The detection image production unit 201 is configured to compare the thermal image Hn obtained from the shooting unit 10 and the reference image Hs read from the reference image storage unit 301 of the storage device 30, extract the change area with a temperature difference greater than a specified value, and thus produce a detection image Dn containing the extracted change area as a detection object in the monitored object space. As described above, the reference image Hs is a thermal image representing the reference surface temperature distribution of the monitored object space, and is a thermal image representing the surface temperature distribution of the monitored object space when there is no constant temperature moving object. As a process for producing the detection image Dn, for example, the difference in brightness of each pixel of the thermal image Hn and the reference image Hs is taken, and pixels whose difference exceeds a specified threshold are extracted. Through such a process, if an image of a constant temperature moving object or the like is included in the thermal image Hn, the image is extracted as a change area having a temperature higher than that of the reference image, thereby producing a detection image Dn containing the change area as a detection object.

[0073] Whenever multiple thermal images Hn-2, Hn-1, Hn are acquired from the shooting unit 10 in a time series, the detection image production unit 201 produces detection image Dn-2, detection image Dn-1, detection image Dn, and stores these images in the detection image storage unit 302 in a time series, and inputs them into the analysis unit 202a at the same time.

[0074] An example of the detection image Dn is shown in Figure 2 The detection image Dn includes a boundary 60 of the monitored space. If there is an object with a temperature above a certain level in the monitored space, the portion is displayed as a change region 61. Therefore, if there is a person in the monitored space, the person is displayed as a change region in the detection image. Even if a chair is heated by the body temperature of a person, it is displayed as a change region as long as the temperature is above the specified temperature after the person leaves the seat.

[0075] In this embodiment, if Figure 2 As shown, the monitored space is divided into an entrance area 62 and an indoor area 63. Here, the entrance area 62 is, for example, an area near the entrance to the floor as the monitored space, and is an area in the monitored space where people may enter and exit. The indoor area 63 is an area after the entrance area 62 is removed from the detection image. The range of the entrance area 62 and the indoor area 63 can be set via the input and output device 50, and the information related to the set area can be stored in the area information storage unit 304 of the storage device 30. In addition, the method of image detection based on such an area will be described later.

[0076] The analysis unit 202a is configured to analyze the multiple detection images produced in time series by the detection image production unit 201, and track the change area included in the multiple detection images over time. More specifically, each time the detection image Dn produced by the detection image production unit 201 is input, the analysis unit 202a detects the change area included in the detection image Dn, adds identification information to each change area, and sequentially stores its coordinates An in the tracking information storage unit 303 of the storage device 30.

[0077] In addition, the analysis unit 202a reads out the detection image Dn-1 produced not long ago, the identification information of the changed area contained in the detection image Dn-1, and the coordinates An-1 from the detection image storage unit 302 and the tracking information storage unit 303, respectively, and compares the detection image Dn produced this time with the detection image Dn-1 produced not long ago. It determines whether the changed area can be tracked based on whether the changed area exists in both the detection image Dn produced this time and the detection image Dn-1 produced last time, or only exists in any one of the two detection images Dn and Dn-1 produced before and after.

[0078] For example, when two detection images Dn and Dn-1 are compared and the change areas detected by the respective detection images overlap at least in part, or when the centroids of the respective change areas are calculated and the distance between the two centroids is less than a predetermined value, it is assumed that these change areas are change areas that exist in common in the detection image Dn produced this time and the detection image Dn-1 produced last time, and it is determined that the movement of the change areas in the monitored object space, that is, the change of the position of the change areas over time, can be tracked. It can be considered that the change areas determined to be trackable in this way show a constant temperature moving object such as a person.

[0079] On the other hand, a change region that exists only in one of the two detection images Dn and Dn-1 produced successively is determined as follows.

[0080] First, the changed area included in the detection image Dn-1 generated last time but not included in the detection image Dn generated this time is considered to have disappeared from the monitoring target space, and is determined to be impossible to track, thereby terminating the tracking.

[0081] In contrast, a change area that is not included in the detection image Dn-1 produced last time but is included in the detection image Dn produced this time, that is, a change area newly detected in the monitored object space, is in principle determined to be trackable and is subsequently tracked.

[0082] The information on whether or not the changed region can be tracked obtained in this way is associated with the identification information and coordinates An of the changed region and stored in the tracking information storage unit 303 .

[0083] The detection target exclusion unit 203 is configured to exclude the change region newly detected in the monitoring target space, that is, the indoor region 63 (see Figure 2 ) are excluded from the detection object.

[0084] As described above, in this embodiment, if Figure 2 As shown, the monitored space is divided into an entrance and exit area 62 and an indoor area 63. In general, the indoor area 63 is an area that can only be accessed through the entrance and exit area 62. Such an indoor area 63 can be said to be an area where people who have not been photographed in the temperature image so far will basically not suddenly appear. Therefore, the newly detected change area in the indoor area 63 in the monitored space may be daily utensils such as chairs that have been heated by the body temperature of people who have previously stayed in the monitored space, rather than people entering the monitored space, so the newly detected change area in the monitored space, that is, the area detected in the indoor area 63, is excluded from the detection object.

[0085] On the other hand, the entrance and exit area 62 is the remaining area after removing the indoor area 63 from the monitored space, and is the area near the entrance and exit leading to the site as the monitored space, and is the area in the monitored space where people may enter and exit. Therefore, it can be considered that in the entrance and exit area 62, when a person who has not been photographed in the thermal image so far intrudes into the monitored space, the person appears as a new change area in the detection image, and when a person who has been photographed in the thermal image so far leaves the monitored space, the change area disappears from the detection image. Therefore, it is appropriate to use the change area that newly appears in the entrance and exit area 62 as the detection object.

[0086] In addition, the indoor area 63 may also be limited to a local area where objects that are not detection targets but have surface temperatures similar to those of the detection targets frequently appear.

[0087] Figure 3 This diagram is used to explain the appearance or disappearance of a changed area, and summarizes the position and meaning of the changed area in the monitored object space that only exists in one of the two detection images Dn and Dn-1 produced before and after.

[0088] For a changed area included in the detection image Dn-1 produced last time (“present”) but not included in the detection image Dn produced this time (“not present”), as long as the changed area in the detection image Dn-1 produced last time is within the entrance and exit area 62, it can be considered that the person has “left”. On the other hand, if the changed area included in the detection image Dn-1 produced last time (“present”) is within the indoor area 63 and is not included in the detection image Dn produced this time (“not present”), it has to be understood as “suddenly disappeared”. In this case, it is suspected that the changed area has been missed (“lost”) in the detection image Dn produced this time.

[0089] In contrast, for a change area that is not included in the detection image Dn-1 produced last time (“absent”) but is included in the detection image Dn produced this time (“present”), if its position is within the entrance and exit area 62 of the monitored space (if the coordinates of the change area in the detection image Dn produced this time are within the entrance and exit area 62), it can be assumed that a person who has not been captured in the thermal image so far suddenly appears, and it is considered that the person “enters”. On the other hand, if the position of the newly detected change area is within the indoor area 63 such as the center of the venue, it has to be understood as “sudden appearance”. In this case, since the indoor area is a place where it is difficult for a person to suddenly appear, there is a high possibility that an object other than a person is mistakenly detected as a person in the detection image Dn produced this time (producing a “ghost”).

[0090] For example, when a new change area appears in the indoor area of ​​the monitored space, it can be considered that the change area does not represent a person, but a non-constant temperature moving object such as a chair heated by the human body temperature and recognized as a change area because the person leaves the seat.

[0091] Therefore, in the image monitoring device 1 of the present embodiment, for a newly detected change area, the detection object exclusion unit 203 confirms the coordinates of the change area, and when the position of the change area is within the entrance and exit area 62 of the monitored object space, the change area is taken as a detection object, that is, it is judged as a person. On the other hand, when the position of the change area is within the indoor area 63, it is judged that the change area is not a person.

[0092] It is only necessary that the information on the region of the monitoring target space in the detection image is stored in the region information storage unit 304 of the storage device 30 and the detection target exclusion unit 203 refers to or reads the information.

[0093] The determination unit 204 is configured to determine the presence of a constant-temperature mobile object in the monitoring target space based on the change regions included in the detection image Dn other than the change regions excluded from the detection target by the detection target exclusion unit 203. More specifically, in the present embodiment, the determination unit 204 is configured to count the number of change regions included in the detection image Dn minus the number of change regions excluded from the detection target by the detection target exclusion unit 203, that is, the number of change regions determined to be people, and determine that the constant-temperature mobile objects of the number exist in the monitoring target space.

[0094] The entry and exit monitoring unit 205 is configured to monitor changes in the number of people staying in the monitoring target space based on the number of constant-temperature mobile objects, that is, people, in the monitoring target space determined by the determination unit 204 .

[0095] The image processing device 20 as described above can be realized, for example, by installing an image monitoring program in a computer system such as a computer system. Figure 4 The computer shown includes a central processing unit (CPU), a memory, and an input and output interface.

[0096] Next, refer to Figure 2 , Figures 5 to 7 The operation of the image monitoring device 1 of the present embodiment is described. In addition, in the image monitoring device 1 of the present embodiment, the thermal image Hn-2, the thermal image Hn-1, the thermal image Hn, ... are inputted into the image processing device 20 in a time series from the shooting unit 10, and the detection image Dn-2 and the detection image Dn-1 are produced in a time series based on the thermal image Hn-2 and the thermal image Hn-1.

[0097] First, when the thermal image Hn is acquired from the imaging unit 10 ( Figure 5 In S10), the image processing device 20 compares the thermal image Hn with the reference image Hs pre-stored in the reference image storage unit 301 of the storage device 30 in the detection image production unit 201, thereby producing a detection image Dn ( Figure 5 The produced detection image Dn is input to the analysis unit 202a and stored in the detection image storage unit 302 of the storage device 30.

[0098] When the analysis unit 202a receives the detection image Dn produced in time series, it detects the change area included in the detection image Dn, determines the coordinates An for each change area, and stores them in the tracking information storage unit 303 of the storage device 30 ( Figure 5In addition, the coordinates An-1 of the change area of ​​the detection image Dn-1 immediately before are read from the tracking information storage unit 303 of the storage device 30, and compared with the coordinates An of the change area of ​​the detection image Dn ( Figure 5 S13 of the invention) to determine whether the tracking is possible ( Figure 5 S14).

[0099] The changed area included in the detection image Dn-1 not long ago but not included in the detection image Dn this time is determined to be untrackable ( Figure 5 S14: N), excluding this change area from the detection object of whether it is a human ( Figure 5 S17a).

[0100] On the other hand, for the change region ( Figure 5 S14: Y), confirm whether the changed area is a newly detected changed area in the detection image Dn ( Figure 5 S15).

[0101] If the changed region is a region tracked from a changed region included in the detection image Dn-1 not long ago, that is, if the changed region is not a changed region newly detected in the detection image Dn ( Figure 5 S15: N), the determination unit 204 determines that the changed area is a person ( Figure 5 S17b).

[0102] On the other hand, in the case where the changed region is a changed region newly detected in the detection image Dn ( Figure 5 S15: Y), confirm the position of the changed area ( Figure 5 S16).

[0103] In the case where the newly detected changed area in the detection image Dn is detected in the indoor area 63 ( Figure 5 S16:N), excluding the changed area from the detection object of whether it is a person ( Figure 5 In the case where the newly detected change area in the detection image Dn is detected in the entrance / exit area 62 ( Figure 5 S16: Y), the changed area is judged as a person ( Figure 5 S17b).

[0104] The determination result is counted by the determination unit 204 ( Figure 5The statistical result is the number of changed areas included in the detected image Dn minus the number of changed areas excluded from the detected object by the detected object exclusion unit 203, that is, the number of changed areas judged to be people ( Figure 5 The statistical result represents the number of constant temperature moving objects existing in the monitored space and is output to the output unit 40 ( Figure 5 S19).

[0105] Then, if there is an input of the next thermal image from the imaging unit 10 ( Figure 5 If S20: Y), the process from S10 is repeated. If there is no input of the next thermal image within the specified time ( Figure 5 S20:N), ending a series of processing.

[0106] An example of a detection image when a person has entered the monitored space is shown in FIG. Figure 6 . Figure 6 The figure shows the case where the changed area detected in the entrance area 62 at time N is tracked at time N+1. A new changed area 61 appears in the detection image Dn at time N, and when the site is the entrance area 62, the changed area 61 is determined to be a person. By taking this new changed area 61 as the tracking target, even if it moves to the indoor area 6, it will be determined to be a person as in the detection image Dn+1.

[0107] On the other hand, an example in which a person appears suddenly in the indoor area 63 is shown in FIG. Figure 7 . Figure 7 It is shown that the change area that is not shown in the detection image Dn-1 at time N-1 and the detection image Dn at time N+1 appears in the indoor area 63 in the detection image Dn+1 at time N+1. For example, it can be considered that the chair placed in the indoor area 63 is detected as a change area after a person sitting on the chair leaves the seat. In this case, when the change area is detected for the first time at time N+1, although it seems that the person suddenly appears, it is located in the indoor area 63, which is a range that can only be entered through the entrance and exit area 62, so this change area is excluded from the detection object and is determined to be not a person.

[0108] According to the image monitoring device 1 of the present embodiment, when a changed area is newly detected, it is confirmed whether the location of the changed area is the entrance and exit area of ​​the monitored space or the indoor area. If it is the indoor area, the changed area is excluded from the detection object. Therefore, it is possible to prevent a chair heated by a person's body temperature from being mistakenly detected as a person even after the person leaves the seat in the monitored space.

[0109] [Second embodiment]

[0110] In the first embodiment, the detection image Dn-1 produced not long ago and the identification information and coordinates An-1 of the changed area contained in the detection image Dn-1 are read out, and the detection image Dn produced this time is compared with the detection image Dn-1 produced not long ago to determine whether a certain changed area can be tracked.

[0111] In contrast, the image monitoring device 1a of the second embodiment of the present invention determines whether the changed region can be tracked using only the detected image Dn created this time, the identification information of the changed region, and the coordinates An-1. The image monitoring device 1a of the second embodiment differs from the image monitoring device 1 of the first embodiment in that it does not include the detected image storage unit 302.

[0112] The structure of the image monitoring device 1a according to the second embodiment of the present invention is shown in FIG. Figure 8 . In addition, the same reference numerals are used for the components common to the image monitoring device 1 of the first embodiment, and detailed description thereof will be omitted. In addition, the operation of the image monitoring device 1a of this embodiment is shown in FIG. Fig.10 Flow chart of Fig.10 The flowchart in the embodiment 1 and the action flowchart of the embodiment 1 Figure 5 The same, therefore, detailed description is omitted.

[0113] like Figure 8 As shown in FIG. 1 , the image monitoring device 1a of the present embodiment includes: a photographing unit 10, an image processing device 20a, a storage device 30a, and an output unit 40 that outputs the determination result obtained by the image processing device 20a. Here, the image processing device 20a includes: a detection image generating unit 201, an analyzing unit 202b, a detection object excluding unit 203, a determining unit 204, and a room entry and exit monitoring unit 205. In addition, the storage device 30a includes: a reference image storage unit 301, a tracking information storage unit 303, and a region information storage unit 304.

[0114] Whenever a plurality of thermal images Hn-2, Hn-1, Hn are acquired from the imaging unit 10 in time series ( Fig.10 The detection image production unit 201 produces detection images Dn-2, Dn-1, and Dn-2, and inputs them to the analysis unit 202b ( Fig.10 of S11).

[0115] The analyzing unit 202b reads the identification information and coordinates An-1 ( Fig.10 S12), compare the detection image Dn produced this time with the identification information and coordinates An-1 of the changed area produced not long ago ( Fig.10S13), whether a certain change area can be tracked is determined based on whether it exists in both the detection image Dn produced this time and the identification information and coordinates An-1 of the change area produced not long ago, or only exists in the detection image Dn, the identification information of the change area, and the coordinates An-1 ( Fig.10 S14).

[0116] For example, when the detection image Dn, the identification information of the change area, and the coordinates An-1 are compared and the change area detected in the detection image Dn, the identification information of the change area, and the coordinates An-1 overlap at least in part, or when the center of gravity of the change area of ​​the detection image Dn is calculated and the distance between the center of gravity and the coordinates of the identification information of the change area and the coordinates An-1 is less than a specified value, the identification information of the change area and the coordinates of the coordinates An-1 produced not long ago are assumed to be the change area existing in the detection image Dn produced this time, and it is determined that the movement of the change area in the monitored object space, that is, the change of the position of the change area over time, can be tracked. It can be considered that the change area determined to be trackable in this way shows a constant temperature moving object such as a person.

[0117] On the other hand, for a change region that exists only in one of the detection image Dn, the identification information of the change region, and the coordinates An-1, determination is performed as follows.

[0118] First, the changed area that was included in the identification information and coordinates An-1 of the changed area being tracked until the last time but is not included in the detection image Dn generated this time is considered to have disappeared from the monitored space, and the tracking is terminated by determining that it cannot be tracked.

[0119] On the other hand, the change area that was not included in the identification information and coordinates An-1 of the change area that was the tracking target until the last time but is included in the detection image Dn generated this time, that is, the change area newly detected in the monitoring target space, is determined to be trackable in principle and is used as the tracking target thereafter ( Fig.10 S15).

[0120] The information on whether or not the changed region can be tracked obtained in this way is associated with the identification information and coordinates An of the changed region and stored in the tracking information storage unit 303 .

[0121] Fig. 9 This diagram is used to explain the interpretation of the new appearance or disappearance of a change area, and summarizes the position and meaning in the monitored object space of a change area that only exists in the detection image Dn, the identification information of the change area, and the coordinates An-1.

[0122] For a change area that is included in the identification information and coordinates An-1 of the last change area (“present”) but not included in the detection image Dn produced this time (“not present”), if the coordinates in the identification information and coordinates An-1 of the last change area are within the entrance and exit area 62, it can be considered that the person has “left”. On the other hand, if the change area included in the identification information and coordinates An-1 of the last change area (“present”) is within the indoor area 63 but not included in the detection image Dn produced this time (“not present”), it has to be understood as “suddenly disappeared”. In this case, it is suspected that the change area has been missed (“lost”) in the detection image Dn produced this time.

[0123] On the other hand, for a change area that is not included in the identification information and coordinates An-1 of the last change area (“not present”) but is included in the detection image Dn produced this time (“present”), if its position is within the entrance and exit area 62 of the monitored space (the coordinates of the change area in the detection image Dn produced this time are within the entrance and exit area 62), it can be assumed that a person who has not been captured in the thermal image so far suddenly appears, and it is considered that the person “enters” ( Fig.10 On the other hand, if the position of the newly detected change area is in the indoor area 63 such as the center of the venue, it has to be understood as "sudden appearance". In this case, since the indoor area is a place where it is difficult for people to suddenly appear, there is a high possibility that objects other than people are mistakenly detected as people in the detection image Dn produced this time (generating "ghost images") ( Fig.10 S16: N).

[0124] For example, when a new change area appears in the indoor area of ​​the monitored space, it can be considered that the change area does not represent a person, but a non-constant temperature moving object such as a chair heated by the human body temperature and recognized as a change area because the person leaves the seat.

[0125] Therefore, in the image monitoring device 1a of the present embodiment, for a newly detected change area, the detection object exclusion unit 203 confirms the coordinates of the change area, and when the position of the change area is within the entrance and exit area 62 of the monitoring object space, the change area is regarded as a detection object, that is, it is judged as a person ( Fig.10 On the other hand, when the position of the changed area is within the indoor area 63, it is determined that the changed area is not a human area ( Fig.10 S17a).

[0126] [Third Embodiment]

[0127] In the first embodiment, when a changed region newly detected in a detection image generated in time series is detected in the indoor area 63 of the monitoring target space, it is determined that the changed region is not a person and is excluded from the detection target.

[0128] In contrast, the image monitoring device 1b of the third embodiment of the present invention is similar to the image monitoring device 1 of the first embodiment in that a newly detected change area within the indoor area 63 of the monitored object space is excluded from the detection object, and is different from the image monitoring device 1 of the first embodiment in that when the change area excluded from the detection object is tracked and moves in the detection image within a specified period, the change area is treated as a detection object.

[0129] The structure of the image monitoring device 1b according to the third embodiment of the present invention is shown in FIG. Fig.11 In addition, the same reference numerals are used for the components common to the image monitoring device 1 of the first embodiment, and detailed description thereof will be omitted. Fig.12 The operation of the image monitoring device 1b of this embodiment is described with reference to the flowchart.

[0130] like Fig.11 As shown, the image monitoring device 1b of this embodiment includes: a photographing unit 10, an image processing device 20b, a storage device 30, and an output unit 40 that outputs the determination result obtained by the image processing device 20b. Here, the image processing device 20b includes: a detection image production unit 201, an analysis unit 202a, a detection object exclusion unit 203, a detection object restoration unit 206, a determination unit 204, and a room entry and exit monitoring unit 205.

[0131] The detection target restoration unit 206 that features the image monitoring device 1 b of the third embodiment is configured to track a changed region excluded from the detection target by the detection target exclusion unit 203 and, when the changed region moves within a predetermined period, treat the changed region as a detection target.

[0132] like Fig.12 As shown, the image monitoring device 1b of the third embodiment is the same as the image monitoring device 1 of the first embodiment, and performs the processes from S10 to S16, and tracks the changed area excluded from the detection object by the detection object exclusion unit 203 ( Fig.12 S16a) and the change area moves within the prescribed period ( Fig.12 S16a: Y), the changed area is restored to the detection object and judged as a person ( Fig.12 If the vehicle is not moved within the prescribed period ( Fig.12S16a: N), it is determined that the changed area is not a person and is excluded from the detection object ( Fig.12 S17b).

[0133] If it is applied to Figure 7 For example, when a change region that does not appear in the detection images Dn-1 and Dn at time N-1 and time N appears in the detection image Dn+1 at time N+1 within the indoor area 63, it is once determined that the change region is not a person and is excluded from the detection object ( Fig.12 However, if the change region is tracked within a certain period of time and features such as movement of the change region are observed ( Fig.12 S16a: Y), then it is judged as a human ( Fig.12 S17b).

[0134] Thus, according to the image monitoring device 1b of the third embodiment, even when no changed area is detected in the detection image just before, the detection object recovery unit 206 can be used as a detection object, thereby accurately grasping the number of constant temperature moving objects existing in the monitoring object space.

[0135] [Fourth Embodiment]

[0136] In the third embodiment, the detection image Dn-1 produced not long ago, the identification information of the changed area contained in the detection image Dn-1 and the coordinates An-1 are read out, the detection image Dn produced this time and the detection image Dn-1 produced not long ago are compared, and it is determined whether a certain changed area can be tracked.

[0137] In contrast, the image monitoring device 1c of the fourth embodiment of the present invention uses only the detection image Dn produced this time, the identification information of the change area, and the coordinates An-1 to determine whether the change area can be tracked. The image monitoring device 1a related to the second embodiment is different from the image monitoring device 1a related to the second embodiment in that it includes the detection object recovery unit 206, and is different from the image monitoring device 1b of the third embodiment in that it does not have the detection image storage unit 302.

[0138] The structure of the image monitoring device 1c according to the fourth embodiment of the present invention is shown in FIG. Fig.13 . In addition, regarding the components common to the image monitoring device 1b of the third embodiment, the same symbols are marked and detailed descriptions are omitted. In addition, regarding the components not common to the image monitoring device 1b of the third embodiment, since the differences are the same as those of the image monitoring device 1 of the first embodiment and the image monitoring device 1a of the second embodiment, the same symbols are marked and detailed descriptions are omitted. In addition, the operation of the image monitoring device 1c of this embodiment is shown in Fig.14Flow chart of Fig.14 Flowchart of the embodiment 3 and the action flow chart of the embodiment 3 Fig.12 The same, so the detailed description is omitted.

[0139] like Fig.13 As shown, the image monitoring device 1c of this embodiment includes: a photographing unit 10, an image processing device 20c, a storage device 30a, and an output unit 40 that outputs the determination result obtained by the image processing device 20c. Here, the image processing device 20c includes: a detection image production unit 201, an analysis unit 202b, a detection object exclusion unit 203, a detection object restoration unit 206, a determination unit 204, and a room entry and exit monitoring unit 205.

[0140] According to the image monitoring device 1c of this embodiment, similarly to the second embodiment, there is no need to provide a detection image storage unit for storing detection images in the storage device 30, thereby reducing the storage capacity for storing images. Similar to the third embodiment, even if a changed area cannot be detected in a detection image just recently, by including a detection object recovery unit 206, it can be used as a detection object, thereby achieving the advantage of being able to accurately grasp the number of constant temperature moving objects existing in the monitored object space.

[0141] <Application Examples>

[0142] The image monitoring device according to one embodiment of the present invention can be applied to various control systems having different control objects.

[0143] For example, Fig.15 As shown, a lighting control system 7 can be formed by combining an image monitoring device 1, an image monitoring device 1a, an image monitoring device 1b, an image monitoring device 1c and a lighting control unit 70 according to an embodiment of the present invention. The lighting control unit 70 controls the lighting / extinguishing state of the lighting equipment 71 in the monitored space based on the number of constant temperature moving objects determined by the determination unit 204 to exist in the monitored space and the positions of the constant temperature moving objects.

[0144] In addition, if Fig.16 As shown, an air conditioning control system 8 can be formed by combining an image monitoring device 1, an image monitoring device 1a, an image monitoring device 1b, an image monitoring device 1c and an air conditioning control unit 80 according to an embodiment of the present invention. The air conditioning control unit 80 controls the air conditioning equipment 81 in the monitored space based on the number of constant temperature moving objects determined by the determination unit 204 to exist in the monitored space and the positions of the constant temperature moving objects.

[0145] At this time, if the image monitoring device 1, image monitoring device 1a, image monitoring device 1b, and image monitoring device 1c are constructed in a manner that divides the monitored space into multiple partitions and grasps the number of constant temperature moving objects existing in each partition, it is possible to perform detailed lighting control or air conditioning control corresponding to the presence or absence of people in each partition.

[0146] Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. Various changes that can be understood by those skilled in the art within the scope of the present invention can be made to the structure and details of the present invention.

[0147] [Industrial Applicability]

[0148] The invention can be used in image monitoring devices and control systems.

Claims

1. An image monitoring device, comprising: a detection image producing unit configured to compare a thermal image representing a surface temperature distribution of a monitoring target space with a reference image representing a surface temperature distribution serving as a reference of the monitoring target space, extract a change region having a temperature difference greater than a predetermined value, and produce a detection image including the extracted change region as a detection target in the monitoring target space; an analyzing unit configured to analyze the plurality of detection images produced in time series by the detection image producing unit and track the change region over time; a detection object exclusion unit configured to exclude the change area from the detection object when the position of the change area extracted for the first time in the plurality of detection images is within the first area of ​​the monitoring object space, and to treat the change area as the detection object when the position of the change area extracted for the first time in the plurality of detection images is within the second area of ​​the monitoring object space; as well as The determination unit is configured to determine the presence of the constant-temperature moving object in the monitoring target space based on the change regions other than the change regions excluded from the detection target by the detection target exclusion unit among the change regions included in the detection image.

2. The image monitoring device according to claim 1, further comprising a detection object recovery unit, The detection target restoration unit is configured to set the changed region as a detection target when the changed region excluded from the detection target by the detection target exclusion unit moves with time in the detection image within a predetermined period.

3. The image monitoring device according to claim 1 or 2, wherein: The first area is an indoor area of ​​the monitored space, and the second area is an entrance and exit area of ​​the monitored space.

4. The image monitoring device according to claim 3, further comprising an area setting unit, The area setting unit sets the range of an indoor area and an entrance and exit area for the monitored space.

5. The image monitoring device according to claim 1 or 2, wherein: The first area is a local area where objects that are not detection targets but have surface temperatures similar to those of the detection targets frequently appear.

6. The image monitoring device according to claim 1 or 2, wherein: The determination unit is configured to determine that constant-temperature moving objects exist in the monitoring target space as a number obtained by subtracting a number of change regions excluded from the detection targets by the detection target exclusion unit from a number of change regions included in the detection image.

7. The image monitoring device according to claim 1 or 2, further comprising an entry and exit monitoring unit, The entry-exit monitoring unit monitors a change in the number of the constant-temperature moving objects in the monitoring target space based on the number of the constant-temperature moving objects in the monitoring target space determined by the determination unit.

8. An image monitoring method, comprising: A photographing step of photographing the monitored space and sequentially generating thermal images representing the surface temperature distribution in the monitored space; A detection image preparation step of comparing each of the plurality of thermal images generated sequentially with a reference image representing a surface temperature distribution as a reference of the monitoring target space, extracting a change region having a temperature difference greater than a predetermined value, thereby preparing a detection image including the extracted change region as a detection target in the monitoring target space; An analysis step of analyzing a plurality of detection images produced in time series and tracking the change areas contained in the plurality of detection images over time; a detection object exclusion step of excluding the change area from the detection object when the position of the change area first extracted in the plurality of detection images produced in time series is within the first area of ​​the monitoring object space, and treating the change area as the detection object when the position of the change area first extracted in the plurality of detection images is within the second area of ​​the monitoring object space; as well as The determination step determines the presence of the constant-temperature moving object in the monitoring target space based on the change region obtained by excluding the change region excluded from the detection target from the change region included in the detection image.

9. A control system comprising: The image monitoring device according to claim 1 or 2; and The lighting control unit controls the lighting / extinguishing state of the lighting device in the monitoring target space based on the number of the constant-temperature moving objects determined by the determination unit to be present in the monitoring target space and the positions of the constant-temperature moving objects.

10. A control system comprising: The image monitoring device according to claim 1 or 2; and The air conditioning control unit controls the air conditioning capacity in the monitoring target space based on the number of the constant temperature moving objects determined by the determination unit to be present in the monitoring target space and the positions of the constant temperature moving objects.

Images