Detection system, detection method, and program

Abstract

To provide a detection system, a detection method, and a program, which can efficiently detect an object to be measured.SOLUTION: A detection system 100 includes a first infrared camera, a first calibration processing section, an equalization processing section 172, an edge image acquisition section 174, and an object detection section 153. The first infrared camera captures a first thermal image while keeping a region to be measured within an angle of view. The first calibration processing section calibrates an output value of each pixel of the first thermal image to calculate a first offset calibration value. The first infrared camera captures a second thermal image at a time point when an arbitrary period has elapsed after capturing the first thermal image while keeping the region to be measured within the angle of view.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a detection system, a detection method, and a program, and particularly to a detection system, a detection method, and a program for detecting an object using an infrared camera.

Background Art

[0002] The detection system disclosed in Patent Document 1 extracts a directional edge having a predetermined direction from a thermal image or the like, and then discriminates a vehicle to be measured based on the connection relationship between the directional edge and an object candidate region.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The inventors of the present application have discovered the following problems. In order to detect whether an object is moving in such a detection system, after recording all-screen data showing a thermal image one frame before, the recorded thermal image and all-screen data showing the thermal image of the current frame are compared to detect the measurement target. Therefore, it is necessary to store all-screen data of one frame, and there is room for consideration regarding the efficiency of detecting the measurement target.

[0005] In view of the above problems, an object of the present invention is to provide a detection system, a detection method, and a program capable of improving the efficiency of detecting a measurement target.

Means for Solving the Problems

[0006] A detection system according to an aspect of the present embodiment is a first infrared camera, and The first calibration processing unit, Homogenization processing unit, Edge image acquisition unit, It comprises an object detection unit, The first infrared camera captures a first thermal image while keeping the measurement target area within its field of view. The first calibration processing unit calculates a first offset calibration value by performing calibration on the output value of each pixel of the first thermal image. After the first infrared camera captures the first thermal image while keeping the measurement target area within its field of view, it captures a second thermal image at a time after a specified period of time has elapsed. The homogenization processing unit corrects the output value of each pixel of the second thermal image using the first offset calibration value, thereby outputting a motion detection image. The edge image acquisition unit performs image processing on the motion detection image and outputs an edge image. The object detection unit determines whether the motion detection image is non-uniform based on the edge image, and if it determines that the motion detection image is non-uniform, it detects the presence of a moving object in the inner region of the edge image, which is a part where the brightness difference is large compared to other parts.

[0007] A detection method according to one aspect of this embodiment is: The first step is to capture a thermal image while keeping the measurement target area within the field of view, The steps include: calculating a first offset calibration value by performing calibration on the output value of each pixel of the first thermal image; The steps include taking a first thermal image while keeping the measurement target area within the field of view, and then taking a second thermal image at a time after a specified period of time has elapsed, The steps include: outputting a motion detection image by correcting the output value of each pixel of the second thermal image using the first offset calibration value; The steps include: applying image processing to the motion detection image and outputting an edge image; The method includes the step of determining whether the motion detection image is non-uniform based on the edge image, and if it is determined that the motion detection image is non-uniform, detecting the presence of a moving object in the inner region, which is a part of the edge image where the brightness difference is large compared to other parts.

[0008] A program according to one aspect of this embodiment is: The computer, which functions as the control unit for the detection system, The first step is to capture a thermal image while keeping the measurement target area within the field of view, The steps include: calculating a first offset calibration value by performing calibration on the output value of each pixel of the first thermal image; The steps include taking a first thermal image while keeping the measurement target area within the field of view, and then taking a second thermal image at a time after a specified period of time has elapsed, The steps include: outputting a motion detection image by correcting the output value of each pixel of the second thermal image using the first offset calibration value; The steps include: applying image processing to the motion detection image and outputting an edge image; Based on the edge image, determine whether the motion detection image is non-uniform, and if it is determined that the motion detection image is non-uniform, perform the step of detecting the presence of a moving object in the inner region, which is a part of the edge image where the brightness difference is large compared to other parts. [Effects of the Invention]

[0009] According to this embodiment, the object to be measured can be detected efficiently. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram showing the configuration of the detection system according to Embodiment 1. [Figure 2] This is a diagram showing the configuration of the infrared camera in the detection system according to Embodiment 1. [Figure 3]It is a block diagram of a control device of a detection system according to Embodiment 1. [Figure 4] It is a block diagram of a first image processing unit of a detection system according to Embodiment 1. [Figure 5] It is a block diagram of a second image processing unit of a detection system according to Embodiment 1. [Figure 6] It is a processing block diagram of a detection system according to Embodiment 1. [Figure 7] It is a flowchart of a detection method according to Embodiment 1. [Figure 8] It is a diagram showing a first example of detection by a detection system. [Figure 9] It is a diagram showing a first example of detection by a detection system. [Figure 10] It is a diagram showing a first example of detection by a detection system. [Figure 11] It is a diagram showing a first example of detection by a detection system. [Figure 12] It is a diagram showing an example of a correspondence relationship between pixels. [Figure 13] It is a diagram showing a second example of detection by a detection system. [Figure 14] It is a diagram showing a second example of detection by a detection system. [Figure 15] It is a configuration diagram of a detection system according to Embodiment 2. [Figure 16] It is a configuration diagram of an infrared camera of a detection system according to Embodiment 2. [Figure 17] It is a block diagram of a control device of a detection system according to Embodiment 2. [Figure 18] It is a block diagram of an image processing unit of a detection system according to Embodiment 2. [Figure 19] It is a processing block diagram of a detection system according to Embodiment 2. [Figure 20] It is a flowchart of a detection method according to Embodiment 2. [Figure 21] It is a flowchart of a thermal image generation method of a detection method according to Embodiment 2. [Figure 22] This figure shows an example of the hardware configuration included in the detection system. [Figure 23] This figure shows an example of a thermal image. [Figure 24] This figure shows an example of a motion detection image. [Figure 25] This figure shows an example of an edge image that has undergone pre-denoising processing. [Figure 26] This figure shows an example of an edge image that has not undergone prior noise reduction processing. [Modes for carrying out the invention]

[0011] Specific embodiments of the present invention will be described in detail below with reference to the drawings. However, the present invention is not limited to the following embodiments. Also, for clarity of explanation, the following description and drawings have been simplified as appropriate.

[0012] (Embodiment 1) Embodiment 1 will be described with reference to Figure 1. Figure 1 is a configuration diagram of the detection system according to Embodiment 1. As shown in Figure 1, the detection system 100 includes a control device 11, an infrared camera 10a, and an infrared camera 10b. Although the example of the detection system 100 shown in Figure 1 further includes a display 12, the detection system 100 does not necessarily have to include a display 12.

[0013] The infrared camera 10a is preferably a far-infrared camera. The infrared camera 10a is connected to the control device 11 in a communicative manner, receives predetermined instruction signals from the control device 11, and operates according to the received instruction signals. The infrared camera 10a also supplies the control device 11 with image data (also called thermal image data) related to the image (also called thermal image) captured by the infrared camera 10a. The infrared camera 10a only needs to be installed in a location where it can photograph the measurement target area. The measurement target area is the area where the measurement target is expected to exist. The measurement target is, for example, a moving body having a body surface temperature higher than the ambient temperature. Such moving bodies are, for example, vehicles, animals, or people. Vehicles are, for example, trucks, passenger cars, motorcycles, construction machinery, agricultural machinery, etc. Animals include, for example, homeothermic animals. Homeothermic animals are, for example, mammals and birds. Specific examples of homeothermic animals are pests. Pests are, for example, wild boars, deer, foxes, etc. The installation location of the infrared camera 10a is not particularly limited, but it is often a fixed location outdoors or indoors, and can be a wide variety of places such as roads, plains, pastures, office building districts, elevators, etc.

[0014] The infrared camera 10b has the same configuration as the infrared camera 10a. The infrared camera 10b only needs to be positioned to capture the same measurement target area as the infrared camera 10a, and specifically, it should face the same location and the same shooting direction as the infrared camera 10a. The field of view of the image captured by the infrared camera 10b does not need to be exactly the same as the field of view of the image captured by the infrared camera 10a. In other words, both fields of view should be approximately the same. A slight difference between the two fields of view is acceptable. In the overview of the processing and detection method performed by the detection system 100 described later, one or both of the infrared cameras 10a and 10b may be used as appropriate in each step. For example, the infrared camera 10a may be mainly used to detect an object or a measurement target, and the infrared camera 10b may be mainly used to capture a thermal image.

[0015] The control device 11 is communicatively connected to the infrared camera 10a, the infrared camera 10b, and the display 12. The control device 11 may be connected to the infrared camera 10a, the infrared camera 10b, and the display 12 by wire or by wireless connection. The control device 11 controls the infrared camera 10a and the infrared camera 10b, respectively, and acquires image data generated by the infrared camera 10a and the infrared camera 10b, respectively. The control device 11 may display the acquired image data on the display 12. The detection system 100 may connect multiple displays 12 and display the image data generated by the infrared camera 10a and the infrared camera 10b on multiple displays 12, respectively.

[0016] The display 12 is a display device including, for example, a liquid crystal panel or an organic electroluminescent panel, and is positioned in a location visible to the user. The display 12 displays images captured by at least one of the infrared cameras 10a and 10b via the control device 11. The display 12 may be disconnected from the control device 11 from the time the detection system 100 starts to detect the object to be measured until it finishes, and may be located away from the control device 11.

[0017] With the above configuration, the control device 11 displays the images captured by at least one of the infrared cameras 10a and 10b on the display 12 in a manner that is visible to the user. This allows the control device 11 to enable the user to recognize surrounding objects. If the detection system 100 does not have a display 12, the control device 11 may store the images captured by the infrared cameras 10a and 10b in an external storage device (not shown).

[0018] (Infrared camera) Next, the configuration of the infrared camera 10a will be described with reference to Figure 2. Figure 2 is a diagram of the infrared camera configuration of the detection system shown in Figure 1. The infrared camera 10a shown in Figure 2 mainly comprises a housing 101, an objective lens 102, a mechanical shutter 103, an infrared sensor 104, a camera control circuit 105, and a temperature sensor 106.

[0019] The housing 101 houses the components of the infrared camera 10a. The housing 101 may be fixed in any location. The objective lens 102 receives infrared light incident from the range captured by the infrared camera 10a and projects it onto the infrared sensor 104. The mechanical shutter 103 includes a light-shielding plate and is interposed between the objective lens 102 and the infrared sensor 104 so as to be openable and closable. The light-shielding plate is, for example, black-painted metal or black anodized aluminum. The mechanical shutter 103 may be positioned outside the objective lens 102.

[0020] When the mechanical shutter 103 is closed, it blocks the light incident from the objective lens 102 to the infrared sensor 104. On the other hand, when the mechanical shutter 103 is open, it does not block the light incident from the objective lens 102 to the infrared sensor 104. Furthermore, the mechanical shutter 103 is a thermally uniform blackbody as seen from the infrared sensor 104, and calibration of the infrared sensor 104 is performed with the mechanical shutter 103 closed. Also, a uniformization process is performed with the mechanical shutter 103 open.

[0021] The infrared sensor 104 is composed of thermal elements arranged in an array and receives infrared light incident through the objective lens 102, generating image data based on the change in the resistance value of each thermal element. In other words, the image data generated by the infrared sensor 104 includes the output values ​​of the thermal elements arranged in an array. The infrared sensor 104 is communicatively connected to the camera control circuit 105 and operates by receiving predetermined control signals from the camera control circuit 105. When the infrared sensor 104 generates image data, it supplies the generated image data to the camera control circuit 105.

[0022] Furthermore, the dynamic range of the pixel values ​​set for displaying the sensor output value of the infrared sensor 104 on the display 12 is set to a resolution that allows for suitable recognition of the measurement target. Therefore, even within the temperature range where the above-mentioned abnormal conditions do not occur, the pixel values, which are the detection signals, may saturate.

[0023] The camera control circuit 105 is a control circuit including an MCU (Micro Controller Unit) that controls the mechanical shutter 103 and the infrared sensor 104. The camera control circuit 105 is also communicatively connected to the control device 11 shown in Figure 1, receives control signals from the control device 11, and controls each component of the infrared camera 10a according to the received control signals. The camera control circuit 105 controls the mechanical shutter 103 to remain closed when the infrared camera 10a is not taking a picture. The camera control circuit 105 controls the mechanical shutter 103 to open when the infrared camera 10a is taking a picture. At this time, the camera control circuit 105 also supplies image data, including the sensor output value generated by the infrared sensor 104, to the control device 11.

[0024] The camera control circuit 105 also temporarily closes the mechanical shutter 103 under predetermined conditions. These predetermined conditions include, for example, temporarily closing the mechanical shutter 103 to protect the infrared sensor 104, or performing calibration. In this case, for example, the camera control circuit 105 receives an instruction from the control device 11 to temporarily close the mechanical shutter 103. Furthermore, if the camera control circuit 105 closes the mechanical shutter 103 for calibration, it supplies the control device 11 with thermal image data relating to a thermal image (also called a shutter image) of the mechanical shutter 103 in its closed state. Furthermore, if the camera control circuit 105 opens the mechanical shutter 103 for calibration, it supplies the control device 11 with thermal image data relating to a thermal image (also called a non-shutter image) of the measurement target area captured through the open mechanical shutter 103.

[0025] The camera control circuit 105 is connected to the temperature sensor 106 in a communicative manner and receives measurement data regarding the temperature inside the infrared camera 10a from the temperature sensor 106. The camera control circuit 105 also supplies the measurement data received from the temperature sensor 106 to the control device 11.

[0026] The temperature sensor 106 is installed inside the infrared camera 10a and measures the temperature of the infrared camera 10a, supplying the measurement data to the camera control circuit 105. Preferably, the temperature sensor 106 is installed near the infrared sensor 104. Specifically, the temperature sensor 106 and the infrared sensor 104 are mounted on the same substrate, and the temperature sensor 106 may be, for example, a temperature sensor IC (integrated circuit).

[0027] (Control device) Next, the control device 11 will be described with reference to Figure 3. Figure 3 is a block diagram of the control device according to Embodiment 1. The control device 11 mainly consists of a control IF 120, ROM 130, RAM 140, control circuit 150, image data acquisition unit 160, first image processing unit 170, second image processing unit 180, and image data output unit 190. These components are connected via the bus 110 so as to allow communication as appropriate.

[0028] The control IF120 is a communication line interface for controlling infrared cameras 10a and 10b. The control IF120 supplies control signals to infrared cameras 10a and 10b for the control device 11 to control infrared cameras 10a and 10b.

[0029] ROM130 (Read Only Memory) is a non-volatile memory that stores pre-configured information or data. For example, ROM130 stores programs and data for the control device 11 to implement the functions according to this embodiment. Alternatively, an external recording device may be connected to the detection system 100 in a communicative manner and perform similar operations to ROM130.

[0030] RAM140 (RAM (Random Access Memory)) is a volatile memory having a storage area from which the control device 11 can temporarily expand data. RAM140 may be, for example, DRAM (Dynamic Random Access Memory), or it may include registers associated with the control circuit 150, etc. RAM140 includes an area for expanding and executing programs stored in ROM130. RAM140 can also be used, for example, when processing image data supplied from infrared cameras 10a and 10b.

[0031] The control circuit 150 includes computing devices such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and controls each component of the control device 11. The control circuit 150 also outputs predetermined instructions to the infrared camera 10a, infrared camera 10b, or display 12. In other words, the control circuit 150 receives predetermined signals from each component of the control device 11 and performs predetermined judgments and calculations based on the received signals. Alternatively, the control circuit 150 outputs the results of predetermined judgments and calculations performed based on the signals received from each component to each component. The control circuit 150 includes, as functional blocks, an object detection unit 153, an object temperature measurement unit 154, a measurement target determination unit 155, and an imaging control unit 156.

[0032] The object detection unit 153 determines whether the motion detection image generated by the uniformization processing unit 172 (described later) is non-uniform or not, based on the edge image extracted by the edge image acquisition unit 174 (described later). For example, the object detection unit 153 determines that the motion detection image is non-uniform when edges exist in the edge image extracted by the edge image acquisition unit 174, and determines that it is not non-uniform, i.e., uniform, when no edges exist. Here, the edges in the edge image refer to, for example, contour lines. If the object detection unit 153 determines that the motion detection image is non-uniform, it detects an area in the edge image with a large difference in brightness compared to other parts, for example, an area surrounded by contour lines, as an inner region, and detects that a moving object exists in this inner region. The object has a surface temperature different from the surrounding environment of the infrared camera 10a or infrared camera 10b, is moving, and has the potential to be the object to be measured.

[0033] The object temperature measurement unit 154 determines the absolute temperature of the inner region based on the thermal image captured by the infrared camera 10b. The absolute temperature can be determined using a pre-determined approximation formula. The approximation formula shows the relationship between the output value of the surface temperature of the object to be measured, as captured by the infrared camera, and the absolute temperature. Specifically, the object to be measured is photographed using the infrared camera 10b at multiple ambient temperature levels, and the output value of the surface temperature of the object is determined. This ambient temperature is the temperature measured by the temperature sensor 106 of the infrared camera 10b. The approximation formula is then determined from the output value of the surface temperature of the object and the measured value of the surface temperature of the object. The ROM 130 may store such an approximation formula in advance.

[0034] The measurement target determination unit 155 compares the absolute temperature measured by the object temperature measurement unit 154 with the average surface temperature of the object to be measured, which has been measured in advance. Here, the average surface temperature of the object to be measured refers to the average value of the surface temperature of the object to be measured, or the average temperature range of the surface temperature of the object to be measured. Based on the result of this comparison, the measurement target determination unit 155 determines whether or not the object is a target for measurement. For example, if the difference between the absolute temperature and the average surface temperature is 0 (zero) or less than or equal to a predetermined value, the measurement target determination unit 155 determines that the object is a target for measurement. If the difference exceeds a predetermined value, the measurement target determination unit 155 determines that the object is not a target for measurement. The ROM 130 may store such average surface temperatures of objects to be measured in advance.

[0035] The imaging control unit 156 stores image data related to the thermal image captured by the infrared camera 10a or infrared camera 10b in the ROM 130 or an external recording device. Alternatively, a thermal image video, which is a compilation of multiple such image data, may also be stored in the ROM 130 or an external recording device.

[0036] The image data acquisition unit 160 is an interface for acquiring thermal image data (input thermal image data), which is data relating to a thermal image, from the infrared camera 10a or the infrared camera 10b. The image data acquisition unit 160 periodically acquires image data from the infrared camera 10a, for example. For example, the image data acquisition unit 160 receives one frame of image every 1 / 15th of a second. When the image data acquisition unit 160 receives image data from the infrared camera 10a, it supplies the received image data to the first image processing unit 170. When the image data acquisition unit 160 receives image data from the infrared camera 10b, it supplies the received image data to the second image processing unit 180.

[0037] The first image processing unit 170 and the second image processing unit 180 are image processing circuits, including, for example, a GPU (Graphics Processing Unit). The first image processing unit 170 and the second image processing unit 180 work in conjunction with the RAM 140 to perform predetermined processing on the thermal image data. Details of the first image processing unit 170 and the second image processing unit 180 will be described later.

[0038] The image data output unit 190 is an interface for outputting image data (output thermal image data) processed by at least one of the first image processing unit 170 and the second image processing unit 180 to the display 12 shown in Figure 1. The image data output by the image data output unit 190 is output according to a data format corresponding to the specifications of the display 12. This data format is, for example, HDMI (High-Definition Multimedia Interface) (registered trademark) or DVI (Digital Visual Interface). The image data output unit 190 outputs image data relating to the calibrated thermal image. The image data output unit 190 may be configured not to output image data relating to the shutter image.

[0039] Next, the first image processing unit 170 will be described with reference to Figure 4. Figure 4 is a block diagram of the first image processing unit 170 according to Embodiment 1. The first image processing unit 170 works in conjunction with the RAM 140 and the control circuit 150 to perform predetermined processing. The first image processing unit 170 mainly consists of a defective pixel correction unit 171, a uniformization processing unit 172, a calibration processing unit 173, and an edge image acquisition unit 174.

[0040] The calibration processing unit 173 of the first image processing unit 170 is responsible for calibrating the offset processing, one of the correction processes performed by the uniformization processing unit 172, using a non-shuttered image. The calibration processing unit 173 receives image data of a non-shuttered image captured by the infrared camera 10a, on which defective pixels have been interpolated, from the defective pixel correction unit 171, and performs a predetermined calibration process on the received image data. The calibration processing unit 173 performs calibration by referring to the ambient temperature to the sensor output value of the thermal image data when the mechanical shutter 103 that blocks the light incident on the infrared sensor 104 of the infrared camera 10a is open. In the calibration process, the calibration processing unit 173 calculates a first offset calibration value for each element of the infrared sensor 104 of the infrared camera 10a from the pixel values ​​of the non-shuttered image. By performing the calibration process, the calibration processing unit 173 outputs first offset calibration data for calibrating the offset processing performed by the uniformization processing unit 172 and stores it in the RAM 140.

[0041] The defective pixel correction unit 171 in the first image processing unit 170 pre-stores the defective pixels of the infrared sensor 104 of the infrared camera 10a and performs interpolation processing (interpolation processing) to interpolate the pixel values ​​of the stored defective pixels from the pixel values ​​of surrounding pixels. The defective pixel correction unit 171 receives input thermal image data from the infrared camera 10a via the image data acquisition unit 160 and performs the above-described interpolation processing on the received image data. The defective pixel correction unit 171 supplies the interpolated image data to the uniformization processing unit 172 and the calibration processing unit 173.

[0042] The uniformization processing unit 172 performs correction processing to convert the thermal image into a motion detection image. The uniformization processing unit 172 has pre-configured gain settings corresponding to the characteristics of each pixel of the infrared sensor 104. The uniformization processing unit 172 corrects the pixel values ​​in the image captured by the infrared camera 10a based on the image data received from the defective pixel correction unit 171, according to these pre-configured settings and the first offset calibration data output by the calibration processing unit 173. In other words, the control device 11 outputs motion detection image data from the image captured by the infrared camera 10a via the defective pixel correction unit 171 and the uniformization processing unit 172.

[0043] After the first image processing unit 170 has performed calibration processing on the non-shutter image, the equalization processing unit 172 uses the first offset calibration data and gain data generated by the calibration processing unit 173 to output motion detection image data from the thermal image data captured by the infrared camera 10a. More specifically, if the RAM 140 has stored the first offset calibration data, the equalization processing unit 172 uses this data to perform offset correction. When the first offset calibration data stored in the RAM 140 is updated, the equalization processing unit 172 uses the updated offset calibration data.

[0044] The edge image acquisition unit 174 performs image processing on the motion detection image generated by the uniformization processing unit 172 and outputs edge image data. It is preferable to use a filter for the process by which the edge image acquisition unit 174 generates the edge image data. Examples of such filters include the Sobel filter, Laplacian filter, and Previtt filter.

[0045] The edge image acquisition unit 174 may also perform image processing on the motion detection image based on the difference between the average surface temperature of the object being measured and the ambient temperature, and output edge image data. Here, ambient temperature refers to the outside temperature if the area being measured is outdoors, and the indoor temperature if the area being measured is indoors. For example, the edge image acquisition unit 174 may output edge image data using multiple filters based on the difference between the average surface temperature of the object being measured and the ambient temperature. For example, the edge image acquisition unit 174 may output edge image data by changing the settings such as the strength of the filters used based on the difference between the average surface temperature of the object being measured and the ambient temperature. It is advisable to appropriately change the filters and their settings depending on the surroundings and ambient temperature.

[0046] Here, with reference to Figures 23 to 26, specific examples of the thermal images, motion detection images, and edge image data described above will be explained.

[0047] Figure 23 shows an example of a thermal image. Figure 23 is a thermal image IM31 processed by the second image processing unit 180 from image data captured by the infrared camera 10b at time t31, for example. In thermal image IM31, the object of measurement is truck TV1, and the measurement area is the road including truck TV1 and its surroundings. Vegetation GT1 is provided around the road. Figure 24 shows an example of a motion detection image. Figure 24 is a motion detection image IM32 processed by the homogenization processing unit 172 from image data captured by the infrared camera 10a at time t32, after an arbitrary period has elapsed from time t31, for example.

[0048] In this specific example, the average surface temperature of truck TV1 is higher than the ambient temperature, while the average surface temperature of vegetation GT1 is acclimatized to the ambient temperature, so the average surface temperature of vegetation GT1 is the same as or close to the ambient temperature. The temperature difference between the average surface temperature of vegetation GT1 and the ambient temperature is smaller than the temperature difference between the average surface temperature of truck TV1 and the ambient temperature. Therefore, even if vegetation GT1 is shaken by external forces such as wind, the edge component of vegetation GT1 is weaker compared to truck TV1 in the motion detection image IM32.

[0049] In such a situation, the edge image acquisition unit 174 may perform noise reduction processing on the motion detection image using a Gaussian filter or the like as a preprocessing step for edge extraction, and then perform edge extraction processing using an edge extraction filter. In other words, the edge image acquisition unit 174 performs noise reduction processing on the motion detection image IM32 using a Gaussian filter, then performs edge extraction processing using an edge extraction filter, and outputs edge image data. Figure 25 shows an example of an edge image IM33 that has undergone pre-denoising processing. As shown in Figure 25, in image IM33, the edge component of the vegetation GT1 is weak, and the presence of the contour line indicating the vegetation GT1 can hardly be confirmed. On the other hand, in image IM33, the edge component of the truck TV1 is strong, and the presence of the contour line indicating the truck TV1 can be easily confirmed.

[0050] For example, if the detection system 100 takes thermal images during the summer, the difference between the average surface temperature of the object being measured and the ambient temperature is smaller compared to other seasons, resulting in weaker edge components of the object being measured in the motion detection image. In such cases, the edge image acquisition unit 174 may perform only edge extraction processing without noise reduction processing. When this processing is applied in this specific example, the edge image acquisition unit 174 does not perform noise reduction processing on the motion detection image IM32, but performs edge extraction processing using an edge extraction filter and outputs edge image data. Figure 26 shows edge image IM34, an example of an edge image that has not undergone prior noise reduction processing. As shown in Figure 26, in edge image IM34, the edge component of the vegetation GT1 is slightly stronger than the edge component of the vegetation GT1 in edge image IM33 shown in Figure 25, and the presence of a contour line indicating vegetation GT1 can be confirmed. On the other hand, in image IM34, the edge component of the truck TV1 is significantly stronger than the edge component of vegetation GT1, and the presence of a contour line indicating truck TV1 can be easily confirmed.

[0051] The edge image acquisition unit 174 may increase the strength of the filter used and output edge image data when the difference between the average surface temperature of the object being measured and the ambient temperature is large, such as when the detection system 100 is taking thermal images during the winter. The edge image acquisition unit 174 may decrease the strength of the filter used and output edge image data when the difference between the average surface temperature of the object being measured and the ambient temperature is small, such as when the infrared camera 10b is taking thermal images during the summer.

[0052] The difference between the average surface temperature of the object being measured and the ambient temperature is often larger than the difference between the temperatures of any two surrounding areas that are not the object being measured. Therefore, the detection system 100 can suppress the extraction of edges in the surrounding area that are not the object being measured by outputting edge image data based on the difference between the average surface temperature of the object being measured and the ambient temperature. In other words, in the image shown by this output edge image data, only the edges between the object being measured and its surroundings can be extracted. Thus, the object being measured and its surroundings are easily distinguishable. Furthermore, with this configuration, the edges between the object being measured and its surroundings can be easily distinguished in the motion detection image, making it easy to pinpoint the location of the object being measured.

[0053] Furthermore, this configuration is particularly suitable for motion detection images in which it is difficult to extract only the edges between the object being measured and its surroundings. Specific examples of motion detection images in which it is difficult to extract only the edges between the object being measured and its surroundings include motion detection images in which noise is generated due to wind blowing in the area being measured, and motion detection images in which the difference between the average surface temperature of the object being measured and the ambient temperature is small.

[0054] Up to this point, the edge image acquisition unit 174 has performed image processing on the motion detection image generated by the uniformization processing unit 172. However, it may also perform image processing on a normal thermal image based on the difference between the average surface temperature of the object being measured and the ambient temperature, and output edge image data. Here, a normal thermal image is, for example, a thermal image captured by an infrared camera 10b and processed by the NUC unit 182. In this case, the object detection unit 153 determines whether the thermal image is non-uniform based on the edge image output by the edge image acquisition unit 174. If it determines that the thermal image is non-uniform, it detects the presence of an object in the inner region, which is a part of the edge image with a large difference in brightness compared to other parts. However, the object detection unit 153 does not detect whether the object is moving or not. In this case, the detection system 100 does not include an infrared camera 10a, the control device 11 does not include a first image processing unit 170, and the second image processing unit 180 may include an edge image acquisition unit 174.

[0055] In this way, the control device 11 or the detection system 100 can detect from the thermal image whether or not the object to be measured is present in the measurement area with a simple configuration and processing.

[0056] Next, the second image processing unit 180 will be described with reference to Figure 5. Figure 5 is a block diagram of the second image processing unit 180 according to Embodiment 1. The second image processing unit 180 performs predetermined processing in cooperation with the RAM 140 and the control circuit 150. The second image processing unit 180 mainly consists of a defective pixel correction unit 181, a NUC unit 182, and a calibration processing unit 183.

[0057] The calibration processing unit 183 of the second image processing unit 180 is responsible for calibrating the offset processing, one of the correction processes performed by the NUC unit 182, using the shutter image. The calibration processing unit 183 receives image data of the shutter image captured by the infrared camera 10b, which has been interpolated for defective pixels, from the defective pixel correction unit 181, and performs a predetermined calibration process on the received image data. The calibration processing unit 183 performs calibration by referring to the ambient temperature to the sensor output value of the thermal image data when the mechanical shutter 103 that blocks the light incident on the infrared sensor 104 of the infrared camera 10b is closed. In the calibration process, the calibration processing unit 183 calculates a second offset calibration value for each element of the infrared sensor 104 of the infrared camera 10b from the pixel values ​​of the non-shutter image. By performing the calibration process, the calibration processing unit 183 outputs second offset calibration data for calibrating the offset processing performed by the NUC unit 182 and stores it in the RAM 140.

[0058] The defective pixel correction unit 181 in the second image processing unit 180 pre-stores the defective pixels of the infrared sensor 104 of the infrared camera 10b and performs interpolation processing (interpolation) to interpolate the pixel values ​​of the stored defective pixels from the pixel values ​​of surrounding pixels. The defective pixel correction unit 181 receives input thermal image data from the infrared camera 10b via the image data acquisition unit 160 and performs the above-described interpolation processing on the received image data. The defective pixel correction unit 181 supplies the interpolated image data to the NUC unit 182 and the calibration processing unit 183.

[0059] The NUC unit 182 performs NUC (Non-Uniformity Correction), a correction process to suppress variations in the output pixel values ​​in response to the input light. The NUC unit 182 has pre-configured gain settings corresponding to the characteristics of each pixel of the infrared sensor 104. The NUC unit 182 corrects the pixel values ​​in the image captured by the infrared camera 10b based on the image data received from the defective pixel correction unit 181, according to these pre-configured settings and the second offset calibration data output by the calibration processing unit 183. In other words, the control device 11 improves the image quality of the image captured by the infrared camera 10b through the defective pixel correction unit 181 and the NUC unit 182.

[0060] After the second image processing unit 180 performs calibration processing on the shutter image, the NUC unit 182 uses the second offset calibration data and gain data generated by the calibration processing unit 183 to suppress the degradation of image quality of the thermal image data captured by the infrared camera 10b. More specifically, if the RAM 140 has stored the second offset calibration data, the NUC unit 182 uses this data to perform the second offset correction. When the second offset calibration data stored in the RAM 140 is updated, the NUC unit 182 uses the updated offset calibration data.

[0061] (Summary of the process) Next, with reference to Figure 6, an overview of the processes performed by the detection system 100 will be described. Figure 6 is a block diagram of the processing of the detection system according to Embodiment 1.

[0062] Using the infrared camera 10a, the measurement target area is imaged with the mechanical shutter 103 open, and a thermal image is generated (step ST1).

[0063] Next, the calibration processing unit 173 performs calibration processing on the thermal image (non-shutter image) of the measurement target area captured by the infrared camera 10a with the mechanical shutter 103 open, and calculates a first offset calibration value (step ST2). Then, the infrared camera 10a captures a non-shutter image again to generate a thermal image (step ST3), and the homogenization processing unit 172 corrects the thermal image data of the non-shutter image to generate a motion detection image (step ST4). Specifically, the homogenization processing unit 172 shown in Figure 4 corrects the output value of each pixel of the thermal image using the first offset calibration data and gain data, and outputs a motion detection image. From here on, the acquisition of non-shutter images and the generation of motion detection images are repeated at a constant frame rate.

[0064] Meanwhile, the infrared camera 10b captures a thermal image (shutter image) of the area to be measured with the mechanical shutter 103 closed, and generates a thermal image (step ST21). Next, the calibration processing unit 183 performs calibration processing on the shutter image captured by the infrared camera 10b and calculates a second offset calibration value (step ST22). Then, the infrared camera 10b captures a thermal image of the area to be measured with the mechanical shutter 103 open, and generates a thermal image (step ST22), and the NUC unit 182 corrects the thermal image data (step ST24). Specifically, the NUC unit 182 shown in Figure 5 corrects the output value of each pixel of the thermal image using the second offset calibration data and gain data.

[0065] Next, the edges of the motion detection image are detected (step ST5). Specifically, the edge image acquisition unit 174 performs image processing on the motion detection image output by the uniformization processing unit 172 and outputs an edge image. The object detection unit 153 determines whether the motion detection image is non-uniform or not based on the edge image output by the edge image acquisition unit 174. If the motion detection image is determined to be non-uniform, it is determined that an edge of the motion detection image has been detected. If an edge of the motion detection image is detected, the object detection unit 153 detects the part of the edge image with a large difference in brightness compared to other parts as an inner region, and extracts any pixel in the inner region as a pixel of interest. Here, the inner region may be one region or multiple regions. If there are multiple inner regions, the object detection unit 153 extracts a pixel of interest for each inner region. The pixel of interest may be one pixel, multiple pixels, or all pixels in one inner region. When the object detection unit 153 detects an edge in the motion detection image, it detects that a moving object exists in the inner region.

[0066] Next, the absolute temperature of the extracted pixel of interest is measured (step ST6). Specifically, the object temperature measurement unit 154 shown in Figure 3 determines the absolute temperature of the pixel of interest in the image corrected from the thermal image captured by the infrared camera 10b, which corresponds to the inner region of the motion detection image.

[0067] Next, the absolute temperature of the selected pixel of interest is compared with the average surface temperature of the object to be measured (step ST7), and the object to be measured is detected (step ST8). Specifically, the object to be measured determination unit 155 shown in Figure 3 determines that an object to be measured has been detected if the absolute temperature of the extracted pixel of interest falls within a predetermined temperature range that indicates the average surface temperature of the object to be measured. Here, the predetermined temperature range is, for example, 30°C to 40°C. Of course, the predetermined temperature range is not limited to this. If there are multiple inner regions, the object to be measured determination unit 155 determines that an object to be measured has been detected if the absolute temperature of the pixel of interest in at least one of the multiple inner regions falls within the predetermined temperature range that indicates the average surface temperature of the object to be measured.

[0068] Next, recording of the thermal image of the measurement target area to the recording unit begins (step ST9). The infrared camera 10b captures an image of the measurement target area, and the second image processing unit 180 processes the thermal image, which is then stored in the ROM 130 or an external storage device.

[0069] (Detection method) Next, an example of a detection method performed by the control device 11 will be described with reference to Figure 7. Figure 7 is a flowchart of the detection method according to Embodiment 1. The processes shown in the following flowchart include processes performed by the infrared camera 10a, infrared camera 10b, or each component of the control device 11 as instructed by the control circuit 150, or each component of the control circuit 150.

[0070] At the start of the flow shown in Figure 7, the infrared camera 10b is assumed to be open and capturing images at a constant frame rate. Here, it is assumed that the second offset calibration value for correcting the infrared camera 10b has already been acquired, or that calibration processing is performed asynchronously with respect to the flow shown in Figure 7 at regular intervals (e.g., every 2 minutes), or when the temperature of the temperature sensor 106 changes by 0.5°C or more, and the second offset calibration value is updated.

[0071] The control circuit 150 starts image processing of the infrared camera 10a (step ST101). Specifically, the infrared camera 10a opens the mechanical shutter 103 and captures a first thermal image while keeping the measurement target area within the field of view.

[0072] Next, with the mechanical shutter 103 of the infrared camera 10a open, the thermal image captured by the infrared camera 10a over the measurement target area is calibrated, and a first offset calibration value is calculated (step ST102).

[0073] Next, the first image processing unit 170 generates a motion detection image based on the thermal image captured by the infrared camera 10a (step ST103). Specifically, the first image processing unit 170 corrects the output value of each pixel of the thermal image captured by the infrared camera 10a using first offset calibration data and gain data, and outputs a motion detection image.

[0074] Next, image processing is applied to the motion detection image and an edge image is output (step ST104). Specifically, the edge image acquisition unit 174 applies a filter to the motion detection image, performs image processing, and acquires an edge image.

[0075] Next, it is determined whether or not an object has been detected in the inner region of the edge image (step ST105). Specifically, the object detection unit 153 determines whether or not the motion detection image is non-uniform based on the edge image. If the object detection unit 153 determines that the motion detection image is non-uniform, it detects the presence of a moving object in the inner region of the edge image, which is a part where the brightness difference is large compared to other parts.

[0076] Steps ST103 and ST104 are repeated until an object is detected (Step ST105: NO). Steps ST103 and ST104 are repeated at a constant interval (for example, 30 times per second). If an object is detected (Step ST105: YES), the object detection unit 153 extracts the pixel of interest in the thermal image captured by the infrared camera 10b that corresponds to the pixel of interest in the inner region of the edge image (Step ST106). Specifically, the object detection unit 153 detects an area in the edge image with a large difference in brightness compared to other parts as an inner region, and extracts an arbitrary pixel in the inner region as a pixel of interest. Furthermore, the absolute temperature of the inner region is determined (Step ST107). Specifically, the object temperature measurement unit 154 determines the absolute temperature of the pixel of interest in the inner region captured by the infrared camera 10b based on the motion detection image.

[0077] Next, the absolute temperature of the inner region is compared with the average surface temperature of the object being measured (step ST108). Specifically, the object determination unit 155 compares the absolute temperature obtained in step ST107 with the average surface temperature of the object being measured, which has been measured in advance.

[0078] Next, it is determined whether or not the object is a target for measurement (step ST109). Specifically, the measurement target determination unit 155 determines whether or not the object is a target for measurement based on the results of this comparison. For example, the measurement target determination unit 155 determines that the object is a target for measurement if the absolute temperature is within a predetermined temperature range indicating the average surface temperature.

[0079] Steps ST3 to ST9 are repeated until an object is determined to be a target for measurement (step ST109: NO). If it is determined to be a target for measurement (step ST109: YES), the control circuit 150 starts recording the infrared image captured by the infrared camera 10b (step ST110). The image data relating to the infrared image captured by the infrared camera 10b is recorded, for example, in the ROM 130 or an external recording device.

[0080] In addition, after detecting an object in the inner region in step ST105, the infrared camera 10a may return to step ST102 and repeat the same operations as in steps ST102 to ST104.

[0081] Based on the above, the configuration of the detection system 100 allows for object detection based on motion detection images. Therefore, it is not necessary to store the entire screen data of one frame, and objects can be detected efficiently. In other words, the efficiency of object detection can be improved, and the efficiency of detecting the object to be measured can be improved.

[0082] Furthermore, according to the configuration of the detection system 100, it determines whether or not an object is a target for measurement based on the absolute temperature of the detected object and the average surface temperature of the target for measurement. Therefore, the efficiency of detecting targets for measurement can be further improved.

[0083] Furthermore, according to the configuration of the detection system 100, recording of infrared images captured by the infrared camera 10b begins only after the object to be measured has been detected. Therefore, since there is no need to record infrared images captured by the infrared camera 10b until the object to be measured is detected, the recording capacity of the recording unit such as the ROM 130 and the power consumption of the detection system can be reduced. Consequently, the object to be measured can be detected efficiently.

[0084] (A first specific example of the processing of the detection system 100) Next, a first specific example of the processing of the detection system 100 will be described with reference to Figures 8 to 12. Figures 8 to 11 show a first example of detection by the detection system. Figure 12 shows an example of the correspondence between pixels.

[0085] Image IM1, shown in Figure 8, schematically illustrates an example of the measurement target area captured by the infrared camera 10a in step ST101. Here, the measurement target is a wild boar, and the measurement target area is pasture G1. Image IM1 shows the wild boar AN1 and pasture G1, as well as trees TR1, mountains, fences, etc. The wild boar AN1 is facing towards the tree TR1, and the tip of the boar AN1's ​​nose L1 is in a predetermined position.

[0086] In step ST102, the calibration processing unit 173 calibrates the thermal image captured of the measurement target area shown in image IM1 and calculates a first offset calibration value. In step ST103, the first image processing unit 170 generates the motion detection image IM2 shown in Figure 9 based on image IM1. Here, the motion detection image IM2 is a thermal image showing the same measurement target area as image IM1, captured in step ST101, which has been corrected using the first offset calibration value and gain data. Since the output value of each pixel is uniform throughout the entire image of the motion detection image IM2, the brightness of each pixel is uniform. As shown in Figure 9, the motion detection image IM2 captured in step ST101 appears as if the entire image is dyed in one color.

[0087] In step ST103, a motion detection image IM4 is generated at time t2, which is an arbitrary period of time after time t1, when the measurement target area shown in image IM1 was captured. Here, the arbitrary period is, for example, the period from time t1 to the next imaging timing in the shooting frame rate of the infrared camera 10a. For example, if the shooting frame rate of the infrared camera 10a is 60fps, the arbitrary period is 1 / 60 second. Image IM3 shown in Figure 10 is a schematic diagram showing an example of the measurement target area captured by the infrared camera 10a at time t2. Image IM3 shows a boar AN2 and pasture G1, as well as a tree TR1, mountains, fence, etc. The boar AN2 is facing towards the tree TR1, and the tip of the boar AN2's nose L2 is closer to the tree TR1 than the tip of the nose L1. Between time t1 and time t2, the boar AN2 moved towards the tree TR1. The motion detection image IM4 includes the portion AN21 of the boar AN2 that extends from the boar AN1 towards the tree TR1. In the motion detection image IM2, the output values ​​of each pixel in boar AN1 and the output values ​​of each pixel surrounding boar AN1 are uniform, but the magnitude of the correction to the output values ​​of each pixel surrounding boar AN1, including the portion corresponding to partial AN21, is higher than the magnitude of the correction to the output values ​​of each pixel in boar AN1. In image IM3, the surface temperature of boar AN2 is higher than the temperature around boar AN2. Therefore, in the motion detection image IM4, the output value of each pixel in partial AN21 is significantly higher than the output value of each pixel surrounding partial AN21. The output value of each pixel in partial AN21 is high, and the brightness of each pixel is high. In other words, the difference in output values ​​between each pixel in partial AN21 and each pixel surrounding partial AN21 is large. That is, in Figure 11, the inner region is partial AN21, enclosed by the contour lines of boar AN1 and boar AN2. Here, depending on the ambient temperature, the temperature of partial AN may be lower, in which case the output value of each pixel in partial AN21 will be low, and the brightness of each pixel will be low.

[0088] In step ST104, the edge image acquisition unit 174 performs image processing on the motion detection image IM4 and outputs edge image data. In step ST105, the object detection unit 153 determines whether or not it has detected an edge in the motion detection image based on the edge image. Specifically, since the edge image data of the motion detection image IM4 contains an edge (contour) of part AN21, the object detection unit 153 determines that it has detected an edge in the image (step ST105: Yes). For example, since the edge image data of the motion detection image IM2 does not contain an edge (contour), the object detection unit 153 determines that it has not detected an edge in the image and has not detected a moving object (step ST105: No), and returns to the process in step ST103. Also in step ST105, the object detection unit 153 detects part AN21, which has a large difference in brightness compared to other parts, as an inner region based on the edge image data and the motion detection image IM4. In other words, in step ST105, when the object detection unit 153 detects the edge of the motion detection image, it determines that it has detected a moving object in the inner region of part AN21, which is the inner region.

[0089] When the object detection unit 153 determines that it has detected a moving object (step ST105: Yes), in step ST106, it calculates the pixel of interest in the infrared camera 10b from the correspondence between the pixel of interest in the inner region (part AN21) and the pixels of the infrared camera 10a and infrared camera 10b that have been acquired in advance.

[0090] Here, the correspondence between pixels of infrared camera 10a and infrared camera 10b is obtained, for example, from the distance between the mounting positions of each camera and the difference in shooting direction. An example of such a pixel correspondence will be explained with reference to Figure 12. In image IM1 shown in Figure 12, the field of view AF1 of infrared camera 10a and the field of view AF2 of infrared camera 10b are shown. Pixel E1 included in field of view AF1 corresponds to pixel E2 included in field of view AF2. Pixels E1 and E2 indicate the same part of the measurement target area in image IM1. The coordinate (0,0) of pixel E2 in field of view AF2 corresponds to the coordinate (10,10) of pixel E1 in field of view AF1. That is, the correspondence between pixels E1 and E2 is clear.

[0091] In step ST107, the absolute temperature of the pixel of interest in the inner region, part AN21, is determined.

[0092] In step ST108, the absolute temperature of the pixel of interest in the inner region, part AN21, is compared with the average surface temperature of the boar being measured. In step ST109, it is determined whether the object is the boar being measured based on the result of this comparison. Since boar AN2 was present in the inner region, part AN21, the absolute temperature of the pixel of interest in the inner region, part AN21, is within the predetermined temperature range that represents the average surface temperature of the boar being measured. Therefore, it is determined that the object is the boar being measured.

[0093] In step ST110, the control circuit 150 starts recording the infrared image captured by the infrared camera 10b.

[0094] (A second specific example of the processing of the detection system 100) Next, a second specific example of the processing of the detection system 100 will be described with reference to Figures 13 and 14. Figures 13 and 14 show a second example of detection by the detection system.

[0095] An example of the first thermal image in step ST101 (not shown) shows that the object of measurement is a person, specifically their hand H1.

[0096] In step ST102, the first example of the thermal image is calibrated and a first offset calibration value is calculated. In step ST103, the first image processing unit 170 generates the motion detection image IM21 shown in Figure 13 based on the first example of the thermal image. In the motion detection image IM21, the output value of each pixel is approximately uniform throughout the image, and therefore the brightness of each pixel is approximately uniform. As shown in Figure 13, the motion detection image IM21 is approximately a single color throughout the image. The measurement area shown in the motion detection image IM21 includes the hand H1 of the person being measured.

[0097] In step ST103, a motion detection image IM22 is generated at time t22, after an arbitrary period has elapsed from time t21, when the first example thermal image was taken. Between time t21 and time t22, hand H1 moves to the upper side of the motion detection image IM21 and is located in the same place as hand H2 shown in the motion detection image IM22. In the thermal image (not shown) taken by the infrared camera 10a at time t22, hand H1 is located at the top of the image and is shown as hand H2. Similar to step ST103, the first image processing unit 170 generates the motion detection image IM22 shown in Figure 14 based on the thermal image. The motion detection image IM22 includes portions H21 and H23 of hand H2 that extend from hand H1 upwards in the image. The motion detection image IM22 also includes portion H22 where hand H1 is present at time t21 but not at time t22. In the motion detection image IM22, the output values ​​of each pixel of hand H1 and the output values ​​of each pixel surrounding hand H1 are uniform, but the magnitude of the correction to the output values ​​of each pixel surrounding hand H1, including the parts corresponding to H21 and H23, is higher than the magnitude of the correction to the output values ​​of each pixel of hand H1. In the thermal image (not shown) related to the motion detection image IM22, the output values ​​of each pixel of hand H2 are higher than the output values ​​of each pixel surrounding hand H2. In the motion detection image IM22, the output values ​​of each pixel in the part corresponding to H22 are lower than the output values ​​of each pixel of hand H1 and hand H2. Therefore, in the motion detection image IM22, the output values ​​of each pixel of parts H21 and H23 are significantly higher than the output values ​​of each pixel surrounding parts H21 and H23. The output values ​​of each pixel of parts H21 and H23 are high, and the brightness of each pixel is high. In other words, in Figure 14, the inner region consists of three areas H21, H22, and H23, which have a large difference in brightness compared to other parts.

[0098] In step ST104, the edge image acquisition unit 174 performs image processing on the motion detection image IM22 and outputs edge image data, and based on the edge image data, it detects a portion H21 with a large difference in brightness compared to other portions. In step ST105, it determines whether or not an object has been detected in the inner region, portion H21. If the object detection unit 153 determines that an object has been detected, in step ST106, it calculates the pixel of interest in infrared camera 10b from the correspondence between the pixel of interest in the inner region (portion H21) and the pixels of infrared camera 10a and infrared camera 10b that have been acquired in advance. In step ST107, it determines the absolute temperature of the pixel of interest in the inner region, portion H21.

[0099] In step ST108, the absolute temperature of the pixel of interest in the three inner regions H21, H22, and H23 is compared with the average surface temperature of the human being being measured. In step ST109, it is determined whether the object is the human being being measured based on the results of this comparison. Since a hand H2 was present in the inner regions H21 and H23, the absolute temperature of the pixel of interest in these regions is almost the same as the average surface temperature of the human being being measured. In other words, since the pixel of interest in two of the three inner regions is almost the same as the average surface temperature of the human being being measured, it is determined that the object is the human being being measured. Note that since a hand H2 is not present in the inner region H22, the absolute temperature of the pixel of interest in H23 is different from the average surface temperature of the human being being measured. In this specific example, the criterion used to determine that the object is the human being being measured was that the pixel of interest in two of the three inner regions is almost the same as the average surface temperature of the human being being measured, but other criteria may be used as appropriate.

[0100] In step ST110, the control circuit 150 starts recording the infrared image captured by the infrared camera 10b.

[0101] (Embodiment 2) Embodiment 2 will be described with reference to Figure 15. Figure 15 is a configuration diagram of the detection system according to Embodiment 2. As shown in Figure 15, the detection system 200 includes a control device 21, an infrared camera 10c, and a display 12. Unlike the detection system 100 shown in Figure 1, the detection system 200 does not include infrared cameras 10a and 10b. The detection system 200 only needs to have one infrared camera, infrared camera 10c. The detection system 200 does not need to include a display 12, similar to the example of the detection system 100 shown in Figure 1.

[0102] The infrared camera 10c is preferably a far-infrared camera, similar to the infrared camera 10a. The infrared camera 10c is connected to the control device 21 in a communicative manner, receives predetermined instruction signals from the control device 21, and operates according to the received instruction signals. The infrared camera 10c also supplies the control device 21 with image data (also called thermal image data) relating to the image (also called thermal image) captured by the infrared camera 10c. The infrared camera 10c is preferably installed in a location where it can photograph the measurement target area described above, similar to the infrared camera 10a. The installation location of the infrared camera 10c is not particularly limited, similar to the infrared camera 10a, but is preferably a fixed location outdoors or indoors, such as a road, plain, pasture, office building district, elevator, etc.

[0103] The control device 21 is connected to the infrared camera 10c and the display 12 in a communicative manner. The control device 21 may be connected to the infrared camera 10c and the display 12 by wire or by wireless connection. The control device 21 controls the infrared camera 10c to acquire image data generated by the infrared camera 10c. The control device 21 may display this acquired image data on the display 12. The detection system 200 may also connect multiple displays 12 and display the image data generated by the infrared camera 10c on each of the multiple displays 12.

[0104] The display 12 may also display images captured by the infrared camera 10c via the control device 21. The display 12 may be disconnected from the control device 21 from the time the detection system 200 starts detecting the object to be measured until it finishes, and may be located away from the control device 21.

[0105] With the above configuration, the control device 21 displays the image captured by the infrared camera 10c on the display 12 in a manner that is visible to the user. This allows the control device 21 to enable the user to recognize surrounding objects. If the detection system 200 does not have a display 12, the control device 21 may store the image captured by the infrared camera 10c in an external storage device (not shown).

[0106] (Infrared camera) Next, the configuration of the infrared camera 10c will be described with reference to Figure 16. Figure 16 is a diagram of the configuration of the infrared camera in the detection system shown in Figure 15. The infrared camera 10c shown in Figure 15 mainly comprises a housing 101, an objective lens 102, an infrared sensor 104, a camera control circuit 205, and a temperature sensor 106. The infrared camera 10c has the same configuration as the infrared camera 10a shown in Figure 2, except that it does not have a mechanical shutter 103 and has a camera control circuit 205 instead of a camera control circuit 105. Therefore, the explanation of the housing 101, objective lens 102, infrared sensor 104, and temperature sensor 106 will be omitted. Note that the infrared camera 10c may also have a mechanical shutter 103, similar to the infrared camera 10a shown in Figure 2, but it is preferable to take pictures with the mechanical shutter 103 open. If the infrared camera 10c is equipped with a mechanical shutter 103, this is to protect the sensor from sunlight or to keep the mechanical shutter 103 closed when the power is turned off.

[0107] Since the infrared camera 10c does not have a mechanical shutter 103, the light incident from the objective lens 102 to the infrared sensor 104 is not obstructed. Furthermore, the infrared camera 10c performs calibration and uniformization processing of the infrared sensor 104 without the mechanical shutter 103. In other words, unlike the infrared camera 10a, the infrared camera 10c performs calibration and uniformization processing of the infrared sensor 104 in a state that is substantially the same as if the mechanical shutter 103 were always open.

[0108] The camera control circuit 205 is communicatively connected to the control device 21 shown in Figure 15, receives control signals from the control device 21, and controls each component of the infrared camera 10c according to the received control signals. The camera control circuit 205 supplies thermal image data (also called a non-shuttered image) of the measurement target area to the control device 21 in a state that is substantially the same as when the mechanical shutter 103 is always open. When the infrared camera 10c takes an image, the camera control circuit 205 supplies image data including the sensor output value generated by the infrared sensor 104 to the control device 21.

[0109] The camera control circuit 205 is connected to the temperature sensor 106 in a communicative manner and receives measurement data regarding the temperature inside the infrared camera 10c from the temperature sensor 106. The camera control circuit 205 also supplies the measurement data received from the temperature sensor 106 to the control device 21.

[0110] (Control device) Next, the control device 21 will be described with reference to Figure 17. Figure 17 is a block diagram of the control device according to Embodiment 2. The control device 21 mainly consists of a control IF 120, ROM 130, RAM 140, control circuit 150, image data acquisition unit 260, image processing unit 270, and image data output unit 190. These components are connected to each other via the bus 110 for communication as appropriate. The control device 21 has the same configuration as the control device 11 shown in Figure 3, except that it has an image processing unit 270 instead of a first image processing unit 170 and a second image processing unit 180, and an image data acquisition unit 260 instead of an image data acquisition unit 160. Therefore, the description of the control IF 120, ROM 130, RAM 140, control circuit 150, and image data output unit 190 will be omitted.

[0111] The image data acquisition unit 260 is an interface for acquiring thermal image data (input thermal image data), which is data related to thermal imaging, from the infrared camera 10c. The image data acquisition unit 260 periodically acquires image data from the infrared camera 10c, for example. When the image data acquisition unit 260 receives image data from the infrared camera 10c, it supplies the received image data to the image processing unit 270.

[0112] The image processing unit 270 is, for example, an image processing circuit including a GPU. The image processing unit 270 works in conjunction with the RAM 140 to perform predetermined processing on the thermal image data. Details of the image processing unit 270 will be described later.

[0113] Next, the image processing unit 270 will be described with reference to Figure 18. Figure 18 is a block diagram of the image processing unit 270 according to Embodiment 2. The image processing unit 270 works in conjunction with the RAM 140 and the control circuit 150 to perform predetermined processing. The main components of the image processing unit 270 are a defective pixel correction unit 181, a NUC unit 282, a uniformization processing unit 172, an edge image acquisition unit 174, and a calibration processing unit 173.

[0114] The calibration processing unit 173 of the image processing unit 270 performs the same processing as the calibration processing unit 173 of the first image processing unit 170, so its explanation is omitted.

[0115] The defective pixel correction unit 181 in the image processing unit 270 pre-stores the defective pixels of the infrared sensor 104 of the infrared camera 10c and performs interpolation processing (interpolation) to interpolate the pixel values ​​of the stored defective pixels from the pixel values ​​of surrounding pixels. The defective pixel correction unit 181 receives input thermal image data from the infrared camera 10c via the image data acquisition unit 260 and performs the above-described interpolation processing on the received image data. The defective pixel correction unit 181 supplies the interpolated image data to the NUC unit 282, the uniformization processing unit 172, and the calibration processing unit 173.

[0116] The uniformization processing unit 172 in the image processing unit 270 performs the same processing as the uniformization processing unit 172 in the first image processing unit 170, so its explanation is omitted.

[0117] The NUC unit 282 performs NUC, a correction process to suppress variations in the output pixel values ​​in response to the input light. The NUC unit 282 has pre-set gain and offset values ​​corresponding to the characteristics of each pixel of the infrared sensor 104. In other words, the NUC unit 282 has pre-acquired second offset calibration data and gain data. The NUC unit 282 corrects each pixel value in the image captured by the infrared camera 10c according to these pre-set settings for the image data received from the defective pixel correction unit 181. That is, the control device 21 improves the image quality of the image captured by the infrared camera 10c by going through the defective pixel correction unit 181 and the NUC unit 282.

[0118] The NUC unit 282 suppresses the degradation of image quality of thermal image data captured by the infrared camera 10c by utilizing pre-acquired second offset calibration data and pre-acquired gain data. More specifically, if the RAM 140 has stored second offset calibration data, the NUC unit 282 uses this data to perform second offset correction. The second offset calibration data can be used for correction (shutterless correction) that adjusts the output of pixel values ​​for each pixel of the thermal image using a pre-set algorithm. The second offset calibration data can preferably be used for correction that dynamically adjusts the output of pixel values ​​for each pixel of the thermal image according to, for example, the subject being photographed and the ambient temperature, or only the ambient temperature. The NUC unit 282 should perform second offset correction and gain correction using gain data for each frame of the thermal image.

[0119] (Summary of the process) Next, with reference to Figure 19, an overview of the processes performed by the detection system 200 will be described. Figure 19 is a block diagram of the processing of the detection system according to Embodiment 2.

[0120] Using the infrared camera 10c, the measurement target area is imaged in a state that is substantially the same as when the mechanical shutter 103 is open, and a thermal image (non-shutter image) is generated (step ST31).

[0121] Next, the NUC unit 282 corrects the thermal image of the measurement target area captured in step ST31 (step ST32). Specifically, the NUC unit 282, as shown in Figure 18, corrects the output value of each pixel of the thermal image using pre-acquired second offset calibration data and pre-acquired gain data. Thereafter, non-shutter image acquisition and generation of the corrected thermal image (steps ST31 and ST32) are repeated at a constant frame rate.

[0122] Meanwhile, the calibration processing unit 173 performs calibration processing on the thermal image of the measurement target area captured in step ST31 and calculates a first offset calibration value (step ST42). The processing from steps ST43 to ST45 is the same as the processing from steps ST3 to ST5 shown in Figure 6, so the explanation is omitted.

[0123] The process from step ST7 to step ST9 shown in Figure 19 is identical to the process from step ST7 to step ST9 shown in Figure 6, so its explanation is omitted.

[0124] Next, recording of the thermal image of the measurement target area to the recording unit begins (step ST9). The infrared camera 10c captures an image of the measurement target area, and the thermal image processed by the image processing unit 270 is stored in the ROM 130 or an external storage device.

[0125] (Detection methods, etc.) Next, with reference to Figures 20 and 21, an example of a detection method performed by the control device 21 and a thermal image generation method performed in conjunction therewith will be described. Figure 20 is a flowchart of the detection method according to Embodiment 2. Figure 21 is a flowchart of the thermal image generation method of the detection method according to Embodiment 2. Note that the processes shown in the following flowcharts include processes performed by the infrared camera 10c or each component of the control device 21 as instructed by the control circuit 150 or each component of the control circuit 150.

[0126] At the point when the flow shown in Figure 20 begins, the infrared camera 10c is assumed to be capturing images at a constant frame rate.

[0127] The control circuit 150 starts image processing of the infrared camera 10c (step ST201). Specifically, the infrared camera 10c captures a thermal image in a state that is substantially the same as when the mechanical shutter 103 is open, while keeping the measurement target area within the field of view.

[0128] The processing from step ST202 to step ST205 is the same as the processing from step ST102 to step ST105 shown in Figure 7, so the explanation is omitted. If an object is detected (step ST205: YES), the pixel of interest in the thermal image captured by the infrared camera 10c that corresponds to the pixel of interest in the inner region of the edge image is extracted (step ST206). Specifically, the object detection unit 153 detects the area in the edge image with a large difference in brightness compared to other parts as the inner region, and extracts an arbitrary pixel in the inner region as the pixel of interest. Furthermore, the absolute temperature of the inner region is determined (step ST207). Specifically, the object temperature measurement unit 154 determines the absolute temperature of the pixel of interest in the inner region captured by the infrared camera 10c based on the motion detection image. The processing from step ST208 to step ST210 is the same as the processing from step ST108 to step ST110 shown in Figure 7, so the explanation is omitted.

[0129] Furthermore, the following thermal image generation method is performed in conjunction with the example of the detection method described above. The following thermal image generation method is preferably performed in parallel with the example of the detection method described above.

[0130] First, a thermal image is taken (step ST301), similar to step ST201 in the example of the detection method described above. For the sake of explanation, step ST201 was described in the detection method and step ST301 was described in the thermal image generation method, but steps ST201 and ST301 are the same step.

[0131] Next, an infrared image is generated using the previously acquired second offset calibration data and previously acquired gain data (step ST302). Specifically, the image processing unit 270 corrects the output value of each pixel of the thermal image captured by the infrared camera 10c using the second offset calibration data and previously acquired gain data to generate an infrared image. As described above, the image data relating to the infrared image thus generated is recorded in step ST210 shown in Figure 20.

[0132] Therefore, unlike detection system 100, detection system 200 only requires one infrared camera 10c, and does not need two infrared cameras. Consequently, detection system 200 has a simpler configuration and lower system cost compared to detection system 100. Detection system 200 acquires thermal images and motion detection images using one infrared camera 10c. On the other hand, detection system 100 acquires thermal images and motion detection images using two infrared cameras 10a and 10b, respectively. Consequently, the optical axes of the thermal images and motion detection images acquired by detection system 100 are on different axes, while the optical axes of the thermal images and motion detection images acquired by detection system 200 are on the same axis. Therefore, the detection accuracy of detection system 200 is higher than that of detection system 100.

[0133] (First specific example of the processing of the detection system 200) Next, a first specific example of the processing of the detection system 200 will be described with reference to Figures 8 to 11, Figure 20, and Figure 21. Note that the same measurement target area is set in the first specific example of the processing of the detection system 200 as in the first specific example of the processing of the detection system 100. Here, we will specifically explain the differences between the first specific example of the processing of the detection system 200 and the first specific example of the processing of the detection system 100.

[0134] Image IM1 in Figure 8 schematically shows an example of a measurement target area captured by the infrared camera 10c in step ST201. The processing from step ST202 to step ST205 is the same as the processing from step ST102 to step ST105 shown in Figure 7, so the explanation is omitted. In step ST205, when the object detection unit 153 determines that an object has been detected in the inner region of part AN21, in step ST206, the pixel of interest in the infrared camera 10c is extracted from the pixel of interest in the inner region (part AN21). In this first specific example of the processing of the detection system 200, the same measurement target area is captured using one infrared camera 10c. Therefore, in this specific example, unlike the first specific example of the processing of the detection system 100 described above, there is no need to pay attention to the correspondence between pixels of infrared camera 10a and infrared camera 10b.

[0135] In step ST207, the absolute temperature of the pixel of interest in the inner region of part AN21 captured by the infrared camera 10c is determined. The processing from steps ST208 to ST210 is the same as the processing from steps ST108 to ST110 shown in Figure 7, so the explanation is omitted.

[0136] Meanwhile, in step ST301 shown in Figure 21, the infrared camera 10c captures an example of the measurement target area indicated by images IM1 and IM3, and generates a thermal image. In step ST302, the NUC unit 282 corrects the thermal image generated in step ST301 using previously acquired second offset calibration data and previously acquired gain data, and generates an infrared image. In other words, the NUC unit 282 corrects the thermal image at time t1 and the thermal image at time t2 after an arbitrary period has elapsed from time t1, and generates an infrared image.

[0137] (Other embodiments, etc.) The detection system according to the above embodiment may have the following hardware configuration. Figure 22 shows an example of the hardware configuration included in the detection system. As the processing procedures in the detection system have been described in the various embodiments described above, this disclosure can also take the form of a processing method.

[0138] The detection system 100a shown in Figure 22 includes a processor 201 and a memory 202, along with an interface 203. The configuration of the control device 11 (see Figure 3) and the control device 21 (see Figure 17) described in the above-described embodiment is realized by the processor 201 reading and executing a control program stored in the memory 202. In other words, this program is a program that causes the processor 201 to function as the detection system 100, the detection system 200, or as a part thereof.

[0139] The program described above, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically, or otherwise propagating signals.

[0140] Furthermore, as described in the various embodiments above, the processing procedures in the detection system 100 and the detection system 200 have been explained, and this disclosure can also take the form of a control method for the detection system 100 and the detection system 200. The program described above can also be said to be a program that causes the detection system 100 and the detection system 200 to execute such a control method.

[0141] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, in the overview of the process executed by the detection system 100, the infrared camera 10a is used in steps ST1 to ST5 shown in Figure 6, while the infrared camera 10b is used in steps ST21 to ST24 and ST6 to ST9. However, in each step ST21 to ST24 and ST1 to ST9, the infrared camera 10a and the infrared camera 10b may be used as appropriate. Also, in the detection method executed by the control device 11, the infrared camera 10a is used in steps ST101 to ST105 shown in Figure 7, while the infrared camera 10b is used in steps ST106 to ST110. However, in each step ST101 to ST110, the infrared camera 10a and the infrared camera 10b may be used as appropriate.

[0142] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A first infrared camera (for example, the infrared camera 10b of the detection system 100), Edge image acquisition unit, It comprises an object detection unit, The first infrared camera captures a first thermal image while keeping the measurement target area within its field of view. After the first infrared camera captures the first thermal image while keeping the measurement target area within its field of view, it captures a second thermal image at a time after a specified period of time has elapsed. The edge image acquisition unit performs image processing on the first thermal image based on the difference between the average surface temperature of the object being measured and the ambient temperature, and outputs an edge image. The object detection unit determines whether the motion detection image is non-uniform based on the edge image, and if it determines that the motion detection image is non-uniform, it detects the presence of an object in the inner region of the edge image, which is a part where the brightness difference is large compared to other parts. Detection system. [Explanation of Symbols]

[0143] 100, 100a, 200 detection system 10a, 10b, 10c Infrared Cameras 11, 21 Control device 12 Display 101 Housing 102 Objective lens 103 Mechanical shutter 104 Infrared sensor 105, 205 Camera control circuit 106 Temperature sensor 110 Bus 120 Control IF 150 Control circuits 153 Object detection unit 154 Object temperature measurement unit 155 Measurement target determination unit 156 Image capture control unit 160, 260 Image data acquisition unit 170 First Image Processing Unit 171 Defective pixel correction unit 172 Uniformization processing unit 173, 183 Calibration processing unit 174 Edge image acquisition unit 180 Second Image Processing Unit 181 Defective pixel correction unit 182, 282 NUC unit 190 Image Data Output Unit 201 Processor 202 Memory 203 Interface 270 Image Processing Unit AN1, AN2 Boar G1 Pasture H1, H2 hand AN21, H21, H22, H23 part ST1-ST6, ST21-ST24, ST101-ST110 steps IM1, IM3 images; IM2, IM21 motion detection images. IM22, IM4, IM32 Motion detection images; IM31 Thermal image IM33, IM34 edge images L1, L2 Nose tip TR1 Tree TV1 Truck GT1 Grass and Trees

Claims

1. The first infrared camera, The first calibration processing unit and Homogenization processing unit, Edge image acquisition unit, It comprises an object detection unit, The first infrared camera captures a first thermal image while keeping the measurement target area within its field of view. The first calibration processing unit calculates a first offset calibration value by performing calibration on the output value of each pixel of the first thermal image. After the first infrared camera captures the first thermal image while keeping the measurement target area within its field of view, it captures a second thermal image at a time after a specified period of time has elapsed. The homogenization processing unit corrects the output value of each pixel of the second thermal image using the first offset calibration value, thereby outputting a motion detection image. The edge image acquisition unit performs image processing on the motion detection image and outputs an edge image. The object detection unit determines whether the motion detection image is non-uniform based on the edge image, and if it determines that the motion detection image is non-uniform, it detects the presence of a moving object in the inner region of the edge image, which is a region with a large difference in brightness compared to other parts. Detection system.

2. NUC Department and, Object temperature measurement unit, It further comprises a measurement target determination unit, The NUC unit corrects the output value of each pixel of the second thermal image using a previously acquired second offset calibration value, thereby outputting a corrected thermal image. When the object detection unit detects the presence of a moving object in the inner region, the object temperature measurement unit measures the absolute temperature of the inner region based on the corrected thermal image. The measurement target determination unit compares the absolute temperature with the average surface temperature of the measurement target that has been measured in advance to determine whether or not the object is a measurement target. The detection system according to claim 1.

3. A second infrared camera, The second calibration processing unit, NUC Department and, Object temperature measurement unit, It further comprises a measurement target determination unit, The second infrared camera captures a third thermal image with its mechanical shutter closed. The second calibration processing unit calculates a second offset calibration value by performing calibration on the output value of each pixel of the third thermal image. The second infrared camera opens its mechanical shutter and captures a fourth thermal image while keeping the measurement target area within its field of view. The NUC unit corrects the output value of each pixel of the fourth thermal image using the second offset calibration value, thereby outputting a corrected thermal image. The object temperature measuring unit, when the object detection unit detects the presence of a moving object in the inner region, Based on the corrected thermal image, the absolute temperature of the inner region is measured. The measurement target determination unit compares the absolute temperature with the average surface temperature of the measurement target that has been measured in advance to determine whether or not the object is a measurement target. The detection system according to claim 1.

4. The detection system according to any one of claims 1 to 3, wherein the edge image acquisition unit performs image processing on the motion detection image based on the difference between the average surface temperature of the object to be measured and the ambient temperature of the object to be measured and outputs an edge image.

5. It also has a recording section, If the recording unit determines that the object is the object to be measured, it records the image data relating to the corrected thermal image. The detection system according to claim 2 or 3.

6. The first step is to capture a thermal image while keeping the measurement target area within the field of view, The steps include: calculating a first offset calibration value by performing calibration on the output value of each pixel of the first thermal image; The steps include taking a first thermal image while keeping the measurement target area within the field of view, and then taking a second thermal image at a time after a specified period of time has elapsed, The steps include: outputting a motion detection image by correcting the output value of each pixel of the second thermal image using the first offset calibration value; The steps include: applying image processing to the motion detection image and outputting an edge image; The method includes the step of determining whether the motion detection image is non-uniform based on the edge image, and if it is determined that the motion detection image is non-uniform, detecting the presence of a moving object in the inner region, which is a part of the edge image where the brightness difference is large compared to other parts. Detection method.

7. The computer, which functions as the control unit for the detection system, The first step is to capture a thermal image while keeping the measurement target area within the field of view, The steps include: calculating a first offset calibration value by performing calibration on the output value of each pixel of the first thermal image; The steps include taking a first thermal image while keeping the measurement target area within the field of view, and then taking a second thermal image at a time after a specified period of time has elapsed, The steps include: outputting a motion detection image by correcting the output value of each pixel of the second thermal image using the first offset calibration value; The steps include: applying image processing to the motion detection image and outputting an edge image; The system performs the following steps: determine whether the motion detection image is non-uniform based on the edge image, and if it is determined that the motion detection image is non-uniform, detect the presence of a moving object in the inner region of the edge image, which is a region with a large difference in brightness compared to other parts. program.

Images