Detection system and detection method

Abstract

By performing high-precision image matching between multiple images captured by different infrared cameras, the system detects the target object with high accuracy. [Solution] The detection system 100 is a system for detecting the target gas 300 in the observation space 200, and comprises a plurality of infrared cameras 12-k whose optical axis Lk passes through the observation space 200, and a detection processing unit for detecting the target gas 300. The detection processing unit performs image matching between image G1 captured by infrared camera 12-1 and image G2 captured by infrared camera 12-2, using image features related to the brightness gradient in each of image G1 and image G2, and detects the target gas 30 using the results of the image matching.

Description

Technical Field

[0001] The present disclosure relates to a technique for detecting an observation target.

Background Art

[0002] Various techniques for detecting an observation target have been proposed heretofore. For example, Patent Document 1 discloses a technique for generating a pair of left and right thermal images by a pair of left and right infrared cameras, and emphasizing a target location by moving the pair of left and right thermal images in the left - right direction and superimposing the target locations.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, when the observation target is a gas, etc., since the contour (edge) of the observation target is unclear, it is difficult to perform image matching between a plurality of images with high precision. In view of the above circumstances, one aspect of the present disclosure aims to detect an observation target with high precision by performing image matching between a plurality of images captured by different infrared cameras with high precision.

Means for Solving the Problems

[0005] To solve the above problems, a detection system according to one aspect of the present disclosure is a detection system for detecting an object to be observed in an observation space, comprising a plurality of infrared cameras whose optical axes pass through the observation space, and a detection processing unit for detecting the object to be observed, wherein the detection processing unit performs image matching using image features relating to the brightness gradient in each of the first and second images between a first image captured by a first infrared camera among the plurality of infrared cameras and a second image captured by a second infrared camera different from the first infrared camera among the plurality of infrared cameras, and detects the object to be observed using the result of the image matching.

[0006] A detection method according to one aspect of the present disclosure is a method for detecting an object to be observed in an observation space using a plurality of infrared cameras whose optical axes pass through the observation space, wherein image matching is performed between a first image captured by a first infrared camera among the plurality of infrared cameras and a second image captured by a second infrared camera different from the first infrared camera among the plurality of infrared cameras, using image features relating to the brightness gradient in each of the first and second images, and the object to be observed is detected using the results of the image matching. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram illustrating the configuration of a detection system according to the first embodiment of this disclosure. [Figure 2] This is a block diagram illustrating the configuration of the analysis system. [Figure 3] This is a block diagram illustrating the functional configuration of the analysis system. [Figure 4] This is a diagram illustrating the processes performed by the analysis system. [Figure 5] This is a flowchart for image matching. [Figure 6] This is a flowchart of the analysis process. [Figure 7] This is a flowchart of the detection process. [Figure 8] This is a schematic diagram of the observation system in the second embodiment. [Figure 9] This is an explanatory diagram of the detection process in the second embodiment. [Modes for carrying out the invention]

[0008] The embodiments for implementing this disclosure will be described with reference to the drawings. Note that the dimensions and scale of the elements in each drawing may differ from those of the actual product. Furthermore, the embodiments described below are illustrative examples of embodiments that may be envisioned when implementing this disclosure. Therefore, the scope of this disclosure is not limited to the embodiments exemplified below.

[0009] A: First Embodiment Figure 1 is a block diagram illustrating the configuration of a detection system 100 according to the first embodiment of this disclosure. The detection system 100 is a measurement system for detecting the presence or absence of a gas to be observed (hereinafter referred to as "observation target gas") 300.

[0010] The gas to be observed 300 is a gas that may be present in the observation space 200. The observation space 200 is the space within a flow path (e.g., a flue) through which the gas to be observed 300, generated in an industrial or chemical process, flows. The wavelength of the absorption line of the gas to be observed 300 (hereinafter referred to as "absorption wavelength") is known. In the first embodiment, it is assumed that the absorption wavelength of the gas to be observed 300 is within the infrared wavelength range (hereinafter referred to as "infrared range"). The type of gas to be observed 300 is arbitrary, but ammonia is an example.

[0011] As illustrated in Figure 1, the detection system 100 comprises an observation system 10 and an analysis system 20. The observation system 10 is a system that optically observes the state of the target gas 300. The analysis system 20 detects the presence or absence of the target gas 300 in the observation space 200 by analyzing the results of the observation by the observation system 10. In Figure 1, the observation system 10 and the analysis system 20 are shown as separate components, but the observation system 10 and the analysis system 20 may be integrated. In other words, the detection system 100 may be realized by multiple devices that are configured separately from each other, or by a single device.

[0012] The observation system 10 comprises a plurality of infrared cameras 12-k (12-1, 12-2). Each infrared camera 12-k is an imaging device that images the observation space 200. Specifically, each infrared camera 12-k comprises an optical system such as a photographic lens and an image sensor that receives incident light from the optical system. The image sensor includes, for example, a light-receiving element formed of vanadium oxide. The optical characteristics (e.g., angle of view or focal length) and electrical characteristics (e.g., light-receiving sensitivity or light-receiving wavelength) of the infrared cameras 12-k are common to the plurality of infrared cameras 12-k.

[0013] As illustrated in Figure 1, the optical axis Lk of each of the multiple infrared cameras 12-k passes through the observation space 200. Infrared camera 12-1 is an example of a "first infrared camera," and infrared camera 12-2 is an example of a "second infrared camera."

[0014] Each infrared camera 12-k is capable of detecting infrared light corresponding to the absorption wavelength of the target gas 300. Specifically, each infrared camera 12-k is capable of detecting incident light within the infrared range that includes the absorption wavelength of the target gas 300. In other words, each infrared camera 12-k is sensitive to the absorption wavelength of the target gas 300 in the infrared range. As described above, according to the first embodiment, the target gas 300 whose absorption wavelength is within the infrared range can be detected with high accuracy.

[0015] FIG. 1 shows the positional relationship between infrared camera 12-1 and infrared camera 12-2. As illustrated in FIG. 1, infrared cameras 12-1 and 12-2 are arranged with their optical axes Lk parallel to each other and spaced apart by an interval Px in the X direction. The X direction is, for example, the horizontal direction. The interval Px is the distance between the optical axis L1 of infrared camera 12-1 and the optical axis L2 of infrared camera 12-2.

[0016] Infrared cameras 12-1 and 12-2 are installed such that the imaging ranges of infrared camera 12-1 and infrared camera 12-2 overlap each other. FIG. 1 shows the distance D between each infrared camera 12-k and the observation target gas 300, and the viewing angle θ of each infrared camera 12-k.

[0017] In order to observe the observation target gas 300 by the observation system 10, it is necessary that the observation target gas 300 exists at a position where the imaging ranges of infrared camera 12-1 and infrared camera 12-2 overlap each other. Therefore, the minimum value of the distance D (hereinafter referred to as "minimum detection distance") Dmin is expressed by the following formula (1).

Equation

[0018] For example, when the viewing angle θ of each infrared camera 12-k is 60° and the minimum detection distance Dmin is 10 cm, as understood from formula (1), it is necessary to set the interval Px between infrared camera 12-1 and infrared camera 12-2 to 11.5 cm (=2 / √3) or more.

[0019] The overlap ratio r between the imaging range of infrared camera 12-1 and the imaging range of infrared camera 12-2 is expressed by the following formula (2).

Equation

[0020] Each infrared camera 12-k captures an image Gk (G1, G2) of the imaging space. Image Gk is a thermal image representing the temperature distribution within the imaging range. Infrared camera 12-1 captures image G1 of the observation space 200. Infrared camera 12-2 captures image G2 of the observation space 200. Note that image G1 is an example of the "first image," and image G2 is an example of the "second image."

[0021] Each image Gk(G1,G2) is represented by a set of multiple pixel values ​​arranged in a matrix. The pixel value of each pixel is the intensity (i.e., brightness) of infrared light received by the photodetector in the portion of the imaging surface of the image sensor corresponding to that pixel. The acquisition of image G1 by infrared camera 12-1 and the acquisition of image G2 by infrared camera 12-2 are repeated periodically. That is, infrared camera 12-1 generates a time series (i.e., video) of image G1, and infrared camera 12-2 generates a time series (i.e., video) of image G2. Infrared camera 12-1 transmits image data representing image G1 to the analysis system 20, and infrared camera 12-2 transmits image data representing image G2 to the analysis system 20.

[0022] The analysis system 20 detects the presence or absence of the target gas 300 within the observation space 200 by analyzing the images Gk(G1, G2) captured by each infrared camera 12-k. Figure 2 is a block diagram illustrating the configuration of the analysis system 20. As illustrated in Figure 2, the analysis system 20 comprises a control device 21, a storage device 22, a communication device 23, a display device 24, and an operating device 25. The analysis system 20 can be implemented as a single device or as multiple devices configured separately from each other.

[0023] The control device 21 consists of one or more processors that control each element of the analysis system 20. Specifically, the control device 21 is composed of one or more types of processors, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0024] The communication device 23 communicates with each infrared camera 12-k. Specifically, the communication device 23 receives image data representing image G1 from infrared camera 12-1 and image data representing image G2 from infrared camera 12-2.

[0025] The storage device 22 is one or more memories that store the program executed by the control device 21 and the data used by the control device 21. The storage device 22 is composed of a recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 22 may be composed of a combination of multiple types of recording media. Sufficient capacity is reserved in the storage device 22 to store the image data of image G1 and the image data of image G2.

[0026] The display device 24 displays images under the control of the control device 21. Specifically, the display device 24 displays the detection results for the observed gas 300. The operating device 25 is an input device that receives operations from the user. Note that the display device 24 or operating device 25, which are separate from the analysis system 20, may be connected to the analysis system 20 by wire or wireless connection.

[0027] Figure 3 is a block diagram illustrating the functional configuration of the analysis system 20. Figure 4 is an explanatory diagram of the processes performed by the analysis system 20. As illustrated in Figure 3, the control device 21 realizes a detection processing unit 30 that detects the target gas 300 by executing a program stored in the storage device 22. The detection processing unit 30 includes an image processing unit 31 and an analysis processing unit 32.

[0028] The image processing unit 31 performs image matching between image G1 and image G2. Image matching is a process that analyzes the correspondence between image G1 and image G2. Specifically, as illustrated in Figure 4, image matching is an analytical process (pattern matching) that estimates mutually corresponding feature points F between image G1 and image G2.

[0029] The image processing unit 31 performs image matching between image G1 and image G2 using image features related to the luminance gradient in each of the images G1 and G2. In the first embodiment, the image features applied to image matching are HOG (Histogram of Oriented Gradients) features. HOG features are local image features that represent the gradient direction of pixel values ​​in local regions (cells) of image Gk using a histogram, and are calculated for each local region of image Gk. In the first embodiment, the observed gas 300, which is the target of detection, has an unclear contour in image Gk. That is, it is difficult to extract contours (edges) from image Gk with high accuracy. Therefore, as described above, image matching using image features related to the luminance gradient is particularly suitable.

[0030] Figure 5 is a flowchart of the image matching process performed by the image processing unit 31. When image matching is initiated, the image processing unit 31 calculates HOG features for each local region of image G1 (S21). The image processing unit 31 also calculates HOG features for each local region of image G2 (S22). By comparing the HOG features of each local region between image G1 and image G2, the image processing unit 31 estimates the mutually corresponding feature points F between image G1 and image G2 (S23). The image processing unit 31 may perform various image processing operations (e.g., cropping or filtering) on ​​image G1 and image G2 before performing image matching.

[0031] The analysis processing unit 32 in Figure 3 detects the target gas 300 using the results of image matching by the image processing unit 31. Specifically, as illustrated in Figure 4, the analysis processing unit 32 generates a composite image G12 by combining image G1 and image G2 using the results of image matching, and detects the target gas 300 by analyzing the composite image G12. As described above, the detection processing unit 30 performs image matching between image G1 and image G2 using image features (specifically HOG features) related to the brightness gradient in each of image G1 and image G2, and detects the target gas 300 using the results of image matching.

[0032] Figure 6 is a flowchart of the process performed by the analysis processing unit 32 (hereinafter referred to as "analysis processing"). When the analysis processing starts, the analysis processing unit 32 calculates a transformation matrix A to map image G1 to image G2 (S31). Transformation matrix A is a homography matrix that represents the projection transformation (e.g., scaling, translation, or rotation) applied to image G1. Specifically, as illustrated in Figure 4, the analysis processing unit 32 calculates transformation matrix A such that the feature points F of image G1 and the feature points F of image G2, estimated by image matching, match each other.

[0033] The analysis processing unit 32 generates a transformed image G1' by transforming (i.e., projectively transforming) image G1 using the transformation matrix A (S32). Specifically, as illustrated in Figure 4, the transformed image G1' is generated by multiplying image G1 by the transformation matrix A.

[0034] The analysis processing unit 32 generates a composite image G12 by combining the transformed image G1' and image G2 (S33). The pixel value of each pixel constituting the composite image G12 (hereinafter referred to as the "pixel of interest") is calculated by adding the pixel value of the pixel corresponding to the pixel of interest in the transformed image G1' and the pixel value of the pixel corresponding to the pixel of interest in image G2.

[0035] As described above, the analysis processing unit 32 generates a composite image G12 by combining the portion of image G1 estimated to correspond to image G2 through image matching and the portion of image G2 estimated to correspond to image G1 through image matching. That is, the analysis processing unit 32 generates the composite image G12 by adding the pixel values ​​between the portions of image G1 and image G2 that are estimated to correspond to each other through image matching. Therefore, the composite image G12 is represented as a set of sums of the pixel values ​​of image G1 and image G2. The image data of the composite image G12 generated by the analysis processing unit 32 is stored in the storage device 22. Alternatively, the analysis processing unit 32 may generate the composite image G12 by trimming the region of image G1 corresponding to image G2, trimming the region of image G2 corresponding to image G1, and adding the trimmed regions.

[0036] As illustrated in Figure 6, the analysis processing unit 32 detects the presence or absence of the target gas 300 in the observation space 200 by analyzing the composite image G12 (S34). Specifically, the analysis processing unit 32 calculates the concentration of the target gas 300 by performing a predetermined calculation on the composite image G12, and determines the presence or absence of the target gas 300 by comparing this concentration with a threshold. For example, if the concentration exceeds the threshold, the analysis processing unit 32 determines that the target gas 300 is present in the observation space 200, and if the concentration falls below the threshold, it determines that the target gas 300 is not present in the observation space 200.

[0037] To calculate the concentration of the target gas 300, a calculation formula is used that approximately represents the relationship between the pixel value of the composite image G12 and the concentration of the target gas 300. In calculating the concentration of the target gas 300, the average concentration within the area covered by the composite image G12 within the observation space 200 may be calculated, or the concentration may be calculated for each of the multiple regions into which the composite image G12 is divided. The analysis processing unit 32 displays the results of the detection of the target gas 300 on the display device 24 (S35). For example, the presence or absence of the target gas 300 in the observation space 200, the concentration of the target gas 300, or its concentration distribution is displayed on the display device 24.

[0038] Figure 7 is a flowchart of the process executed by the control device 21 (detection processing unit 30) (hereinafter referred to as "detection process"). For example, the detection process is initiated in response to instructions from the user to the operating device 25. Note that the detection process is an example of the "detection method" related to this disclosure.

[0039] When the detection process begins, the control device 21 acquires the images Gk(G1, G2) captured by each infrared camera 12-k of the observation system 10 (S1). That is, the control device 21 acquires image G1 captured by infrared camera 12-1 and image G2 captured by infrared camera 12-2. Images G1 and G2 are, for example, images captured simultaneously. Specifically, the control device 21 acquires the image data (image G1) received by the communication device 23 from infrared camera 12-1 and the image data (image G2) received by the communication device 23 from infrared camera 12-2.

[0040] The control device 21 (image processing unit 31) performs image matching between image G1 and image G2 (S2). The specific procedure for image matching is as described above with reference to Figure 5. After image matching is performed, the control device 21 (analysis processing unit 32) performs analysis processing to detect the target gas 300 (S3). The specific procedure for analysis processing is as described above with reference to Figure 6. As a result of the above operations, the detection result of the target gas 300 (e.g., presence or absence) is displayed on the display device 24 (S35).

[0041] The control device 21 determines whether a predetermined termination condition has been met (S4). The termination condition is, for example, that the termination of the detection process has been instructed by an operation on the operating device 25. If the termination condition is not met (S4: NO), the control device 21 moves the process to step S1. That is, the acquisition of images G1 and G2 (S1), image matching (S2), and analysis processing (S3) are repeated at predetermined intervals until the termination condition is met. The detection process ends when the termination condition is met (S4: YES).

[0042] As described above, in the first embodiment, image matching is performed between image G1 captured by infrared camera 12-1 and image G2 captured by infrared camera 12-2, using image features related to the brightness gradient. Therefore, even when the contour (edge) of the object to be observed is unclear, it is possible to perform image matching between image G1 and image G2 with high accuracy. As a result of high-precision image matching, the object to be observed gas 300 can be detected with high accuracy. In the first embodiment, between image G1 and image G2, parts that are estimated to correspond to each other through image matching are combined. Therefore, the object to be observed can be detected with high accuracy by analyzing the combined image G12. In particular, in the first embodiment, HOG features are used as image features related to the brightness gradient. Therefore, the effect of being able to detect the object to be observed gas 300 in the observation space 200 with high accuracy is particularly remarkable.

[0043] For example, let's assume that the temperature difference between the background imaged by each infrared camera 12-k and the target gas 300 is 10°C. If we assume that the target gas 300 is 25 ppm ammonia, the temperature difference observed depending on the presence or absence of the target gas 300 is approximately 23.6 mK. As described above, the temperature change depending on the presence or absence of the target gas 300 is very small, so it is difficult to detect the presence or absence of the target gas 300 with high accuracy by analyzing images captured by a single infrared camera alone.

[0044] Another method for detecting the target gas 300 with high accuracy is to reduce noise by averaging the time series of images Gk sequentially captured by a single infrared camera. However, an image sensor that receives infrared light using a photodetector made of vanadium oxide, for example, tends to have a longer response time compared to an image sensor that receives visible light. Therefore, noise correlates between images captured sequentially, making it difficult to effectively reduce noise by averaging the images. Furthermore, in order to quickly detect the target gas 300 flowing within the observation space 200, it is necessary to shorten the time period for averaging the images, but if the averaging time period is shortened, the effect of noise reduction by averaging cannot be sufficiently maintained.

[0045] Considering the above circumstances, in the first embodiment, the target gas 300 is detected using the result of image matching between image G1 and image G2. Therefore, even when the temperature difference between the background imaged by the infrared camera 12-k and the target gas 300 is sufficiently small, the influence of noise in each image Gk can be sufficiently reduced compared to a configuration that averages the time series of images Gk. Thus, the target gas 300 can be detected with high accuracy.

[0046] This section explains noise reduction by combining the transformed image G1' (image G1) and image G2. We assume that the pixel value of one pixel in each image Gk fluctuates over time with a fluctuation range S, and the noise intensity associated with image Gk fluctuates over time with a fluctuation range N. By adding the transformed image G1' and image G2, the fluctuation range of the signal component changes to "2S", while the fluctuation range of the noise component changes to "√N". Therefore, the signal-to-noise ratio R of the pixel value is expressed by the following equation (3).

number

[0047] As can be understood from equation (3), when compared with the signal-to-noise ratio R (R=S / N) of image Gk captured by a standalone infrared camera 12-k, the signal-to-noise ratio R of the composite image G12, which is obtained by combining the transformed image G1' and image G2, is improved by √2 times. In other words, according to the first embodiment, the observation sensitivity of the target gas 300 can be improved by √2 times.

[0048] B: Second Embodiment A second embodiment of this disclosure will now be described. For elements whose function is the same as in the first embodiment in each of the embodiments described below, the same reference numerals as in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0049] Figure 8 is a schematic diagram of the observation system 10 in the second embodiment. As illustrated in Figure 8, the observation system 10 in the second embodiment includes four infrared cameras 12-1 to 12-4. The optical axis Lk of each infrared camera 12-k passes through the observation space 200. Furthermore, the optical and electrical characteristics of each infrared camera 12-k are common.

[0050] Infrared cameras 12-1 and 12-2 are arranged with a distance Px between them in the X direction (e.g., horizontal direction), similar to the first embodiment. Similarly, infrared cameras 12-3 and 12-4 are arranged with a distance Px between them in the X direction. Infrared camera 12-3 is an example of a "third infrared camera," and infrared camera 12-4 is an example of a "fourth infrared camera."

[0051] Furthermore, the pairs of infrared cameras 12-1 and 12-2, and the pairs of infrared cameras 12-3 and 12-4 are installed with a distance Py between them in the Y direction. The Y direction is the direction that intersects the X direction, for example, the vertical direction. That is, infrared cameras 12-1 and 12-3 are arranged with a distance Py between them in the Y direction, and infrared cameras 12-2 and 12-4 are arranged with a distance Py between them in the Y direction. The distance Py is set under the same conditions as the distance Px. As described above, in the second embodiment, the multiple infrared cameras 12-1 to 12-4 are arranged in a matrix of 2 rows x 2 columns.

[0052] Figure 9 is an explanatory diagram of the detection process in the second embodiment. As illustrated in Figure 9, the image processing unit 31 performs image matching between image G1 captured by infrared camera 12-1 and image G2 captured by infrared camera 12-2 (S2). The analysis processing unit 32 generates a composite image G12 by combining image G1 and image G2 according to the result of the image matching (S31-S33). Note that composite image G12 is an example of a "first composite image".

[0053] Furthermore, the image processing unit 31 performs image matching between image G3 captured by infrared camera 12-3 and image G4 captured by infrared camera 12-4 (S2). The analysis processing unit 32 generates a composite image G34 by combining image G3 and image G4 according to the results of the image matching (S31-S33). Image G3 is an example of a "third image," and image G4 is an example of a "fourth image." Also, the composite image G34 is an example of a "second composite image."

[0054] The image processing unit 31 performs image matching between the composite image G12 and the composite image G34 (S2). The analysis processing unit 32 generates a composite image Ga by combining the composite image G12 and the composite image G34 according to the results of the image matching (S31-S33). Note that the composite image Ga is an example of a "third composite image".

[0055] The image data of the composite image Ga is stored in the storage device 22. When increasing the total number K of the images Gk to be composited, the calculation can be performed without causing overflow by converting the integer type pixel values ​​to real number type (floating point). In addition, by dividing the pixel value of each pixel in the composite image Ga by the total number K of the images Gk, it is possible to increase the sensitivity of the image Gk while maintaining the bit depth of each pixel value.

[0056] After the above processing is performed, the analysis processing unit 32 detects the target gas 300 by analyzing the composite image Ga (S34). The operation of displaying the detection result on the display device 24 (S35), and the operation of repeating the above processing until the termination condition is met are the same as in the first embodiment.

[0057] The same effects as in the first embodiment are achieved in the second embodiment. In the second embodiment, four images G1 to G4 captured by different infrared cameras 12-k are used to detect the target gas 300. Therefore, compared to a configuration in which only images G1 and G2 are used to detect the target gas 300 (for example, in the first embodiment), the target gas 300 can be detected with higher accuracy.

[0058] The signal-to-noise ratio R of the pixel values ​​when image G1 and image G2 are combined is expressed by the above formula (3). We will now consider the signal-to-noise ratio R when generalized to a total of K images Gk (G1 to GK) to be combined.

[0059] Mean 0 and variance σ 2Assume that noise, represented by the random variable Nk, is associated with the pixel values ​​of image Gk. Each random variable Nk is independent of the others. The noise when K images G1 to GK are added together is expressed by the following equation (4).

number

[0060] The sum of the variances of mutually independent random variables Nk is equal to the variance σ of each random variable Nk. 2 It is expressed as a linear sum of the following: the variance σ of the noise associated with the image obtained by synthesizing K images G1 to GK. K 2 This can be expressed by the following formula (5).

number

[0061] Therefore, the standard deviation σK is expressed by the following formula (6).

number

[0062] The standard deviation of noise corresponds to the root mean square (RMS), or the effective value of the noise. Therefore, the signal-to-noise ratio R of the pixel values ​​in an image obtained by adding K images G1 to GK (e.g., the composite image Ga) is expressed by the following equation (7).

number

[0063] As can be understood from equation (7), compared to the signal-to-noise ratio R (R=S / N) of an image Gk captured by a single infrared camera 12-k, the signal-to-noise ratio R of the composite image Ga, which is created by combining K images G1 to GK, is improved by √N times. In the embodiment where four images G1 to G4 are combined, as illustrated in the second embodiment, the signal-to-noise ratio R can be improved by a factor of 2 by combining images G1 to G4.

[0064] C: Variant The following are examples of specific modifications that may be added to each of the embodiments exemplified above. Two or more embodiments may be arbitrarily selected from the following examples and merged as appropriate, provided they do not contradict each other.

[0065] (1) The total number K of infrared cameras 12-k included in the observation system 10 is arbitrary. For example, the observation system 10 may consist of 3 or 5 or more infrared cameras 12-k. In a configuration in which the observation system 10 includes 5 or more infrared cameras 12-k, the detection processing unit 30 cumulatively repeats image matching and image Gk synthesis for each pair of 2 images Gk selected from K images G1 to GK, and finally detects the target gas 300 by analyzing the generated composite image.

[0066] (2) In the above-described embodiments, HOG features were given as examples of image features related to the luminance gradient, but the image features applied to image matching are not limited to HOG features. For example, image features such as SIFT (Scale-Invariant Feature Transform), GLOH (Gradient Location and Orientation Histogram), or PHOG (Pyramid Histogram of Oriented Gradients) may be applied to image matching. Furthermore, image matching may utilize a process to estimate optical flow using analytical techniques such as the Lucas-Kanade method.

[0067] (3) In the above-described embodiments, the detection of the target gas 300 was used as an example, but the target of detection using the detection system 100 is not limited to gases. For example, the above-described embodiments can be similarly applied to the detection of solids or liquids.

[0068] (4) In each of the above-described embodiments, the examples given are configurations in which the characteristics (optical characteristics and electrical characteristics) of each infrared camera 12-k of the observation system 10 are common. However, the observation system 10 may include multiple infrared cameras 12 having different optical or electrical characteristics. When synthesizing images Gk from each infrared camera 12-k having different optical or electrical characteristics, the weighted sum of pixel values ​​obtained by applying a weighting value corresponding to the characteristic difference of each infrared camera 12-k may be calculated as the pixel value of each pixel in the synthesized image.

[0069] (5) The functions of the analysis system 20 in each of the above-described forms are realized through the cooperation of one or more processors constituting the control device 21 and the program stored in the storage device 22, as described above. The programs exemplified above can be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium is, for example, a non-transitory recording medium, such as an optical recording medium (optical disc) like a CD-ROM, but also includes any known form of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium except for transient propagation signals (transitory, propagating signals), and volatile recording media are not excluded. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the recording medium in which the program is stored in the distribution device corresponds to the non-transitory recording medium described above.

[0070] (6) The notation "the nth" (where n is a natural number) in this application is used solely as a formal and convenient label to distinguish each element in notation, and has no substantive meaning whatsoever. Therefore, there is no room for restrictive interpretation of the position or order of each element based on the notation "the nth".

[0071] D: Addendum From the forms exemplified above, the following configuration can be understood, for example.

[0072] A detection system according to one aspect of the present disclosure (Aspect 1) is a detection system for detecting an object to be observed in an observation space, comprising a plurality of infrared cameras whose optical axes pass through the observation space, and a detection processing unit for detecting the object to be observed, wherein the detection processing unit performs image matching between a first image captured by a first infrared camera among the plurality of infrared cameras and a second image captured by a second infrared camera different from the first infrared camera among the plurality of infrared cameras, using image features relating to the brightness gradient in each of the first and second images, and detects the object to be observed using the result of the image matching. In the above aspect, image matching is performed between the first image captured by the first infrared camera and the second image captured by the second infrared camera, using image features relating to the brightness gradient. Therefore, even when the contour (edge) of the object to be observed is unclear, it is possible to perform image matching between the first image and the second image with high accuracy, and consequently, the object to be observed can be detected with high accuracy.

[0073] In a specific example of Embodiment 1 (Embodiment 2), the detection processing unit generates a composite image by combining a portion of the first image estimated to correspond to the second image through image matching with a portion of the second image estimated to correspond to the first image through image matching, and detects the object of observation by analyzing the composite image. In the above embodiment, portions of the first image and the second image that are estimated to correspond to each other through image matching are combined. Therefore, the object of observation can be detected with high accuracy by analyzing the composite image.

[0074] In the specific example of Embodiment 1 or Embodiment 2 (Embodiment 3), the image feature is a HOG feature. In the above embodiments, a HOG feature is used as an image feature related to the brightness gradient. Therefore, the observed object in the observation space can be detected with high accuracy.

[0075] In any specific example of Embodiments 1 to 3 (Embodiment 4), the object of observation is gas. The outline of the image is unclear for gas. Therefore, image matching using image features related to the brightness gradient is particularly suitable.

[0076] In a specific example of Embodiment 4 (Embodiment 5), each of the plurality of infrared cameras is capable of detecting infrared radiation corresponding to the absorption wavelength of the gas. According to the above embodiment, gases whose absorption wavelength is within the infrared range can be detected with high accuracy.

[0077] In any specific example of Embodiments 1 to 5 (Embodiment 6), the plurality of infrared cameras include a first infrared camera and a second infrared camera arranged at intervals from each other in a first direction, and a third infrared camera and a fourth infrared camera arranged at intervals from each other in the first direction, and the pair of the first infrared camera and the second infrared camera and the pair of the third infrared camera and the fourth infrared camera are installed at intervals from each other in a second direction intersecting the first direction, and the detection processing unit places an image mat between the first image captured by the first infrared camera and the second image captured by the second infrared camera. Image matching is performed, and a first composite image is generated by combining the first and second images according to the result of the image matching. Image matching is performed between the third image captured by the third infrared camera and the fourth image captured by the fourth infrared camera, and a second composite image is generated by combining the third and fourth images according to the result of the image matching. Image matching is performed between the first and second composite images, and a third composite image is generated by combining the first and second composite images according to the result of the image matching. The object to be observed is detected by analyzing the third composite image. According to the above embodiment, the object to be observed can be detected with higher accuracy compared to an embodiment that uses only the first and second images.

[0078] A detection method according to one aspect of the present disclosure (Aspect 7) is a method for detecting an object to be observed in an observation space using a plurality of infrared cameras whose optical axes pass through the observation space, wherein image matching is performed between a first image captured by a first infrared camera among the plurality of infrared cameras and a second image captured by a second infrared camera different from the first infrared camera among the plurality of infrared cameras, using image features relating to the brightness gradient in each of the first and second images, and the object to be observed is detected using the result of the image matching. [Explanation of Symbols]

[0079] 100...Detection system, 200...Observation space, 300...Observation target gas, 10...Observation system, 12-k...Infrared camera, 20...Analysis system, 21...Control device, 22...Storage device, 23...Communication device, 24...Display device, 25...Operation device, 30...Detection processing unit, 31...Image processing unit, 32...Analysis processing unit.

Claims

1. A detection system for detecting an object to be observed in an observation space, Multiple infrared cameras whose optical axes pass through the aforementioned observation space, The system comprises a detection processing unit for detecting the aforementioned object to be observed, The aforementioned detection processing unit, Between the first image captured by the first infrared camera among the plurality of infrared cameras and the second image captured by the second infrared camera, which is different from the first infrared camera among the plurality of infrared cameras, image matching is performed using image features related to the brightness gradient in each of the first and second images. The observed target is detected using the results of the image matching. Detection system.

2. The aforementioned detection processing unit, A composite image is generated by combining the portion of the first image estimated to correspond to the second image through image matching and the portion of the second image estimated to correspond to the first image through image matching, and the observed object is detected by analyzing the composite image. The detection system according to claim 1.

3. The aforementioned image features are HOG features. The detection system according to claim 1.

4. The object of observation is gas. The detection system according to claim 1.

5. Each of the aforementioned infrared cameras is capable of detecting infrared radiation corresponding to the absorption wavelength of the gas. The detection system according to claim 4.

6. The aforementioned plurality of infrared cameras, The first infrared camera and the second infrared camera are arranged in a first direction with a distance between them from each other, It includes a third infrared camera and a fourth infrared camera arranged at intervals from each other in the first direction, The set of the first infrared camera and the second infrared camera, and the set of the third infrared camera and the fourth infrared camera, are installed with a distance between them in a second direction that intersects the first direction. The aforementioned detection processing unit, Image matching is performed between the first image captured by the first infrared camera and the second image captured by the second infrared camera, and a first composite image is generated by combining the first and second images according to the result of the image matching. Image matching is performed between the third image captured by the third infrared camera and the fourth image captured by the fourth infrared camera, and a second composite image is generated by combining the third image and the fourth image according to the result of the image matching. Image matching is performed between the first composite image and the second composite image, and a third composite image is generated by combining the first composite image and the second composite image according to the result of the image matching. The observed target is detected by analyzing the third composite image. The detection system according to claim 1.

7. A method for detecting an object to be observed in an observation space using multiple infrared cameras whose optical axes pass through the observation space, Between the first image captured by the first infrared camera among the plurality of infrared cameras and the second image captured by the second infrared camera, which is different from the first infrared camera among the plurality of infrared cameras, image matching is performed using image features related to the brightness gradient in each of the first and second images. The observed target is detected using the results of the image matching. A detection method implemented by a computer system.

Images