Detection system and detection method

Abstract

A detection system according to the present invention detects an object to be observed in an observation space, and comprises: a plurality of infrared cameras that each have an optical axis passing through the observation space; and a detection processing unit that detects the object to be observed. The detection processing unit: executes image matching between a first image captured by a first infrared camera among the plurality of infrared cameras and a second image captured by a second infrared camera different from the first infrared camera among the plurality of infrared cameras, the image matching using an image feature related to a luminance gradation in each of the first and second images; and uses the result of the image matching to detect the object to be observed.

Description

Detection System and Detection Method

[0001] The present disclosure relates to a technique for detecting an observation target.

[0002] Various techniques for detecting an observation target have been conventionally proposed. For example, Patent Document 1 discloses a technique for generating a pair of left and right thermal images by a pair of left and right infrared cameras, and emphasizing a target location by moving the pair of left and right thermal images in the left-right direction and superimposing the target locations.

[0003] Japanese Patent Application Laid-Open No. 10-332487

[0004] For example, when the observation target is a gas, etc., since the contour (edge) of the observation target is unclear, it is difficult to perform image matching between a plurality of images with high accuracy. In consideration of the above circumstances, one aspect of the present disclosure aims to detect an observation target with high accuracy by performing image matching with high accuracy between a plurality of images captured by different infrared cameras.

[0005] A detection system according to one aspect of the present disclosure is a detection system for detecting an observation target in an observation space, including a first infrared camera and a second infrared camera, a plurality of infrared cameras through which each optical axis passes in the observation space, and a detection processing unit for detecting the observation target. The detection processing unit performs first image matching using image feature amounts related to the luminance gradients in each of the first image and the second image between the first image captured by the first infrared camera and the second image captured by the second infrared camera, and detects the observation target using the result of the first image matching.

[0006] A detection method according to one aspect of the present disclosure is a method for detecting an object to be observed in an observation space using a plurality of infrared cameras, each having an optical axis passing through the observation space, the first of which is performed between a first image captured by the first infrared camera and a second image captured by the second infrared camera, using image features relating to the brightness gradient in each of the first and second images, and the object to be observed is detected using the result of the first image matching.

[0007] This is a block diagram illustrating the configuration of a detection system according to the first embodiment of this disclosure. This is a block diagram illustrating the configuration of an analysis system. This is a block diagram illustrating the functional configuration of an analysis system. This is an explanatory diagram of the processes performed by the analysis system. This is a flowchart of image matching. This is a flowchart of the analysis process. This is a flowchart of the detection process. This is a schematic diagram of the observation system in the second embodiment. This is an explanatory diagram of the detection process in the second embodiment.

[0008] The embodiments for implementing this disclosure will be described with reference to the drawings. Note that the dimensions and scale of the elements in each drawing may differ from those of the actual product. Furthermore, the embodiments described below are illustrative examples of embodiments that may be envisioned when implementing this disclosure. Therefore, the scope of this disclosure is not limited to the embodiments exemplified below.

[0009] A: Figure 1 of the first embodiment is a block diagram illustrating the configuration of the detection system 100 according to the first embodiment of the present disclosure. The detection system 100 is a measurement system for detecting the presence or absence of a gas to be observed (hereinafter referred to as "observation target gas") 300.

[0010] The gas to be observed 300 is a gas that may be present in the observation space 200. The observation space 200 is, for example, the space within a flow path (e.g., a flue) through which the gas to be observed 300, generated in an industrial or chemical process, flows. The wavelength of the absorption line of the gas to be observed 300 (hereinafter referred to as "absorption wavelength") is known. In the first embodiment, it is assumed that the absorption wavelength of the gas to be observed 300 is within the infrared wavelength range (hereinafter referred to as "infrared range"). The type of gas to be observed 300 is arbitrary, but ammonia is an example.

[0011] As illustrated in Figure 1, the detection system 100 comprises an observation system 10 and an analysis system 20. The observation system 10 is a system that optically observes the state of the target gas 300. The analysis system 20 detects the presence or absence of the target gas 300 in the observation space 200 by analyzing the results of the observation by the observation system 10. In Figure 1, the observation system 10 and the analysis system 20 are shown as separate components, but the observation system 10 and the analysis system 20 may be integrated. In other words, the detection system 100 may be realized by multiple devices that are configured separately from each other, or by a single device.

[0012] The observation system 10 comprises a plurality of infrared cameras 12-k (12-1, 12-2). Each infrared camera 12-k is an imaging device that images the observation space 200. Specifically, each infrared camera 12-k comprises an optical system such as a photographic lens and an image sensor that receives incident light from the optical system. The image sensor includes, for example, a light-receiving element made of vanadium oxide. The optical characteristics (e.g., angle of view or focal length) and electrical characteristics (e.g., light-receiving sensitivity or light-receiving wavelength) of the infrared cameras 12-k are common to the plurality of infrared cameras 12-k.

[0013] As illustrated in Figure 1, the optical axis Lk of each of the multiple infrared cameras 12-k passes through the observation space 200. Infrared camera 12-1 is an example of a "first infrared camera," and infrared camera 12-2 is an example of a "second infrared camera."

[0014] Each infrared camera 12-k is capable of detecting infrared light corresponding to the absorption wavelength of the target gas 300. Specifically, each infrared camera 12-k is capable of detecting incident light within the infrared range that includes the absorption wavelength of the target gas 300. In other words, each infrared camera 12-k is sensitive to the absorption wavelength of the target gas 300 in the infrared range. As described above, according to the first embodiment, the target gas 300 whose absorption wavelength is within the infrared range can be detected with high accuracy.

[0015] Figure 1 illustrates the positional relationship between infrared camera 12-1 and infrared camera 12-2. As illustrated in Figure 1, infrared cameras 12-1 and 12-2 are arranged with their optical axes Lk parallel to each other and a distance Px between them in the X direction. The X direction is, for example, the horizontal direction. The distance Px is the distance between the optical axis L1 of infrared camera 12-1 and the optical axis L2 of infrared camera 12-2.

[0016] Infrared cameras 12-1 and 12-2 are installed so that their imaging ranges overlap. Figure 1 shows the distance D between each infrared camera 12-k and the target gas 300, and the field of view θ of each infrared camera 12-k.

[0017] In order for the observation system 10 to observe the target gas 300, the target gas 300 must be located at a position where the imaging ranges of infrared camera 12-1 and infrared camera 12-2 overlap. Therefore, the minimum value of the distance D (hereinafter referred to as the "minimum detection distance") Dmin is expressed by the following formula (1).

[0018] For example, if the field of view θ of each infrared camera 12-k is 60° and the minimum detection distance Dmin is 10 cm, then, as can be understood from equation (1), the distance Px between infrared camera 12-1 and infrared camera 12-2 must be set to 11.5 cm (= 2 / √3) or more.

[0019] The overlap ratio r between the imaging range of infrared camera 12-1 and the imaging range of infrared camera 12-2 is expressed by the following formula (2). As can be understood from equation (2), if it is necessary to maintain an overlap rate r of 60% or more, the distance D between each infrared camera 12-k and the observed gas 300 must be set to 24 cm or more.

[0020] Each infrared camera 12-k captures an image Gk (G1, G2) of the imaging space. Image Gk is a thermal image representing the temperature distribution within the imaging range. Infrared camera 12-1 captures image G1 of the observation space 200. Infrared camera 12-2 captures image G2 of the observation space 200. Image G1 is an example of the "first image," and image G2 is an example of the "second image."

[0021] Each image Gk (G1, G2) is represented by a set of multiple pixel values ​​arranged in a matrix. The pixel value of each pixel is the intensity (i.e., brightness) of infrared light received by the photodetector in the portion of the imaging surface of the image sensor corresponding to that pixel. The acquisition of image G1 by infrared camera 12-1 and the acquisition of image G2 by infrared camera 12-2 are repeated periodically. That is, infrared camera 12-1 generates a time series (i.e., a video) of image G1, and infrared camera 12-2 generates a time series (i.e., a video) of image G2. Infrared camera 12-1 transmits image data representing image G1 to the analysis system 20, and infrared camera 12-2 transmits image data representing image G2 to the analysis system 20.

[0022] The analysis system 20 detects the presence or absence of the target gas 300 in the observation space 200 by analyzing the images Gk (G1, G2) captured by each infrared camera 12-k. Figure 2 is a block diagram illustrating the configuration of the analysis system 20. As illustrated in Figure 2, the analysis system 20 comprises a control device 21, a storage device 22, a communication device 23, a display device 24, and an operating device 25. The analysis system 20 can be implemented as a single device or as multiple devices configured separately from each other.

[0023] The control device 21 consists of one or more processors that control each element of the analysis system 20. Specifically, the control device 21 is composed of one or more types of processors, such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0024] The communication device 23 communicates with each infrared camera 12-k. Specifically, the communication device 23 receives image data representing image G1 from infrared camera 12-1 and image data representing image G2 from infrared camera 12-2.

[0025] The storage device 22 is one or more memories that store the program executed by the control device 21 and the data used by the control device 21. The storage device 22 is composed of a recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 22 may be composed of a combination of multiple types of recording media. Sufficient capacity is reserved in the storage device 22 to store the image data of image G1 and the image data of image G2.

[0026] The display device 24 displays an image under the control of the control device 21. Specifically, the display device 24 displays the detection results for the observed gas 300. The operating device 25 is an input device that receives operations from the user. Note that the display device 24 or operating device 25, which are separate from the analysis system 20, may be connected to the analysis system 20 by wire or wireless connection.

[0027] Figure 3 is a block diagram illustrating the functional configuration of the analysis system 20. Figure 4 is an explanatory diagram of the processes performed by the analysis system 20. As illustrated in Figure 3, the control device 21 realizes a detection processing unit 30 that detects the target gas 300 by executing a program stored in the storage device 22. The detection processing unit 30 includes an image processing unit 31 and an analysis processing unit 32.

[0028] The image processing unit 31 performs image matching between image G1 and image G2. Image matching is a process that analyzes the correspondence between image G1 and image G2. Specifically, as illustrated in Figure 4, image matching is an analytical process (pattern matching) that estimates mutually corresponding feature points F between image G1 and image G2.

[0029] The image processing unit 31 performs image matching between image G1 and image G2 using image features related to the luminance gradient in each of the images G1 and G2. In the first embodiment, the image features applied to image matching are HOG (Histogram of Oriented Gradients) features. HOG features are local image features that represent the gradient direction of pixel values ​​in a local region (cell) of image Gk using a histogram, and are calculated for each local region of image Gk. In the first embodiment, the observed gas 300, which is the target of detection, has an unclear outline in image Gk. That is, it is difficult to extract the outline (edge) from image Gk with high accuracy. Therefore, as described above, image matching using image features related to the luminance gradient is particularly suitable.

[0030] Figure 5 is a flowchart of the image matching process performed by the image processing unit 31. When image matching is initiated, the image processing unit 31 calculates HOG features for each local region of image G1 (S21). The image processing unit 31 also calculates HOG features for each local region of image G2 (S22). By comparing the HOG features of each local region between image G1 and image G2, the image processing unit 31 estimates the mutually corresponding feature points F between image G1 and image G2 (S23). The image processing unit 31 may perform various image processing operations (e.g., cropping or filtering) on ​​image G1 and image G2 before performing image matching.

[0031] The analysis processing unit 32 in Figure 3 detects the target gas 300 using the results of image matching by the image processing unit 31. Specifically, as illustrated in Figure 4, the analysis processing unit 32 generates a composite image G12 by combining image G1 and image G2 using the results of image matching, and detects the target gas 300 by analyzing the composite image G12. As described above, the detection processing unit 30 performs image matching between image G1 and image G2 using image features (specifically HOG features) related to the brightness gradient in each of image G1 and image G2, and detects the target gas 300 using the results of image matching.

[0032] Figure 6 is a flowchart of the process performed by the analysis processing unit 32 (hereinafter referred to as "analysis processing"). When the analysis processing starts, the analysis processing unit 32 calculates a transformation matrix A to map image G1 to image G2 (S31). Transformation matrix A is a homography matrix that represents the projection transformation (e.g., scaling, translation, or rotation) applied to image G1. Specifically, as illustrated in Figure 4, the analysis processing unit 32 calculates transformation matrix A such that the feature points F of image G1 and the feature points F of image G2, estimated by image matching, match each other.

[0033] The analysis processing unit 32 generates a transformed image G1' by transforming (i.e., projectively transforming) image G1 using the transformation matrix A (S32). Specifically, as illustrated in Figure 4, the transformed image G1' is generated by multiplying image G1 by the transformation matrix A.

[0034] The analysis processing unit 32 generates a composite image G12 by combining the transformed image G1' and image G2 (S33). The pixel value of each pixel constituting the composite image G12 (hereinafter referred to as the "pixel of interest") is calculated by adding the pixel value of the pixel corresponding to the pixel of interest in the transformed image G1' and the pixel value of the pixel corresponding to the pixel of interest in image G2.

[0035] As described above, the analysis processing unit 32 generates a composite image G12 by combining a portion of image G1 estimated to correspond to image G2 through image matching (first portion) and a portion of image G2 estimated to correspond to image G1 through image matching (second portion). That is, the analysis processing unit 32 generates the composite image G12 by adding the pixel values ​​between the portions of image G1 and image G2 that are estimated to correspond to each other through image matching. Therefore, the composite image G12 is represented as a set of sums of the pixel values ​​of image G1 and image G2. The image data of the composite image G12 generated by the analysis processing unit 32 is stored in the storage device 22. Alternatively, the analysis processing unit 32 may generate the composite image G12 by trimming the region of image G1 corresponding to image G2, trimming the region of image G2 corresponding to image G1, and adding the trimmed regions.

[0036] As illustrated in Figure 6, the analysis processing unit 32 detects the presence or absence of the target gas 300 in the observation space 200 by analyzing the composite image G12 (S34). Specifically, the analysis processing unit 32 calculates the concentration of the target gas 300 by performing a predetermined calculation on the composite image G12, and determines the presence or absence of the target gas 300 by comparing the concentration with a threshold. For example, the analysis processing unit 32 determines that the target gas 300 is present in the observation space 200 if the concentration exceeds the threshold, and determines that the target gas 300 is not present in the observation space 200 if the concentration falls below the threshold.

[0037] To calculate the concentration of the target gas 300, a calculation formula is used that approximately represents the relationship between the pixel value of the composite image G12 and the concentration of the target gas 300. In calculating the concentration of the target gas 300, the average concentration within the range targeted by the composite image G12 in the observation space 200 may be calculated, or the concentration may be calculated for each of the multiple regions into which the composite image G12 is divided. The analysis processing unit 32 displays the results of the detection of the target gas 300 on the display device 24 (S35). For example, the presence or absence of the target gas 300 in the observation space 200, the concentration of the target gas 300, or the concentration distribution may be displayed on the display device 24.

[0038] FIG. 7 is a flowchart of a process (hereinafter referred to as “detection process”) executed by the control device 21 (detection processing unit 30). For example, the detection process is started契机 by an instruction from the user to the operation device 25. Note that the detection process is an example of the “detection method” according to the present disclosure.

[0039] When the detection process is started, the control device 21 acquires the images Gk (G1, G2) captured by each infrared camera 12-k of the observation system 10 (S1). That is, the control device 21 acquires the image G1 captured by the infrared camera 12-1 and the image G2 captured by the infrared camera 12-2. The image G1 and the image G2 are, for example, images captured simultaneously. Specifically, the control device 21 acquires the image data (image G1) received by the communication device 23 from the infrared camera 12-1 and the image data (image G2) received by the communication device 23 from the infrared camera 12-2.

[0040] The control device 21 (image processing unit 31) performs image matching between the images G1 and G2 (S2). The specific procedure of image matching is as described above with reference to FIG. 5. When the image matching is performed, the control device 21 (analysis processing unit 32) performs an analysis process for detecting the observation target gas 300 (S3). The specific procedure of the analysis process is as described above with reference to FIG. 6. Through the above operations, the result (for example, presence or absence) of detecting the observation target gas 300 is displayed on the display device 24 (S35).

[0041] The control device 21 determines whether a predetermined end condition is satisfied (S4). The end condition is, for example, that the end of the detection process is instructed by an operation on the operation device 25. If the end condition is not satisfied (S4: NO), the control device 21 transfers the process to step S1. That is, the acquisition of the images G1 and G2 (S1), the image matching (S2), and the analysis process (S3) are repeated at a predetermined cycle until the end condition is satisfied. The detection process ends when the end condition is satisfied (S4: YES).

[0042] As described above, in the first embodiment, image matching is performed between image G1 captured by infrared camera 12-1 and image G2 captured by infrared camera 12-2, using image features related to the brightness gradient. Therefore, even when the contour (edge) of the object to be observed is unclear, it is possible to perform image matching between image G1 and image G2 with high accuracy. As a result of high-precision image matching, the object to be observed gas 300 can be detected with high accuracy. In the first embodiment, between image G1 and image G2, parts that are estimated to correspond to each other through image matching are combined. Therefore, the object to be observed can be detected with high accuracy by analyzing the combined image G12. In particular, in the first embodiment, HOG features are used as image features related to the brightness gradient. Therefore, the effect of being able to detect the object to be observed gas 300 in the observation space 200 with high accuracy is particularly remarkable.

[0043] For example, let's assume that the temperature difference between the background imaged by each infrared camera 12-k and the target gas 300 is 10°C. If we assume that the target gas 300 is 25 ppm ammonia, the temperature difference observed depending on the presence or absence of the target gas 300 is approximately 23.6 mK. As described above, the temperature change depending on the presence or absence of the target gas 300 is very small, so it is difficult to detect the presence or absence of the target gas 300 with high accuracy by analyzing images captured by a single infrared camera alone.

[0044] Another method for detecting the target gas 300 with high accuracy is to reduce noise by averaging the time series of images Gk sequentially captured by a single infrared camera. However, an image sensor that receives infrared light using a photodetector made of vanadium oxide, for example, tends to have a longer response time compared to an image sensor that receives visible light. Therefore, noise correlates between images captured sequentially, making it difficult to effectively reduce noise by averaging the images. Furthermore, in order to quickly detect the target gas 300 flowing within the observation space 200, it is necessary to shorten the time period for averaging the images, but if the averaging time period is shortened, the effect of reducing noise by averaging cannot be sufficiently maintained.

[0045] In consideration of the above circumstances, in the first embodiment, the target gas 300 to be observed is detected by using the result of image matching between the image G1 and the image G2. Therefore, even in a situation where the temperature difference between the background imaged by the infrared camera 12-k and the target gas 300 to be observed is sufficiently small, the influence of the noise of each image Gk can be sufficiently reduced as compared with the configuration of averaging the time series of the images Gk. Therefore, the target gas 300 to be observed can be detected with high accuracy.

[0046] The reduction of noise by synthesizing the converted image G1' (image G1) and the image G2 will be described. Now, assume a case where the pixel value of one pixel of each image Gk varies over time by the fluctuation width S, and the intensity of the noise associated with the image Gk varies over time by the fluctuation width N. By adding the converted image G1' and the image G, the fluctuation width of the signal component changes to "2S", while the fluctuation width of the noise component changes to "√N". Therefore, the signal-to-noise ratio R of the pixel value is expressed by the following mathematical formula (3).

[0047] As can be understood from the mathematical formula (3), the signal-to-noise ratio R of the composite image G12 obtained by synthesizing the converted image G1' and the image G2 is improved by √2 times as compared with the signal-to-noise ratio R (R = S / N) of the image Gk captured by a single infrared camera 12-k. That is, according to the first embodiment, the observation sensitivity of the target gas 300 can be improved by √2 times.

[0048] B: Second Embodiment The second embodiment of the present disclosure will be described. For elements whose functions are the same as those in the first embodiment in each aspect exemplified below, the same reference numerals as those in the description of the first embodiment are used, and the detailed description of each is omitted as appropriate.

[0049] FIG. 8 is a schematic diagram of the observation system 10 in the second embodiment. As illustrated in FIG. 8, the observation system 10 of the second embodiment includes four infrared cameras 12-1 to 12-4. The optical axis Lk of each infrared camera 12-k passes through the observation space 200. In addition, the optical characteristics and electrical characteristics of each infrared camera 12-k are common.

[0050] Infrared cameras 12-1 and 12-2 are arranged with a distance Px between them in the X direction (for example, the horizontal direction), similar to the first embodiment. Similarly, infrared cameras 12-3 and 12-4 are arranged with a distance Px between them in the X direction. Infrared camera 12-3 is an example of a "third infrared camera," and infrared camera 12-4 is an example of a "fourth infrared camera."

[0051] Furthermore, the pair of infrared cameras 12-1 and 12-2, and the pair of infrared cameras 12-3 and 12-4 are installed with a distance Py between them in the Y direction. The Y direction is the direction that intersects the X direction, for example, the vertical direction. That is, infrared cameras 12-1 and 12-3 are arranged with a distance Py between them in the Y direction, and infrared cameras 12-2 and 12-4 are arranged with a distance Py between them in the Y direction. The distance Py is set under the same conditions as the distance Px. As described above, in the second embodiment, the multiple infrared cameras 12-1 to 12-4 are arranged in a matrix of 2 rows x 2 columns.

[0052] Figure 9 is an explanatory diagram of the detection process in the second embodiment. As illustrated in Figure 9, the image processing unit 31 performs a first image matching between image G1 captured by infrared camera 12-1 and image G2 captured by infrared camera 12-2 (S2). The analysis processing unit 32 generates a composite image G12 by combining image G1 and image G2 according to the result of the first image matching (S31 to S33). Note that composite image G12 is an example of a "first composite image".

[0053] Furthermore, the image processing unit 31 performs a second image matching between image G3 captured by infrared camera 12-3 and image G4 captured by infrared camera 12-4 (S2). The analysis processing unit 32 generates a composite image G34 by combining image G3 and image G4 according to the result of the second image matching (S31-S33). Image G3 is an example of a "third image," and image G4 is an example of a "fourth image." Also, the composite image G34 is an example of a "second composite image."

[0054] The image processing unit 31 performs a third image matching between the composite image G12 and the composite image G34 (S2). The analysis processing unit 32 generates a composite image Ga by combining the composite image G12 and the composite image G34 according to the result of the third image matching (S31-S33). Note that the composite image Ga is an example of a "third composite image".

[0055] The image data of the composite image Ga is stored in the storage device 22. When increasing the total number K of images Gk to be composited, the integer type pixel values ​​can be converted to real number type (floating point) to perform calculations without causing overflow. Furthermore, by dividing the pixel value of each pixel in the composite image Ga by the total number K of images Gk, it is possible to increase the sensitivity of image Gk while maintaining the bit depth of each pixel value.

[0056] After executing the above process, the analysis processing unit 32 detects the target gas 300 by analyzing the synthesized image Ga (S34). The operation of displaying the detection result on the display device 24 (S35), and the operation of repeating the above process until the termination condition is met, are the same as in the first embodiment.

[0057] The same effects as in the first embodiment are achieved in the second embodiment. In the second embodiment, four images G1 to G4 captured by different infrared cameras 12-k are used to detect the target gas 300. Therefore, compared to a configuration in which only images G1 and G2 are used to detect the target gas 300 (for example, in the first embodiment), the target gas 300 can be detected with higher accuracy.

[0058] The signal-to-noise ratio R of the pixel values ​​when image G1 and image G2 are combined is expressed by the above formula (3). We will now consider the signal-to-noise ratio R when generalized to a total of K images Gk (G1 to GK) to be combined.

[0059] Mean 0 and variance σ 2 Assume that noise, represented by the random variable Nk, is associated with the pixel values ​​of image Gk. Each random variable Nk is independent of the others. The noise when K images G1 to GK are added together is expressed by the following equation (4).

[0060] The sum of the variances of mutually independent random variables Nk is equal to the variance σ of each random variable Nk. 2 It is expressed as a linear sum of the following: the variance σ of the noise associated with the image obtained by synthesizing K images G1 to GK. K 2 This can be expressed by the following formula (5).

[0061] Therefore, the standard deviation σK is expressed by the following formula (6).

[0062] The standard deviation of noise corresponds to the root mean square (RMS), i.e., the effective value of the noise. Therefore, the signal-to-noise ratio R of the pixel values ​​in an image obtained by adding K images G1 to GK (for example, the composite image Ga) is expressed by the following equation (7).

[0063] As can be understood from equation (7), compared to the signal-to-noise ratio R (R = S / N) of an image Gk captured by a single infrared camera 12-k, the signal-to-noise ratio R of the composite image Ga, which is created by combining K images G1 to GK, is improved by √N times. In the embodiment where four images G1 to G4 are combined, as illustrated in the second embodiment, the signal-to-noise ratio R can be improved by a factor of 2 by combining images G1 to G4.

[0064] C: Examples of specific modifications that may be added to each of the embodiments exemplified above are given below. Two or more embodiments may be arbitrarily selected from the following examples and merged as appropriate, provided they do not contradict each other.

[0065] (1) The total number K of infrared cameras 12-k included in the observation system 10 is arbitrary. For example, the observation system 10 may be composed of three or five or more infrared cameras 12-k. In a configuration in which the observation system 10 includes five or more infrared cameras 12-k, the detection processing unit 30 cumulatively repeats image matching and image Gk synthesis for each pair of two images Gk selected from K images G1 to GK, and finally detects the target gas 300 by analyzing the generated composite image.

[0066] (2) In each of the above embodiments, HOG features were given as examples of image features related to the luminance gradient, but the image features that can be applied to image matching are not limited to HOG features. For example, image features such as SIFT (Scale-Invariant Feature Transform), GLOH (Gradient Location and Orientation Histogram), or PHOG (Pyramid Histogram of Oriented Gradients) may be applied to image matching. Furthermore, for image matching, a process that estimates optical flow using an analysis technique such as the Lucas-Kanade method may be used.

[0067] (3) In the above-described embodiments, the detection of the target gas 300 was used as an example, but the target of detection using the detection system 100 is not limited to gases. For example, the above-described embodiments can be similarly applied to the detection of solids or liquids.

[0068] (4) In each of the above-described embodiments, the examples given are configurations in which the characteristics (optical characteristics and electrical characteristics) of each infrared camera 12-k of the observation system 10 are common. However, the observation system 10 may include multiple infrared cameras 12 having different optical or electrical characteristics. When synthesizing images Gk from each infrared camera 12-k having different optical or electrical characteristics, the weighted sum of pixel values ​​obtained by applying a weighting value corresponding to the difference in characteristics of each infrared camera 12-k may be calculated as the pixel value of each pixel in the synthesized image.

[0069] (5) The functions of the analysis system 20 in each of the above-described forms are realized through the cooperation of one or more processors constituting the control device 21 and the program stored in the storage device 22, as described above. The programs exemplified above can be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium is, for example, a non-transitory recording medium, and optical recording media such as CD-ROMs (optical discs) are good examples, but any known form of recording medium such as semiconductor recording media or magnetic recording media is also included. Note that a non-transitory recording medium includes any recording medium except for transient propagation signals (transitory, propagating signals), and volatile recording media are not excluded. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the recording medium in which the program is stored in the distribution device corresponds to the non-transitory recording medium described above.

[0070] (6) The notation "the nth" (where n is a natural number) in this application is used solely as a formal and convenient label to distinguish each element in notation, and has no substantive meaning whatsoever. Therefore, there is no room for restrictive interpretation of the position or order of each element based on the notation "the nth".

[0071] D: From the forms exemplified above, the following configurations can be understood, for example.

[0072] A detection system according to one aspect of the present disclosure (Aspect 1) is a detection system for detecting an object to be observed in an observation space, comprising a plurality of infrared cameras including a first infrared camera and a second infrared camera, each having an optical axis passing through the observation space, and a detection processing unit for detecting the object to be observed, wherein the detection processing unit performs first image matching between a first image captured by the first infrared camera and a second image captured by the second infrared camera, using image features relating to the brightness gradient in each of the first and second images, and detects the object to be observed using the result of the first image matching. In the above aspect, first image matching is performed between a first image captured by the first infrared camera and a second image captured by the second infrared camera, using image features relating to the brightness gradient. Therefore, even when the contour (edge) of the object to be observed is unclear, it is possible to perform first image matching between the first image and the second image with high accuracy, and consequently, the object to be observed can be detected with high accuracy.

[0073] In a specific example of Embodiment 1 (Embodiment 2), the detection processing unit generates a composite image by combining a first portion of the first image that is estimated to correspond to the second image through first image matching and a second portion of the second image that is estimated to correspond to the first image through first image matching, and detects the object of observation by analyzing the composite image. In the above embodiment, portions (first portion and second portion) that are estimated to correspond to each other through first image matching are combined between the first image and the second image. Therefore, the object of observation can be detected with high accuracy by analyzing the composite image.

[0074] In the specific example of Embodiment 1 or Embodiment 2 (Embodiment 3), the image feature is a HOG feature. In the above embodiments, a HOG feature is used as an image feature related to the brightness gradient. Therefore, the object of observation in the observation space can be detected with high accuracy.

[0075] In any specific example of Embodiments 1 to 3 (Embodiment 4), the object of observation is gas. The outline of the image is unclear for gas. Therefore, image matching using image features related to the brightness gradient is particularly suitable.

[0076] In a specific example of Embodiment 4 (Embodiment 5), each of the plurality of infrared cameras is capable of detecting infrared radiation corresponding to the absorption wavelength of the gas. According to the above embodiment, gases whose absorption wavelength is within the infrared range can be detected with high accuracy.

[0077] In any specific example of Embodiments 1 to 5 (Embodiment 6), the first infrared camera and the second infrared camera are arranged spaced apart from each other in a first direction, the plurality of infrared cameras further include a third infrared camera and a fourth infrared camera arranged spaced apart from each other in the first direction, the pair of the first infrared camera and the second infrared camera and the pair of the third infrared camera and the fourth infrared camera are installed spaced apart from each other in a second direction intersecting the first direction, and the detection processing unit performs a first image match between the first image captured by the first infrared camera and the second image captured by the second infrared camera. Image matching is performed, and a first composite image is generated by combining the first and second images according to the result of the first image matching. A second image matching is performed between the third image captured by the third infrared camera and the fourth image captured by the fourth infrared camera, and a second composite image is generated by combining the third and fourth images according to the result of the second image matching. A third image matching is performed between the first composite image and the second composite image, and a third composite image is generated by combining the first and second composite images according to the result of the third image matching. The object of observation is detected by analyzing the third composite image. According to the above embodiment, the object of observation can be detected with higher accuracy compared to an embodiment that uses only the first and second images.

[0078] A detection method according to one aspect of the present disclosure (Aspect 7) is a method for detecting an object to be observed in an observation space using a plurality of infrared cameras, each having an optical axis passing through the observation space, the first of which is performed between a first image captured by the first infrared camera and a second image captured by the second infrared camera, using image features relating to the brightness gradient in each of the first and second images, and the object to be observed is detected using the result of the first image matching.

[0079] 100...Detection system, 200...Observation space, 300...Observation target gas, 10...Observation system, 12-k...Infrared camera, 20...Analysis system, 21...Control device, 22...Storage device, 23...Communication device, 24...Display device, 25...Operation device, 30...Detection processing unit, 31...Image processing unit, 32...Analysis processing unit.

Claims

1. A detection system for detecting an object to be observed in an observation space, comprising a plurality of infrared cameras, each having an optical axis passing through the observation space, including a first infrared camera and a second infrared camera, and a detection processing unit for detecting the object to be observed, wherein the detection processing unit performs first image matching between a first image captured by the first infrared camera and a second image captured by the second infrared camera, using image features relating to the brightness gradient in each of the first and second images, and detects the object to be observed using the result of the first image matching.

2. The detection system according to claim 1, wherein the detection processing unit generates a composite image by combining a first portion of the first image that is estimated to correspond to the second image by first image matching and a second portion of the second image that is estimated to correspond to the first image by first image matching, and detects the object to be observed by analyzing the composite image.

3. The detection system according to claim 1, wherein the image features are HOG features.

4. The detection system according to claim 1, wherein the object to be observed is a gas.

5. The detection system according to claim 4, wherein each of the plurality of infrared cameras is capable of detecting infrared radiation corresponding to the absorption wavelength of the gas.

6. The first infrared camera and the second infrared camera are arranged spaced apart from each other in a first direction, and the plurality of infrared cameras further include a third infrared camera and a fourth infrared camera arranged spaced apart from each other in the first direction, and the pair of the first infrared camera and the second infrared camera and the pair of the third infrared camera and the fourth infrared camera are installed spaced apart from each other in a second direction intersecting the first direction, and the detection processing unit performs first image matching between the first image captured by the first infrared camera and the second image captured by the second infrared camera, generates a first composite image by combining the first image and the second image according to the result of the first image matching, performs second image matching between the third image captured by the third infrared camera and the fourth image captured by the fourth infrared camera, generates a second composite image by combining the third image and the fourth image according to the result of the second image matching, The detection system according to claim 1, comprising: performing a third image matching between the first composite image and the second composite image; generating a third composite image by combining the first composite image and the second composite image according to the result of the third image matching; and detecting the object to be observed by analyzing the third composite image.

7. A method for detecting an object to be observed in an observation space using a plurality of infrared cameras, including a first infrared camera and a second infrared camera, each having an optical axis passing through the observation space, wherein a first image matching is performed between a first image captured by the first infrared camera and a second image captured by the second infrared camera, using image features relating to the brightness gradient in each of the first and second images, and the object to be observed is detected using the results of the first image matching, and the detection method is implemented by a computer system.

Images