Gas leak detection device, gas leak detection method, and gas leak detection program

Abstract

The gas leak detection device acquires a plurality of pieces of image data obtained by photographing a field in time series, and evaluates a brightness change for each pixel included in the image data by comparing the brightness of each pixel between two pieces of image data. A luminance change frequency distribution in the image data is calculated by integrating luminance changes for each pixel. The leaked gas in the field is detected on the basis of the brightness change frequency distribution thus calculated.

Description

Technical Field

[0001] The invention relates to a gas leakage detection device, a gas leakage detection method and a gas leakage detection program.

[0002] This application claims priority based on Japanese Patent Application No. 2022-154745 filed with the Japan Patent Office on September 28, 2022, and the contents of which are incorporated herein by reference. Background Art

[0003] For example, there is a known technology for detecting the leakage of gas such as CO2. In the case where the site to be detected is relatively narrow, detection can be performed by configuring a sensor capable of detecting gas within the range where the gas may leak. On the other hand, in the case where the site to be detected is relatively wide, it is difficult to deal with the use of a sensor for detecting local gas leakage. In this case, it is considered to detect gas leakage in the site by photographing the site with a camera or other photographic device that can capture the wavelength corresponding to the gas to be detected, and visualizing the leaked gas.

[0004] In the present application, the term “detection” includes “detection” and “monitoring”. In the case of detecting a gas leak, it includes not only the presence or absence of a gas leak but also the acquisition of various information related to the gas leak.

[0005] In order to detect gas leakage at the scene based on the image thus taken, it is necessary to distinguish the background of the scene (e.g., the ground or the wall of a building, etc.) from the foreground including the gas to be detected. For example, although Patent Document 1 is not a technology related to gas leakage detection, it discloses a technology in which, for the gray value of each pixel constituting a photographic image, a pixel classification formula using the Bayesian estimation method is calculated as the background generation probability and the foreground generation probability, respectively, so that each pixel can be classified as the background or the foreground.

[0006] Previous technical literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2010-97507 Summary of the invention

[0009] Technical issues to be solved by the invention

[0010] As described above, gas leakage at a site can be detected based on image data obtained by photographing the site. For example, when CO2 is handled as a gas to be detected, CO2 gas leakage can be detected based on a momentary photographic image (so-called snapshot) obtained by photographing the site using an infrared camera that can capture CO2 gas. However, depending on the temperature or background of the site, it is sometimes difficult to reliably detect gas leakage using a single photographic image.

[0011] For example, an infrared camera is equipped with a detection element for detecting infrared rays. In this detection element, the intensity of infrared rays incident on the infrared camera is detected, and after the detection result is converted into an electrical signal, a photographic image is constructed based on the distribution of brightness values ​​corresponding to the signal intensity. In this infrared camera, the detection element detects electromagnetic waves radiated from the background of the scene (such as the ground or the wall of a building, etc.) and electromagnetic waves whose energy is absorbed by the detection target gas (CO2 gas) existing between the background of the scene and the infrared camera, and visualizes the area where the detection target gas exists based on the difference in the intensity of the electromagnetic waves.

[0012] Here, the intensity of the electromagnetic wave radiated from the background object changes according to the temperature of the background object, so the greater the temperature difference between the background object and the detection target gas, the greater the difference between the intensity of the electromagnetic wave radiated from the background object and the intensity of the electromagnetic wave radiated from the detection target gas, and the detection target gas can be sensitively separated from the background object. On the other hand, when the temperature difference between the background object and the detection target gas is small, the difference between the intensity of the electromagnetic wave radiated from the background object and the intensity of the electromagnetic wave radiated from the detection target gas is small, and it is difficult to separate the detection target gas from the background object (that is, the difference in the brightness value between the background object and the detection target gas in the photographic image is small, and the outline of the range where the detection target gas is located becomes blurred). For example, when an outdoor scene is used as the detection object, soil, sand, grass, etc. are included as background objects in the photographic image, and their temperatures change according to meteorological conditions such as sunshine, wind and rain, and temperature. Therefore, it is also considered that the temperature difference between the background object and the detection target gas becomes smaller according to meteorological conditions.

[0013] At least one embodiment of the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a gas leak detection device, a gas leak detection method, and a gas leak detection program that can detect gas leaks with high accuracy regardless of the conditions at the site.

[0014] Means for solving technical problems

[0015] In order to solve the above-mentioned problems, a gas leakage detection device according to at least one embodiment of the present invention includes:

[0016] An image data acquisition unit for acquiring a plurality of image data obtained by photographing the scene in time series;

[0017] a brightness change evaluation unit for evaluating brightness changes for each pixel included in the image data by comparing brightness of each pixel between two of the image data that are consecutive in time series among the plurality of image data;

[0018] a brightness change frequency distribution calculation unit configured to calculate a brightness change frequency distribution in the image data by accumulating the brightness change for each of the pixels; and

[0019] A gas detection unit is used to detect gas leakage in the scene based on the brightness change frequency distribution.

[0020] In order to solve the above-mentioned problems, a gas leakage detection method according to at least one embodiment of the present invention includes the following steps:

[0021] A step of acquiring a plurality of image data obtained by photographing the scene in time series;

[0022] A step of evaluating a brightness change for each pixel included in the image data by comparing the brightness of each pixel between two image data that are consecutive in time series among the plurality of image data;

[0023] a step of calculating a frequency distribution of brightness changes in the image data by accumulating the brightness changes for each of the pixels; and

[0024] A step of detecting gas leakage in the scene based on the brightness change frequency distribution.

[0025] In order to solve the above-mentioned problems, a gas leak detection program according to at least one embodiment of the present invention can execute the following steps using a computer device:

[0026] A step of acquiring a plurality of image data obtained by photographing the scene in time series;

[0027] A step of evaluating a brightness change for each pixel included in the image data by comparing the brightness of each pixel between two image data that are consecutive in time series among the plurality of image data;

[0028] a step of calculating a frequency distribution of brightness changes in the image data by accumulating the brightness changes for each of the pixels; and

[0029] A step of detecting gas leakage in the scene based on the brightness change frequency distribution.

[0030] Effects of the Invention

[0031] According to at least one embodiment of the present invention, it is possible to provide a gas leak detection device, a gas leak detection method, and a gas leak detection program that can detect gas leaks with high accuracy regardless of the conditions at the site. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram showing, from the side, a UAV equipped with a photographing device that cooperates with the gas leak detection device according to at least one embodiment of the present invention.

[0033] Figure 2 A gas leakage detection device according to an embodiment of the present invention is combined with Figure 1 The structure block diagram of the UAV is shown together.

[0034] Figure 3 1 is a flowchart showing a gas leak detection method according to one embodiment.

[0035] Figure 4 Yes means Figure 3 Schematic diagram of the method for calculating the brightness change frequency distribution in step S104.

[0036] Figure 5 Yes means Figure 3 Schematic diagram of the normalization process in step S105.

[0037] Figure 6 is Figure 3 An example of a brightness change frequency distribution used in step S106.

[0038] Figure 7 1 is a flowchart showing a gas leak detection method according to another embodiment.

[0039] Figure 8 It means in Figure 7 Schematic diagram of the binarization process performed in step S204.

[0040] Fig. 9 It is a block diagram of a gas leak detection device according to another embodiment.

[0041] Fig.10 1 is a flowchart showing a gas leak detection method according to another embodiment.

[0042] Fig.11 is Fig.10 Sub-flow chart of the alignment processing implemented in step S302.

[0043] Fig.12 Yes means confirmation Fig.11Schematic diagram of the state of the feature points in step S402. DETAILED DESCRIPTION

[0044] Hereinafter, several embodiments of the present invention will be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of the structures described or illustrated as the embodiments are not intended to limit the scope of the present invention, but are merely illustrative examples.

[0045] Figure 1 1 is a schematic diagram showing a UAV (Unmanned Aerial Vehicle) 10 equipped with a photographing device 16 cooperating with a gas leakage detection device according to at least one embodiment of the present invention from the side. The gas leakage detection device according to at least one embodiment is a device for detecting leakage of gas G in a site F (for example, leakage from the ground or facilities included in the site F to the atmosphere, etc.). In particular, the gas leakage detection device 100 detects gas leakage in the site F by analyzing image data obtained by photographing the site F by the photographing device. In the embodiments described below, the case where the photographing device for photographing image data is mounted on the UAV 10 (Unmanned Aerial Vehicle) is exemplified, but the photographing device may also be mounted on other mobile objects, or may be mounted on non-mobile objects such as the ground or buildings.

[0046] The UAV 10 includes a UAV body 12, a plurality of propellers 14 (usually, three or more propellers 14) mounted on the UAV body 12, and a camera 16. The plurality of propellers 14 are each configured to be rotationally driven by a motor (not shown). The flight altitude, flight speed, flight direction, etc. of the UAV 10 can be controlled by adjusting the current value of each motor used to drive the plurality of propellers 14. Figure 2 As described later, the signal indicating the current value of each motor can be sent to the gas leakage detection device 100 via the wireless communication network 40.

[0047] The photographing device 16 is an imaging device having a filter that selectively transmits wavelengths absorbed by the gas to be detected. For example, when CO2 gas is set as the detection object, the photographing device 16 is an infrared camera having a filter that selectively transmits infrared light with a wavelength of 4.3 μm absorbed by CO2 gas. In this way, by using the photographing device 16 that selectively filters a specific wavelength to shoot, only the specific gas in the fluid (gas) in the photographing field of view can be photographed as image data. As shown in FIG. Figure 2 In this way, the image data obtained by the imaging device 16 is transmitted to the gas leakage detection device 100 via the wireless communication network 40 .

[0048] In addition, the gas to be detected is not limited to CO2 gas, and may be other gases that can absorb infrared rays, such as methane or ammonia. Also, photography may be performed using a photographing device 16 having an absorption wavelength region other than the infrared region, and in this case, any gas having an absorption wavelength region that can be photographed by the photographing device 16 may be used as the detection object. In the following description, the detection object is gas, but, for example, a medium such as smoke containing particles may also be used as the detection object.

[0049] Furthermore, the photographing device 16 is configured to acquire a plurality of image data by photographing the scene F in a time series. For example, the photographing device 16 may be a still camera that acquires a plurality of image data by repeatedly acquiring still images in a time series, or may be an animation camera that acquires an animation composed of a plurality of frame images that are continuous in a time series corresponding to a predetermined frame frequency (in the animation camera, the plurality of frame images that constitute the animation are processed as a plurality of image data).

[0050] The photographing device 16 is mounted on the UAV body 12 so as to be able to photograph the scene F from the UAV 10 in flight. The mounting posture of the photographing device 16 relative to the UAV body 12 may be fixed or variable. In this way, the photographing device 16 can be mounted on the UAV 10 as a mobile body and can move on the scene F together with the UAV 10, thereby being able to detect gas leaks within any range of the scene F.

[0051] Next, the structure of the gas leak detection device 100 that functions in cooperation with the UAV 10 having the above structure will be described. Figure 2 A gas leakage detection device 100 according to an embodiment is combined with Figure 1 FIG. 1 is a block diagram of the structure of the UAV 10 shown together.

[0052] The gas leakage detection device 100 is composed of, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and a computer-readable storage medium. In addition, a series of processes for realizing various functions are, for example, stored in a storage medium in the form of a program, and the CPU reads the program into the RAM, and performs information processing and calculation processing to realize various functions. In addition, the program can also be applied in a manner of being pre-installed in a ROM or other storage medium, provided in a state of being stored in a computer-readable storage medium, or transmitted via a wired or wireless communication means. A computer-readable storage medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and the like.

[0053] The gas leak detection device 100 is implemented by executing a program stored in a storage medium or the like. Figure 2 Specifically, the gas leakage detection device 100 includes an image data acquisition unit 102 , a brightness change evaluation unit 104 , a brightness change frequency distribution calculation unit 106 , and a gas detection unit 108 .

[0054] The image data acquisition unit 102 is a structure for acquiring a plurality of image data obtained by photographing the scene F in time series. In the present embodiment, the gas leakage detection device 100 can communicate with the UAV 10 via the wireless communication network 40, and the image data acquisition unit 102 can acquire a plurality of image data obtained by the imaging device 16 mounted on the UAV 10 via the wireless communication network 40.

[0055] In addition, when the gas leakage detection device 100 is equipped with a storage device (memory, etc.) for storing multiple image data, each image data obtained by the imaging device 16 can be stored in the storage device by establishing a correspondence with the imaging time, and the image data acquisition unit 102 can access the storage device to obtain multiple image data.

[0056] The brightness change evaluation unit 104 is configured to evaluate the brightness change for each pixel included in the image data by comparing the brightness of each pixel between two image data that are consecutive in time series (hereinafter, appropriately referred to as "first image data" and "second image data" when distinguished) among the plurality of image data. The plurality of image data processed by the brightness change evaluation unit 104 is based on the premise that the photographic ranges are consistent with each other. Assuming that the plurality of image data acquired by the image data acquisition unit 102 includes image data with different photographic ranges, the plurality of image data is subjected to the alignment processing described later as preprocessing, thereby performing correction in a manner that the photographic ranges are consistent with each other.

[0057] In addition, in the present application specification, “successively in time series” is not limited to the case where two image data are consecutive in time series, but also includes the case where other image data is between the two image data.

[0058] Specifically, in the brightness change evaluation unit 104, the brightness value of each pixel of the first image data (hereinafter, appropriately referred to as "first brightness value") and the brightness value of each pixel of the second image data (hereinafter, appropriately referred to as "second brightness value") are determined, and the difference in brightness value (= "first brightness value" - "second brightness value") is calculated for each pixel as the brightness change, thereby generating differential image data. That is, the differential image data is defined as an image in which each pixel has a brightness value corresponding to the absolute value of the aforementioned "difference in brightness value".

[0059] The brightness change frequency distribution calculation unit 106 is a structure for calculating the brightness change frequency distribution in the image data by accumulating the brightness change for each pixel. As described above, when differential image data corresponding to the brightness change of each pixel is produced, the brightness change frequency distribution is calculated by accumulating the brightness value of each pixel in the differential image data. Here, accumulation means accumulating the brightness change of each pixel for each pixel for two or more differential image data that are consecutive in time series.

[0060] In addition, in the following embodiment, the case of accumulating the brightness changes of each pixel evaluated for one set (i.e., two) of differential image data that are consecutively connected is described, but the brightness changes of each pixel evaluated for any number of differential image data of more than three may also be accumulated.

[0061] The gas detection unit 108 is a structure for detecting gas leaking in the scene F based on the brightness change frequency distribution calculated by the brightness change frequency distribution calculation unit 106. In the brightness change frequency distribution, the result obtained by accumulating the brightness value corresponding to the brightness change for each pixel is expressed as the brightness change frequency distribution, and the existence of the gas flowing in the scene F is emphasized compared with the brightness distribution of the single image data. Therefore, the gas detection unit 108 detects the gas flowing in the scene F based on the brightness change frequency, thereby being able to well determine the gas leakage site in the scene F.

[0062] Next, a gas leak detection method performed by the gas leak detection device 100 having the above-described structure will be described. Figure 3 1 is a flowchart showing a gas leak detection method according to one embodiment.

[0063] First, the image data acquisition unit 102 acquires a plurality of image data obtained by photographing the scene F in time series (step S100). In the present embodiment, the photographing device 16 is an animation camera, and the image data acquisition unit 102 acquires a plurality of image data by performing frame division on the animation data acquired by the animation camera. Furthermore, the plurality of image data acquired in step S100 may be subjected to noise removal processing or grayscale conversion processing as required.

[0064] In this embodiment, the plurality of image data acquired in step S100 have a common imaging range. Such plurality of image data having a common imaging range can be acquired by, for example, hovering the UAV 10 at a predetermined position of the scene F and photographing with the imaging device 16 mounted on the UAV 10.

[0065] Next, the brightness change evaluation unit 104 selects two image data that are sequentially located in time series from the plurality of image data acquired in step S100 (step S101). In step S101, for example, the plurality of image data acquired in step S100 are sequentially arranged in time series, and any two image data that are sequentially located in time series are selected.

[0066] Next, the brightness change evaluation unit 104 generates differential image data for the two image data (the first image data and the second image data) selected in step S101 (step S102). As described above, with respect to the differential image data, the brightness value of each pixel of the first image data (hereinafter, appropriately referred to as the "first brightness value") and the brightness value of each pixel of the second image data (hereinafter, appropriately referred to as the "second brightness value") are determined, and the difference in brightness value (= "first brightness value" - "second brightness value") is calculated for each pixel as the brightness change to generate. Furthermore, differential image data is generated in which the brightness value of each pixel becomes the absolute value of the obtained difference.

[0067] Next, the gas leakage detection device 100 determines whether the creation of differential image data has been completed for all the two image data that are sequentially located one after the other in time series (step S103) for the plurality of image data acquired in step S100. If the creation of differential image data has not been completed for all the two image data that are sequentially located one after the other in time series (step S103: "No"), the gas leakage detection device 100 returns the process to step S101 to create differential image data for other two image data that are sequentially located one after the other in time series.

[0068] When the differential image data has been generated for all two image data that are consecutive in time series (step S103: "Yes"), the brightness change frequency distribution calculation unit 106 calculates the brightness change frequency distribution by accumulating the plurality of differential image data that have been repeatedly generated (step S104). The brightness change frequency distribution is obtained by accumulating the brightness values ​​of the plurality of differential image data for each pixel.

[0069] Here, Figure 4 Yes means Figure 3 A schematic diagram of the method for calculating the brightness change frequency distribution in step S104. Figure 4 , the t=n-1th and nth differential image data and a matrix table representing the brightness value of each pixel arranged two-dimensionally in each differential image data (rows: 1, 2, 3, ..., columns: A, B, C, ...) are shown. The brightness change frequency distribution calculation unit 106 calculates the brightness change frequency distribution according to the following formula by integrating the brightness value for each pixel of such a plurality of differential image data.

[0070]

[0071] Here, S xy is the accumulated brightness change frequency of each pixel xy, l xy＿t is the brightness value of each pixel xy in the tth differential image data.

[0072] In addition, Figure 4 , the case where the brightness value of each pixel in the t=n-1th and nth differential image data is binarized to "1" or "0" is illustrated for easy understanding.

[0073] Next, the luminance change frequency distribution calculation unit 106 normalizes the luminance change frequency distribution calculated in step S104 (step S105). The normalization process is a process for determining the maximum value from the luminance change frequency distribution calculated in step S105 and performing normalization calculation so that the maximum value becomes a predetermined value.

[0074] Figure 5 Yes means Figure 3 Schematic diagram of the normalization process in step S105. Figure 5 An example of the brightness change frequency distribution calculated in step S104 is shown in FIG. xy The luminance change frequency distribution calculation unit 106 calculates the luminance change frequency S contained in the luminance change frequency distribution. xy The maximum value (max) is determined in the above formula, and the brightness change frequency S corresponding to each pixel xy is adjusted so that the maximum value becomes a predetermined value (255). xy Standardize (the standardized brightness change frequency S xy Expressed as "normalized brightness change frequency S' xy ”).

[0075] Next, the gas detection unit 108 detects gas leakage based on the brightness change frequency distribution that has been normalized in step S105 (step S106). In step S106, by using the brightness change frequency distribution obtained by accumulating brightness changes between two image data that are sequentially located in time series among a plurality of image data, the state of the gas leaking at the site F can be more clearly captured than when using the brightness distribution of a single image data.

[0076] In addition, the “detection” of the gas leakage in step S106 includes “detecting” whether there is a gas leakage, and in addition to this, the leakage range of the gas may be detected.

[0077] Figure 6 is Figure 3 An example of a brightness change frequency distribution used in step S106. Figure 6 As shown in the brightness change frequency distribution, the pixel with a larger brightness value is displayed as the accumulated value of the brightness change between two image data that are consecutive in time series among the multiple image data. The gas detection unit 108 can detect gas leakage in the scene F well by analyzing this brightness change frequency distribution.

[0078] In addition, in this embodiment, the case of detecting gas leakage based on the brightness change frequency distribution that has been standardized in step S105 is illustrated, but when the standardization process of step S105 is omitted, gas leakage can be detected based on the brightness change frequency distribution that has not been standardized (i.e., the result calculated in step S104).

[0079] Next, another embodiment of the gas leak detection method will be described. Figure 71 is a flowchart showing a gas leak detection method according to another embodiment.

[0080] Figure 7 Steps S200 to S203 are respectively Figure 3 The steps S100 to S103 are the same, but Figure 7 In step S204, each difference image data generated up to step S203 is subjected to binarization processing. This binarization processing is performed, for example, by the brightness change evaluation unit 104 comparing the brightness of each pixel of the difference image data with a preset threshold value.

[0081] Figure 8 It means in Figure 7 Schematic diagram of the binarization process performed in step S204. In the binarization process, a brightness value is determined for each pixel of the differential image data, and when the brightness value is above a threshold, it is converted to a brightness value of "1", and when the brightness value is less than the threshold, it is converted to a brightness value of "0". Figure 8 As shown, each pixel of the difference image data has a brightness value of "1" or "0" after conversion, and the difference image data becomes a black and white image through binarization processing.

[0082] In the binarization process, differential image data in which pixels having brightness values ​​above the threshold are emphasized can be obtained. And, in step S205, the differential image data subjected to the binarization process is accumulated in the same manner as in step S104, thereby calculating the brightness change frequency distribution. In this embodiment, the brightness change frequency distribution is calculated by accumulating the differential image data subjected to the binarization process, thereby obtaining a brightness change frequency distribution in which pixels having brightness values ​​above the threshold are emphasized compared to the above-mentioned embodiment. Therefore, in step S207, gas leakage is detected based on this calculated brightness change frequency distribution, thereby enabling gas leakage to be detected with higher accuracy.

[0083] In addition, in this embodiment, the difference image data subjected to the binarization process in step S204 is processed in the same manner as in the above-mentioned embodiment after step S205 (ie, Figure 7 Steps S205 to S207 are respectively Figure 3 is the same as steps S104 to S106).

[0084] Next, a gas leakage detection device 100 ′ according to another embodiment will be described. Fig. 9 2 is a block diagram of a gas leakage detection device 100 ′ according to another embodiment.

[0085] Fig. 9 The gas leakage detection device 100' is shown with Figure 2 Compared with the gas leakage detection device 100 shown in the figure, the gas leakage detection device 100 ′ of the present embodiment is different in that it includes a positioning processing unit 110. In the present embodiment, for example, when the UAV 10 moves on the site F, the imaging angle of the imaging device 16 mounted on the UAV 10 or the distance to the object (gas leakage site) changes, and thus the imaging ranges of the plurality of image data may be different. In the case of detecting gas leakage using such a plurality of image data, in the aforementioned gas leakage detection device 100, when the brightness change evaluation unit 104 creates differential image data from two consecutive image data, due to the difference in the imaging range, it is easy to detect the brightness change even in the area other than the area where the gas is located, and as a result, the possibility of erroneous detection of the gas leakage site increases. In order to prevent such erroneous detection, the gas leakage detection device 100 ′ of the present embodiment includes a positioning processing unit 110, which is used to perform positioning processing as pre-processing on the plurality of image data used to create the differential image data.

[0086] Next, a gas leakage detection method according to another embodiment implemented by the gas leakage detection device 100 ′ having the above-mentioned structure will be described. Fig.10 1 is a flowchart showing a gas leak detection method according to another embodiment.

[0087] First, steps S300 to S301 are the same as the above-mentioned steps S100 to S101, where the image data acquisition unit 102 acquires a plurality of image data and selects two image data that are sequentially located in time series for producing differential image data from the plurality of image data. Then, the alignment processing unit 110 performs alignment processing on the two image data selected in step S301 (step S302).

[0088] Here, reference Fig.11 and Fig.12 , the alignment processing implemented in step S302 is described in detail. Fig.11 is Fig.10 The sub-flow chart of the alignment processing implemented in step S302, Fig.12 Yes means confirmation Fig.11 Schematic diagram of the state of the feature points in step S402.

[0089] First, the alignment processing unit 110 acquires the image data Di (the two image data selected in step S301) as the alignment processing target (step S400). As described above, the two image data Di acquired here have an arbitrary coordinate system xyz due to changes in the shooting angle or the distance to the object (gas leakage site).

[0090] Next, the alignment processing unit 110 acquires reference image data Diref (step S401). The reference image data Diref is prepared in advance as image data having a reference coordinate system XYZ, and the reference coordinate system XYZ becomes a reference for matching the coordinate system xyz of the image data Di to be subjected to the alignment processing. The reference image data Diref is stored in a storage device such as a memory, for example, and the alignment processing unit 110 acquires the reference image data Diref by accessing the storage device.

[0091] Next, the alignment processing unit 110 determines feature points Pc for each of the image data Di acquired in step S400 and the reference image data Diref acquired in step S401 (step S402). The feature points Pc are coordinate points that serve as references when grasping the coordinate system of each image data, and are determined as points where the brightness variation between pixels is large, such as corners or edges of structures included in the image.

[0092] In this embodiment, for the image data Di (or the reference image data Diref), the inspection area Ad is set to include a plurality of pixels, and when the inspection area Ad is moved, the position where the brightness change in the inspection area Ad is greater than a preset reference value is determined as a feature point Pc. Fig.12 In the image data Di (or reference image data Diref), a portion of the scene F photographed by the photographic device 16 is represented as image data Di (or reference image data Diref). In the image data, an inspection area Ad set as a roughly rectangular area containing a plurality of pixels is superimposed and displayed. The alignment processing unit 110 detects the brightness change contained in the inspection area Ad while scanning the inspection area Ad on the image data. Specifically, the brightness change in the inspection area Ad when the inspection area Ad moves from position n to (n+1) on the scene F is detected, and it is determined whether the brightness change is greater than a preset reference value. As a result, the position where the brightness change contained in the inspection area Ad is greater than the reference value suggests that the scene F contains some characteristic structure (for example, a corner or an edge, etc.), and the alignment processing unit 110 determines this position as a feature point Pc. Fig.12 In FIG. 1 , as several characteristic points Pc thus determined, the edge or corner of the structure located at the site F is shown (refer to FIG. Fig.12 several black dots shown in Figure 3).

[0093] Furthermore, as another method for determining the feature point Pc in step S402, a neural network model learned using training data can be used. At this time, training data obtained by associating the image pattern and the feature point Pc is prepared, and the neural network model is learned using the training data. The alignment processing unit 110 inputs the image data Di (or the reference image data Diref) to the neural network model constructed in this way. The neural network model distinguishes the contrast (black and white), corners, edges, etc. in the input image data Di (or the reference image data Diref) based on the information of the hidden layer constructed by learning based on the training data, and outputs the feature point Pc.

[0094] In addition, the training data used in learning the neural network model can be prepared individually, but for example, a plurality of data produced by rotating a specific sample image around a central axis can also be prepared. In this way, a large amount of training data can be produced from the original limited sample images.

[0095] Next, the alignment processing unit 110 converts the coordinate system xyz of the image data Di into the coordinate system XYZ of the reference image data Diref based on the feature point Pc determined in step S402 (step S403). Specifically, in step S403, an affine transformation sequence for affine transformation of the coordinate system xyz of the image data Di into the coordinate system XYZ of the reference image data Diref is obtained so that the feature point Pc determined in the image data Di is consistent with the feature point Pc determined in the reference image data Diref, and the alignment processing is performed by applying the affine transformation sequence to the image data Di.

[0096] The affine transformation sequence used in step S403 can be obtained using a pre-constructed deep neural network model, for example. In this case, for example, several model cases that use an affine transformation sequence to match specified image data with reference image data are manually prepared and stored in a database, and the database is used as training data to learn the deep neural network model. In this case, as a loss function for deep learning, for example, the brightness change of the feature point Pc determined in the reference image data Diref can be used. By inputting the image data Di and the reference image data Diref into the deep neural network model constructed in this way, the alignment processing unit 110 can obtain an appropriate affine transformation sequence with a small computational burden (for example, even for unexpected fluctuations of UAV10, the affine transformation can be immediately obtained, and each frame image can be aligned).

[0097] Thus, in step S302, the two image data selected in step S301 are subjected to a positional alignment process, whereby the coordinate system xyz of each image data is aligned with the coordinate system XYZ of the reference image data. Fig.10In the subsequent step S303, the two image data subjected to the alignment processing are used to produce differential image data. The following steps S304 to S308 are the same as the aforementioned steps S103 to S106. By performing processing using the image data subjected to the alignment processing, false detection is less likely to occur, and gas leakage can be detected with high accuracy.

[0098] As described above, according to the above embodiment, by comparing the brightness of each pixel between two image data that are sequentially located in a time series among a plurality of image data acquired by photographing the scene F in a time series, the brightness change of each pixel is evaluated. With respect to the brightness change of each pixel, the brightness change frequency distribution is obtained by accumulating each combination of two image data that are sequentially located in a time series included in the plurality of image data. The brightness change frequency distribution thus calculated is emphasized in the following manner: the more likely a pixel is to change in brightness due to the gas flowing in the scene F, the greater the value of the brightness change frequency. Therefore, by performing gas detection based on the brightness change frequency distribution, gas leakage in the scene F can be detected with high accuracy regardless of the condition of the scene F (for example, objects or temperature included as a background, etc.).

[0099] The contents described in the above-mentioned embodiments can be understood, for example, as follows.

[0100] (1) A gas leak detection device according to one embodiment includes:

[0101] An image data acquisition unit for acquiring a plurality of image data obtained by photographing the scene in time series;

[0102] a brightness change evaluation unit for evaluating brightness changes for each pixel included in the image data by comparing brightness of each pixel between two of the image data that are consecutive in time series among the plurality of image data;

[0103] a brightness change frequency distribution calculation unit configured to calculate a brightness change frequency distribution in the image data by accumulating the brightness change for each of the pixels; and

[0104] A gas detection unit is used to detect gas leakage in the scene based on the brightness change frequency distribution.

[0105] According to the above method (1), by comparing the brightness of each pixel between two image data that are sequentially located in time series among a plurality of image data acquired by photographing the scene F in time series, the brightness change of each pixel is evaluated. With respect to the brightness change of each pixel, the brightness change frequency distribution is calculated by accumulating each combination of two image data that are sequentially located in time series included in the plurality of image data. The brightness change frequency distribution thus calculated is emphasized in the following manner: the more likely a pixel is to change in brightness due to the gas flowing in the scene, the greater the value of the brightness change frequency. Therefore, by performing gas detection based on the brightness change frequency distribution, gas leakage in the scene can be detected with high accuracy regardless of the conditions of the scene (for example, objects or temperature included as the background, etc.).

[0106] (2) In another embodiment, in the above embodiment (1),

[0107] The brightness change evaluation unit generates differential image data between the two image data.

[0108] The brightness change frequency distribution calculation unit calculates the brightness change frequency distribution by integrating the difference image data.

[0109] According to the above-mentioned method (2), for two image data that are consecutive in time series, differential image data is generated in which the brightness change of each pixel is used as the brightness value of each pixel. By accumulating the differential image data generated in this way for each combination of two image data that are consecutive in time series included in the plurality of image data, it is possible to obtain a brightness change frequency distribution that can well identify the presence area of ​​gas in the scene.

[0110] (3) In another embodiment, in the above embodiment (2),

[0111] The brightness change evaluation unit performs binarization processing on the difference image data by comparing the brightness of each pixel of the difference image data with a preset threshold value.

[0112] According to the above aspect (3), a binarization process is performed on the difference image data to compare the brightness of each pixel with a threshold value. By integrating the difference image data subjected to such binarization process, a brightness change frequency distribution can be obtained that further emphasizes the area where the gas flowing on the scene is located.

[0113] (4) In another embodiment, in any one of the above (1) to (3),

[0114] The gas detection unit visualizes the brightness change frequency distribution as a gas distribution image having pixel brightness values ​​corresponding to the magnitude of the brightness change frequency.

[0115] According to the above aspect (4), the brightness change frequency distribution is visualized in a manner having pixel brightness values ​​corresponding to the brightness change frequency. The brightness change frequency distribution visualized in this way can be used as a gas distribution image representing the range of the gas flowing in the scene.

[0116] (5) In another embodiment, in any one of the above (1) to (4),

[0117] The device further includes a positioning processing unit for performing a positioning process for aligning the coordinate systems of the plurality of image data with a reference coordinate system of reference image data acquired in advance.

[0118] According to the above-mentioned aspect (5), by performing alignment processing on a plurality of image data, the coordinate system of each image data is aligned with the reference coordinate system of the reference image data. Thus, for example, even if the coordinate system of each image data deviates from the reference coordinate system by taking a plurality of image data while moving, it is possible to calculate the brightness change frequency distribution for detecting gas leakage based on the plurality of image data subjected to the alignment processing.

[0119] (6) In another embodiment, in the above embodiment (5),

[0120] The alignment process is performed by affine transformation for converting the coordinate system into the reference coordinate system so that feature points respectively included in the plurality of image data and the reference image data are aligned.

[0121] According to the above aspect (6), the alignment process can be performed by performing affine transformation so that the feature points included in the plurality of image data coincide with the feature points included in the reference image data.

[0122] (7) In another embodiment, in the above embodiment (6),

[0123] The feature point is specified as a position where a brightness change amount in the inspection area is larger than a reference value when the inspection area set to include a plurality of pixels is moved in the plurality of image data and the reference image data.

[0124] According to the above aspect (7), as the position where the amount of brightness change when moving the inspection area set to include a plurality of images is greater than the reference point, it is possible to well identify the feature point used for the alignment process from the image data and the reference image data.

[0125] (8) In another embodiment, in the above embodiment (6),

[0126] The feature points are determined from the plurality of image data and the reference image data using a neural network model learned using training data.

[0127] According to the above aspect (8), by analyzing the image data and the reference image data using the learned neural network model, it is possible to accurately identify feature points used for the alignment process from the image data and the reference image data.

[0128] (9) In another embodiment, in the above embodiment (8),

[0129] The training data includes a plurality of data generated by rotating a sample image around a central axis.

[0130] According to the above method (9), the training data used in learning the neural network model for determining the feature points used for alignment processing from the image data and the reference image data is prepared as a plurality of data produced by rotating the sample image around the central axis. Thus, a large amount of training data can be efficiently produced based on a relatively small number of sample images.

[0131] (10) In another embodiment, in the above embodiment (6),

[0132] The affine transformation is performed using an affine sequence obtained by inputting the plurality of image data and the reference image data into a pre-built deep neural network model.

[0133] According to the above method (10), a plurality of image data and the reference image data are input to the deep neural network model, and an affine sequence for aligning the coordinate system corresponding to the plurality of image data with the reference coordinate system corresponding to the reference image data is output. Thus, the affine sequence required for the alignment process can be obtained with a small computational burden, and is therefore suitable for high-speed processing.

[0134] (11) In another embodiment, in any one of the above (1) to (10),

[0135] The plurality of image data are captured by a photographing device mounted on a mobile object.

[0136] According to the above aspect (11), it is possible to preferably detect gas leakage at the site based on a plurality of image data captured by the imaging device mounted on the mobile body.

[0137] (12) In another embodiment, in any one of the above (1) to (11),

[0138] The plurality of image data are obtained by photographing the scene using an infrared camera.

[0139] According to the above aspect (12), it is possible to well detect leakage of a gas such as CO 2 gas on site based on a plurality of image data captured by an infrared camera.

[0140] (13) In another embodiment, in any one of the above (1) to (12),

[0141] The plurality of image data are a plurality of frame images constituting animation data.

[0142] According to the above aspect (13), by processing a plurality of frame images constituting the moving image data as the plurality of image data, it is possible to preferably detect gas leakage at the site based on the moving image data.

[0143] (14) A gas leakage detection method according to one embodiment includes the following steps:

[0144] A step of acquiring a plurality of image data obtained by photographing the scene in time series;

[0145] A step of evaluating a brightness change for each pixel included in the image data by comparing the brightness of each pixel between two image data that are consecutive in time series among the plurality of image data;

[0146] a step of calculating a frequency distribution of brightness changes in the image data by accumulating the brightness changes for each of the pixels; and

[0147] A step of detecting gas leakage in the scene based on the brightness change frequency distribution.

[0148] According to the above method (14), by comparing the brightness of each pixel between two image data that are sequentially arranged in time series among a plurality of image data acquired by photographing the scene F in time series, the brightness change of each pixel is evaluated. With respect to the brightness change of each pixel, the brightness change frequency distribution is calculated by accumulating each combination of two image data that are sequentially arranged in time series included in the plurality of image data. The brightness change frequency distribution thus calculated is emphasized in the following manner: the more easily the pixel changes in brightness due to the gas flowing in the scene, the greater the value of the brightness change frequency. Therefore, by performing gas detection based on the brightness change frequency distribution, gas leakage in the scene can be detected with high accuracy regardless of the conditions of the scene (for example, objects or temperature included as the background, etc.).

[0149] (15) A gas leak detection program according to one embodiment can execute the following steps using a computer device:

[0150] A step of acquiring a plurality of image data obtained by photographing the scene in time series;

[0151] A step of evaluating a brightness change for each pixel included in the image data by comparing the brightness of each pixel between two image data that are consecutive in time series among the plurality of image data;

[0152] a step of calculating a frequency distribution of brightness changes in the image data by accumulating the brightness changes for each of the pixels; and

[0153] A step of detecting gas leakage in the scene based on the brightness change frequency distribution.

[0154] According to the above method (15), by comparing the brightness of each pixel between two image data that are sequentially arranged in time series among a plurality of image data acquired by photographing the scene F in time series, the brightness change of each pixel is evaluated. With respect to the brightness change of each pixel, the brightness change frequency distribution is calculated by accumulating each combination of two image data that are sequentially arranged in time series contained in the plurality of image data. The brightness change frequency distribution thus calculated is emphasized in the following manner: the more easily the pixel changes in brightness due to the gas flowing in the scene, the greater the value of the brightness change frequency. Therefore, by performing gas detection based on the brightness change frequency distribution, gas leakage in the scene can be detected with high accuracy regardless of the conditions of the scene (for example, objects or temperature included as the background, etc.).

[0155] Explanation of symbols

[0156] 10-AUV, 12-main body, 14-propeller, 16-photographic device, 40-wireless communication network, 100-gas leakage detection device, 102-image data acquisition unit, 104-brightness change evaluation unit, 106-brightness change frequency distribution calculation unit, 108-gas detection unit, 110-alignment processing unit.

Claims

1. A gas leak detection device, comprising: An image data acquisition unit for acquiring a plurality of image data obtained by photographing the scene in time series; a brightness change evaluation unit for evaluating brightness changes for each pixel included in the image data by comparing brightness of each pixel between two of the image data that are consecutive in time series among the plurality of image data; a brightness change frequency distribution calculation unit configured to calculate a brightness change frequency distribution in the image data by accumulating the brightness change for each of the pixels; and A gas detection unit is used to detect gas leakage in the scene based on the brightness change frequency distribution.

2. The gas leakage detection device according to claim 1, wherein: The brightness change evaluation unit generates differential image data between the two image data. The brightness change frequency distribution calculation unit calculates the brightness change frequency distribution by integrating the difference image data.

3. The gas leakage detection device according to claim 2, wherein: The brightness change evaluation unit performs binarization processing on the difference image data by comparing the brightness of each pixel of the difference image data with a preset threshold value.

4. The gas leakage detection device according to claim 1 or 2, wherein: The gas detection unit visualizes the brightness change frequency distribution as a gas distribution image having pixel brightness values ​​corresponding to the magnitude of the brightness change. 5 . The gas leakage detection device according to claim 1 , further comprising a positioning processing unit configured to perform a positioning process for aligning a coordinate system of the plurality of image data with a reference coordinate system of reference image data acquired in advance.

6. The gas leakage detection device according to claim 5, wherein: The alignment process is performed by affine transformation for converting the coordinate system into the reference coordinate system so that feature points respectively included in the plurality of image data and the reference image data are aligned.

7. The gas leakage detection device according to claim 6, wherein: The feature point is specified as a position where a brightness change amount in the inspection area is larger than a reference value when the inspection area set to include a plurality of pixels is moved in the plurality of image data and the reference image data.

8. The gas leakage detection device according to claim 6, wherein: The feature points are determined from the plurality of image data and the reference image data using a neural network model learned using training data.

9. The gas leakage detection device according to claim 8, wherein: The training data includes a plurality of data generated by rotating a sample image around a central axis.

10. The gas leakage detection device according to claim 6, wherein: The affine transformation is performed using an affine sequence obtained by inputting the plurality of image data and the reference image data into a pre-built deep neural network model.

11. The gas leakage detection device according to claim 1 or 2, wherein: The plurality of image data are captured by a photographing device mounted on a mobile object.

12. The gas leakage detection device according to claim 1 or 2, wherein: The plurality of image data are obtained by photographing the scene using an infrared camera.

13. The gas leakage detection device according to claim 1 or 2, wherein: The plurality of image data are a plurality of frame images constituting animation data.

14. A gas leakage detection method, comprising the following steps: A step of acquiring a plurality of image data obtained by photographing the scene in time series; A step of evaluating a brightness change for each pixel included in the image data by comparing the brightness of each pixel between two image data that are consecutive in time series among the plurality of image data; a step of calculating a frequency distribution of brightness changes in the image data by accumulating the brightness changes for each of the pixels; and A step of detecting gas leakage in the scene based on the brightness change frequency distribution.

15. A gas leak detection program, which can use a computer device to execute the following steps: A step of acquiring a plurality of image data obtained by photographing the scene in time series; A step of evaluating a brightness change for each pixel included in the image data by comparing the brightness of each pixel between two image data that are consecutive in time series among the plurality of image data; a step of calculating a frequency distribution of brightness changes in the image data by accumulating the brightness changes for each of the pixels; and A step of detecting gas leakage in the scene based on the brightness change frequency distribution.

Images