Gas leakage monitoring device, gas leakage monitoring method, and program

Abstract

This gas leakage monitoring device comprises: an image data acquisition unit that acquires, as original image data, image data generated by capturing images at different times; a vector generation unit that selects two pieces of original image data in chronological order from the original image data, and performs predetermined image processing for calculating the movement direction and movement speed of an object included in the images, for the selected two pieces of original image data, to generate a vector indicating the movement direction and movement speed for each pixel included in the original image data; and a frequency distribution generation unit that classifies the vectors generated by the vector generation unit into sections of predetermined vector lengths, and generates a frequency distribution for gas leakage monitoring.

Description

Gas leakage monitoring device, gas leakage monitoring method, and program

[0001] This application claims priority to Japanese Patent Application No. 2023-204587, filed on December 4, 2023, the contents of which are incorporated herein by reference.

[0002] Gases such as CO2 gas each have their own unique absorption wavelength bands, and a gas leak detection method utilizing the characteristics of these absorption wavelength bands is known (see, for example, Patent Document 1). For example, CO2 gas has an absorption wavelength band around 4.3 μm. Therefore, when capturing an image using an infrared camera equipped with a filter that transmits 4.3 μm infrared light, the following phenomenon occurs: If CO2 gas is present in a region between the object being captured and the infrared camera, a difference in intensity occurs due to absorption by the CO2 gas between the electromagnetic waves emitted from the object being captured and directly reaching the infrared camera and the electromagnetic waves that pass through the CO2 gas and reach the infrared camera. This difference in electromagnetic wave intensity appears as a difference in brightness value in the image data generated by the infrared camera, making it possible to visualize the CO2 gas.

[0003] As a method for detecting visualized CO2 gas in image data, for example, Patent Document 1 discloses the following method. Image data generated by capturing images with an infrared camera has a difference in brightness value as described above. This method utilizes this difference in brightness value to detect pixels whose brightness value change per unit time is equal to or less than a preset threshold as pixels containing gas.

[0004] Patent No. 6665863

[0005] It is desirable to determine the presence or absence of a gas leak with high accuracy and in as short a time as possible. From this perspective, the process of observing the amount of change in luminance value per unit time for each pixel, which is included in the method disclosed in Patent Document 1, is considered to have the following problem. That is, in order to improve the accuracy of the amount of change in luminance value per unit time, it is necessary to take a large number of captured images, but increasing the number of images poses the problem of longer processing time.

[0006] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a gas leak monitoring device, a gas leak monitoring method, and a program that can accurately and in a shorter time indicate the presence or absence of a gas leak.

[0007] In order to solve the above problem, the gas leak monitoring device of the present disclosure comprises an image data acquisition unit that acquires image data generated by photographing at different times as original image data; a vector generation unit that selects two original image data items in chronological order from the original image data and performs predetermined image processing on the two selected original image data items to calculate the movement direction and movement speed of an object included in the image, thereby generating vectors indicating the movement direction and movement speed for each pixel included in the original image data; and a frequency distribution generation unit that classifies the vectors generated by the vector generation unit into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring.

[0008] The gas leak monitoring method according to the present disclosure includes the steps of acquiring image data generated by photographing at different times as original image data, selecting two pieces of original image data in chronological order from the acquired original image data, and performing predetermined image processing on the two selected original image data to calculate the direction of movement and speed of movement of an object included in the image, thereby generating vectors indicating the direction of movement and speed of movement for each pixel included in the original image data, and classifying the generated vectors into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring.

[0009] The program according to the present disclosure causes a computer to execute the following steps: acquiring image data generated by photographing at different times as original image data; selecting two pieces of original image data in chronological order from the acquired original image data; and performing predetermined image processing on the two selected original image data to calculate the direction of movement and speed of movement of an object included in the image, thereby generating vectors indicating the direction of movement and speed of movement for each pixel included in the original image data; and classifying the generated vectors into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring.

[0010] According to the gas leakage monitoring device, gas leakage monitoring method, and program of the present disclosure, it is possible to indicate the presence or absence of a gas leakage with high accuracy and in a shorter time.

[0011] 1 is a block diagram illustrating an example configuration of a gas leak monitoring system according to an embodiment of the present disclosure. FIG. 2 is a flowchart illustrating an example operation of an image data acquisition unit included in a gas leak monitoring device according to an embodiment of the present disclosure. FIG. 3 is a flowchart illustrating an example operation of an image processing unit, a vector generation unit, a filter unit, a frequency distribution generation unit, and a determination unit included in a gas leak monitoring device according to an embodiment of the present disclosure. FIG. 4 is a flowchart of a subroutine of image processing performed by an image processing unit included in a gas leak monitoring device according to an embodiment of the present disclosure. FIG. 5 is a diagram illustrating an example of an image represented by original image data according to an embodiment of the present disclosure. FIG. 6 is a diagram illustrating an example of an image represented by processed image data according to an embodiment of the present disclosure. FIG. 7 is a diagram illustrating an example of original image data and a first vector generated from the original image data according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating an example of processed image data and a second vector generated from the processed image data according to an embodiment of the present disclosure. FIG. 9 is a diagram illustrating an overview of filtering by a filter unit according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of processed image data and a vector extracted by filtering by the filter unit according to an embodiment of the present disclosure. FIG. 11 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment.

[0012] Hereinafter, a gas leakage monitoring device, a gas leakage monitoring method, and a program according to embodiments of the present disclosure will be described with reference to the drawings. Note that the same or corresponding components in the drawings will be denoted by the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0013] (System Configuration) Fig. 1 is a block diagram showing the configuration of a gas leak monitoring system 1 according to an embodiment of the present disclosure. The gas leak monitoring system 1 includes an imaging device 2, a communication network 3, and a gas leak monitoring device 10. In the gas leak monitoring system 1, the gas to be monitored is assumed to be CO2 gas. The communication network 3 is, for example, a communication network operated by a telecommunications carrier, and is a communication network that can be connected via wired or wireless connections.

[0014] The imaging device 2 is, for example, an infrared camera equipped with a filter that transmits infrared light of 4.3 μm, which is the absorption wavelength band of CO2 gas. The object to be imaged by the imaging device 2 is, for example, a field in which a plant or the like is installed. The imaging device 2 is mounted, for example, on an unmanned aerial vehicle (hereinafter referred to as a UAV), and captures video from the sky toward the field while traveling together with the UAV. The imaging device 2 is equipped with communication equipment that wirelessly connects to the communication network 3, and transmits video data generated by the image capture to the gas leak monitoring device 10 via the communication network 3.

[0015] (Configuration of Gas Leak Monitoring Device) The gas leak monitoring device 10 can be configured using, for example, a computer such as a server, a personal computer, or a microcomputer, and peripheral devices of that computer, and is equipped with an image data acquisition unit 11, an original image data storage unit 12, an image processing unit 13, a processed image data storage unit 14, a vector generation unit 15, a filter unit 16, a frequency distribution generation unit 17, a determination unit 18, and a display unit 19 as a functional configuration formed by a combination of hardware of the computer or the like and software such as a program executed by the computer.

[0016] The image data acquisition unit 11 is connected to the communication network 3 via a wired or wireless connection and receives video data transmitted by the imaging device 2. The video data received by the image data acquisition unit 11 includes frames arranged in the order in which they were captured, with each frame associated with time information indicating the time at which it was captured. The image data acquisition unit 11 divides the frames included in the video data it receives into chronological order. The image data acquisition unit 11 generates original image data from each of the divided frames, associates the generated original image data with the time information associated with the frame corresponding to the original image data, and records the original image data in the original image data storage unit 12.

[0017] The image processing unit 13 performs predetermined image processing to enhance the gas on the original image data recorded in the original image data storage unit 12, and generates processed image data corresponding to the original image data. The image processing unit 13 associates the generated processed image data with time information associated with the original image data corresponding to the processed image data, and records the resulting image data in the processed image data storage unit 14.

[0018] The vector generation unit 15 selects two pieces of original image data that are adjacent in chronological order from the original image data storage unit 12. The vector generation unit 15 selects two pieces of processed image data that correspond to each of the two selected pieces of original image data. Here, the two pieces of processed image data that correspond to each of the two pieces of original image data are, specifically, two pieces of processed image data that are associated with the same time information as the time information of each of the two pieces of original image data, and the two pieces of processed image data are also adjacent in chronological order.

[0019] The vector generation unit 15 performs predetermined image processing on the two selected original image data to calculate the movement direction and movement speed of an object included in the image, and generates vectors indicating the movement direction and movement speed (hereinafter, these vectors will be referred to as first vectors) for each pixel.The vector generation unit 15 performs predetermined image processing on the two selected processed image data to calculate the movement direction and movement speed of an object included in the image, and generates vectors indicating the movement direction and movement speed (hereinafter, these vectors will be referred to as second vectors) for each pixel.

[0020] As a predetermined image processing for calculating the moving direction and moving speed of an object included in an image, for example, optical flow processing of the Gunnar-Farneback method, which is one of the flow estimation methods, is applied.

[0021] The filter unit 16 performs a process called filtering, in which each second vector for each pixel is filtered using the first vector whose pixel matches each second vector, thereby emphasizing and extracting vectors corresponding to gas. The frequency distribution generation unit 17 classifies the vectors for each pixel output by the filter unit 16 into intervals of predetermined vector lengths to generate a frequency distribution. The determination unit 18 determines the presence or absence of gas leakage based on the frequency of a predetermined interval in the frequency distribution generated by the frequency distribution generation unit 17 and a predetermined threshold.

[0022] The display unit 19 is, for example, a display device such as a liquid crystal display, and displays the frequency distribution generated by the frequency distribution generating unit 17 as a histogram, and also displays the determination result of the determining unit 18 .

[0023] (Example of Operation of Gas Leakage Monitoring Device) An example of operation of the gas leakage monitoring device 10 will be described with reference to the flowcharts of FIGS.

[0024] As shown in FIG. 2, the image data acquisition unit 11 receives video data transmitted by the imaging device 2 (Sa1). While continuing to receive the video data transmitted by the imaging device 2, the image data acquisition unit 11 sequentially divides the video data, starting with the most recently received frame, and generates image data from the divided frames. The image data acquisition unit 11 performs noise reduction processing on the generated image data. Here, the video data captured by the imaging device 2 is color video data, and the image data acquisition unit 11 converts the image data into grayscale image data so as to preserve the luminance of the noise-removed image data, thereby generating original image data. The pixel value of each pixel in the original image data is a value between 0 and 255 that represents a luminance value (Sa2).

[0025] The image shown in Fig. 5 is an example of an image represented by the original image data. An object indicated by reference numeral 100 in Fig. 5 is, for example, a structure that is fixedly installed in a plant or the like.

[0026] The image data acquisition unit 11 associates the original image data with the time information that was associated with the frame from which the original image data was generated, and records the associated original image data in the original image data storage unit 12 (Sa3). The image data acquisition unit 11 determines whether or not it is continuing to receive video data (Sa4). If it determines that it is continuing to receive video data (Sa4, Yes), it performs the process of Sa2 on frames that have not been divided into frames. On the other hand, if it determines that it is not continuing to receive video data (Sa4, No), it ends the process.

[0027] The image data acquisition unit 11 may divide one frame and perform the processes Sa2 and Sa3 before performing the determination process Sa4, or may divide a certain number of frames and perform the processes Sa2 and Sa3 for each of the divided frames before performing the determination process Sa4. In this way, the original image data storage unit 12 stores original image data arranged in chronological order and associated with time information.

[0028] 3, image processing unit 13 determines whether a predetermined number of original image data have been stored in original image data storage unit 12 (Sb1). Here, the predetermined number is the number of original image data required for image processing unit 13 to perform predetermined image processing for enhancing gas, and is set to a value of, for example, approximately "5" or "6."

[0029] If the image processing unit 13 determines that the predetermined number of original image data items have not been recorded in the original image data storage unit 12 (Sb1, No), it performs the process of Sb1 again. On the other hand, if the image processing unit 13 determines that the predetermined number of original image data items have been recorded in the original image data storage unit 12 (Sb1, Yes), it reads out the original image data recorded in the predetermined number position in the original image data storage unit 12 and the time information associated with the original image data from the original image data storage unit 12. The image processing unit 13 sets the read-out original image data as the reference original image data (Sb2-1). After determining the reference original image data, the image processing unit 13 starts the image processing subroutine shown in FIG. 4, which is the predetermined image processing for enhancing the gas described above (Sb3).

[0030] (Predetermined Image Processing to Enhance Gas by Image Processing Unit) The image processing unit 13 records the time information corresponding to the reference original image data that was read together with the reference original image data in an internal storage area (Sc1). The image processing unit 13 reads (a predetermined number minus 1) pieces of original image data that are consecutive in chronological order and that are from a time earlier than the time indicated by the time information corresponding to the reference original image data from the original image data storage unit 12 (Sc2). Because the process of Sc2 is performed via the process of Sb1, when the process of Sc2 is performed, at least (a predetermined number minus 1) pieces of original image data that are chronologically earlier than the reference original image data are stored in the original image data storage unit 12.

[0031] Here, the definitions of the terms "earlier time" and "later time" will be explained. For example, suppose there are three pieces of original image data associated with three pieces of time information: "00:00:01," "00:00:02," and "00:00:03." In this case, the original image data associated with time information indicating a time earlier than the time indicated by the time information associated with the original image data of "00:00:02" is the original image data of "00:00:01." In contrast, the original image data associated with time information indicating a time later than the time indicated by the time information associated with the original image data of "00:00:02" is the original image data of "00:00:03." The definitions of the terms "earlier time" and "later time" will be the same hereinafter.

[0032] The image processing unit 13 generates a combination of two original image data sets that are successive in chronological order from a predetermined number of original image data sets, which is the original image data set that has been read and the reference original image data set (Sc3). Here, successive in chronological order does not necessarily mean that the original image data sets are consecutive in chronological order, but simply that they are successive in chronological order. Therefore, the image processing unit 13 generates a predetermined number C2 of combinations.

[0033] The image processing unit 13 performs the following process for each combination to generate differential image data for that combination. The image processing unit 13 reads the luminance values ​​of pixels at the same position in the two original image data included in a given combination, and uses the absolute value of the difference between the two read luminance values ​​as the luminance value of the pixel at that position in the differential image data. The image processing unit 13 performs this process for all pixels to generate differential image data corresponding to that combination (Sc4).

[0034] The image processing unit 13 performs a process of binarizing the luminance values ​​of the pixels in each of the generated differential image data for each combination. The image processing unit 13 compares the luminance values ​​of all pixels in the differential image data with a predetermined threshold, and performs a binarization process in which, for example, if the luminance value is equal to or greater than the threshold, the luminance value of the pixel is set to "1," and if the luminance value is less than the threshold, the luminance value of the pixel is set to "0" (Sc5).

[0035] The image processing unit 13 integrates the luminance values ​​of all the binarized differential image data for each pixel as shown in the following equation (1), and calculates the luminance change frequency, which is the sum of the integrated luminance values ​​(Sc6).

[0036]

[0037] In equation (1), S(x, y) is the luminance change frequency of pixel (x, y) after the integration, and lt(x, y) is the luminance value of pixel (x, y) in the t-th differential image data. The image processing unit 13 detects the maximum value of the luminance change frequency S(x, y) in the generated luminance change frequency distribution indicating the luminance change frequencies of all pixels, and normalizes the luminance change frequency S(x, y) of each pixel in the luminance change frequency distribution so that the maximum value becomes a predetermined value, for example, "255." Specifically, the image processing unit 13 performs normalization by dividing the luminance change frequency S(x, y) of each pixel in the luminance change frequency distribution by the maximum value of the luminance change frequency S(x, y) in the luminance change frequency distribution and multiplying the result by a predetermined value (Sc7).

[0038] For example, if the predetermined value is "255," the pixel values ​​of the pixels included in the normalized luminance change frequency distribution will be between 0 and 255. If these pixel values ​​are considered to be luminance values, the normalized luminance change frequency distribution will be grayscale image data that includes the same number of pixels as the pixels in the original image data. Therefore, the normalized luminance change frequency distribution will be referred to as processed image data hereinafter.

[0039] The image processing unit 13 reads out the time information associated with the reference original image data stored in the internal storage area, associates the read out time information with the processed image data, and records the time information in the processed image data storage unit 14 (Sc8), thereby completing the processing of the subroutine.

[0040] The image shown in Fig. 6 is an example of an image represented by processed image data obtained when the original image data of Fig. 5 is used as the reference original image data. In Fig. 6, pixels having brightness values ​​shown in the area indicated by reference numeral 101 are pixels representing gas. It can be seen that the gas, which was unclear in the image of the original image data of Fig. 5 because of its small difference from the brightness value of the background, is emphasized and becomes clearer in the image of the processed image data shown in Fig. 6.

[0041] Returning to FIG. 3 , the image processing unit 13 determines whether two or more processed image data sets are stored in the processed image data storage unit 14 (Sb4). If the image processing unit 13 determines that two or more processed image data sets are not stored in the processed image data storage unit 14 (No in Sb4), it reads from the original image data storage unit 12 the original image data associated with time information indicating the time immediately after the time indicated by the time information stored in the internal storage area, and the time information associated with the original image data. The image processing unit 13 sets the read original image data as the reference original image data (Sb2-2). After determining the reference original image data, the image processing unit 13 again starts the image processing subroutine shown in FIG. 4, which is the process of Sb3.

[0042] If the image processing unit 13 determines that two or more processed image data are stored in the processed image data storage unit 14 (Sb4, Yes), it outputs the time information associated with the most recently processed image data recorded in the processed image data storage unit 14 to the vector generation unit 15 (Sb5).

[0043] The vector generation unit 15 acquires the time information output by the image processing unit 13. The vector generation unit 15 reads from the original image data storage unit 12 original image data corresponding to the acquired time information and original image data that is chronologically adjacent to the original image data and corresponds to a time earlier than the time indicated by the acquired time information. The vector generation unit 15 performs optical flow processing on the two read original image data to generate a first vector, which is a vector indicating the movement direction and movement speed of each pixel (Sb6). Note that the processing of Sb6 is performed via the processing of Sb1 and Sb4, and therefore, when processing Sb6 is performed, at least one piece of original image data earlier than the original image data corresponding to the time information is stored in the original image data storage unit 12.

[0044] The image shown in Fig. 7 is a diagram showing first vectors obtained by performing optical flow processing on the original image data shown in Fig. 5 and original image data adjacent to the original image data in chronological order, superimposed on the image of the original image data shown in Fig. 5. Note that the first vectors are obtained for each pixel, but in Fig. 7, the first vectors for each pixel spaced at regular intervals are shown to make the first vectors easier to see.

[0045] The vector generation unit 15 reads out the processed image data corresponding to the imported time information and the processed image data that is chronologically adjacent to the processed image data and corresponds to a time earlier than the time indicated by the imported time information from the processed image data storage unit 14. By reading out the processed image data in this way, the vector generation unit 15 reads out from the processed image data storage unit 14 two pieces of processed image data that are associated with the same time information as the time information of each of the two original image data that it read out in the process of Sb6, i.e., two pieces of processed image data that correspond to each of the two original image data.

[0046] The vector generation unit 15 performs the same optical flow processing as that performed in Sb6 on the two processed image data that have been read out, to generate a second vector, which is a vector indicating the movement direction and movement speed of each pixel.

[0047] The vector generation unit 15 associates each of the generated second vectors and each of the first vectors generated in the process of Sb6 with information indicating the position of the corresponding pixel, and outputs the result to the filter unit 16 (Sb7). Note that the process of Sb7 is performed after the process of Sb4, and therefore, when the process of Sb7 is performed, at least one or more pieces of processed image data prior to the processed image data corresponding to the time information are stored in the processed image data storage unit 14.

[0048] The image shown in Fig. 8 is a diagram showing second vectors obtained by performing optical flow processing on the processed image data shown in Fig. 6 and processed image data adjacent to the processed image data in chronological order, superimposed on the image of the processed image data shown in Fig. 6. Note that the second vectors are obtained for each pixel, but in Fig. 8, the second vectors for each pixel are shown spaced at regular intervals to make the second vectors easier to see. As can be seen from the images shown in Fig. 7 and Fig. 8, while Fig. 7 shows very few first vectors indicating the movement direction and movement speed of gas, Fig. 8 shows many second vectors indicating the movement direction and movement speed of gas.

[0049] The filter unit 16 receives the first vector and the second vector, which are associated with information indicating the pixel position and output by the vector generation unit 15. The filter unit 16 selects the first vector and the second vector whose pixels match, and performs a process of filtering the second vector using the first vector, for each pixel. For example, as shown in FIG. 9 , the first vector at a pixel (x, y) is the vector Vorg(x, y) indicated by the solid arrow 201, and the second vector at the same pixel (x, y) is the vector Vprocessed(x, y) indicated by the dashed arrow 202. Here, the angle formed between the vector Vorg(x, y) 201 and the vector Vprocessed(x, y) 202 is represented by "θ", where θ is an angle between 0° and 180°.

[0050] In this case, the filter unit 16 performs filtering to calculate a vector Vhistogram(x, y) 204, which is a vector Vhistogram(x, y) having a length calculated by the following equation (2) and having the same direction as the second vector, vector Vprocessed(x, y) 202. However, if the length calculated by equation (2) is a negative value, the direction of vector Vhistogram(x, y) 204 will be opposite to the direction of vector Vprocessed(x, y) 202.

[0051]

[0052] The filtering performed by the filter unit 16 can be explained using vectors as follows. As shown in the second term on the right side of equation (2), the filter unit 16 calculates a multiplication value by multiplying the norm of vector Vorg(x, y) 201, which is a first vector, by cos(θ / 2). As shown in FIG. 9 , the filter unit 16 determines a vector 203 having a length opposite to that of vector Vprocessed(x, y) 202, which is a second vector, using the calculated multiplication value. The filter unit 16 adds the determined vector 203 and vector Vprocessed(x, y) 202 to calculate vector Vhistogram(x, y) 204.

[0053] Since the gas is unclear in the original image data, the first vector shown in FIG. 7 is primarily a calculation target point, which is a point for calculating optical flow, and is a vector that indicates the direction and speed of movement of the calculation target point corresponding to a fixed object such as a structure. Note that fixed objects are not limited to structures installed in plants, etc., but also include objects such as soil, sand, and grass. The vector of the calculation target point corresponding to this fixed object is a vector that is generated by the imaging device 2 moving while capturing images. Therefore, many of the first vectors are vectors that depend on the direction and speed of movement of the imaging device 2.

[0054] In contrast, the second vector shown in Figure 8 is a vector that includes a vector indicating the movement direction and movement speed of a calculation target point corresponding to a fixed object and a vector indicating the movement direction and movement speed of a calculation target point corresponding to the diffusing gas. The direction and speed of gas diffusion are also affected by the pressure applied to the gas, the direction in which the gas is discharged, the strength and direction of the wind around the gas, etc. Therefore, the vector indicating the movement direction and movement speed of the gas in the second vector is a vector that is a combination of a vector that depends on the movement direction and movement speed of the image capture device 2 and a vector that is generated by influences other than the movement direction and movement speed of the image capture device 2.

[0055] The purpose of filtering performed by the filter unit 16 is to remove vectors of calculation points corresponding to fixed objects and to further enhance vectors of calculation points corresponding to gas. To achieve the former goal of removing vectors of calculation points corresponding to fixed objects, it is sufficient to subtract the first vector from the second vector of the same pixel. However, if a calculation point corresponding to a fixed object exists in a pixel in the first vector where gas is present, simply subtracting the first vector from the second vector has the disadvantage of shortening the length of the vector of the calculation point corresponding to gas. Therefore, in equation (2), the norm of the first vector, vector Vorg(x, y), is multiplied by cos(θ / 2).

[0056] When θ = 0°, the first vector and the second vector are pointing in the same direction, so it is assumed that both the first vector and the second vector are vectors of the calculation point corresponding to the fixed object. Since cos(θ / 2) is "1" when θ = 0°, the first vector is simply subtracted from the second vector. Therefore, the vector of the calculation point corresponding to the fixed object can be removed from the second vector.

[0057] It is estimated that as θ, the angle between the first vector and the second vector, increases, the vector element of the second vector at the calculation point corresponding to gas increases. cos(θ / 2) decreases as θ increases, becoming "0" when θ = 180°. Therefore, as the vector element of the second vector at the calculation point corresponding to gas increases, the influence of the first vector can be reduced, and the vector of the calculation point corresponding to gas included in the second vector can be emphasized. Therefore, the vector Vhistogram(x, y) 204 calculated for each pixel by the filter unit 16 can be said to be a vector extracted by removing vectors of calculation points corresponding to fixed objects from the second vector and emphasizing vectors of calculation points corresponding to gas.

[0058] In the following description, the vector of a calculation target point corresponding to a fixed object will be simply referred to as the vector corresponding to the fixed object, and the vector of a calculation target point corresponding to gas will be simply referred to as the vector corresponding to gas.

[0059] The image shown in Fig. 10 is an image in which the vector Vhistogram(x, y) 204 for each pixel calculated by the filter unit 16 by filtering the first vector shown in Fig. 7 and the second vector shown in Fig. 8 using equation (2) is superimposed on the image of the processed image data shown in Fig. 6. As can be seen from the image shown in Fig. 10, the filtering by the filter unit 16 removes vectors near areas where structures exist, and mainly extracts vectors near areas where gas exists.

[0060] The filter unit 16 associates information indicating the position of each pixel with the vector calculated for each pixel by filtering. Optical flow processing may result in large errors in detecting the speed of movement of pixels at the outer edge of the image, resulting in the calculation of unreliable vectors. Furthermore, in the image shown in FIG. 10 , the outer edge of the image may contain information indicating the index of the image, such as the area indicated by reference numeral 110. Therefore, the filter unit 16 excludes the vectors at the outer edge based on the information indicating the positions of the pixels associated with the vectors. The filter unit 16 outputs the vectors, from which the outer edge vectors have been excluded, to the frequency distribution generation unit 17 (Sb8).

[0061] The frequency distribution generator 17 takes in the vectors output by the filter unit 16. The frequency distribution generator 17 classifies the taken-in vectors into sections of predetermined vector lengths and counts the number of vectors for each section. The frequency distribution generator 17 calculates the frequency of each section as a ratio between the number of vectors in each section as the numerator and the total number of taken-in vectors as the denominator, and generates a frequency distribution based on the frequency of each section. The frequency distribution generator 17 generates a histogram from the generated frequency distribution data and displays the generated histogram on the display unit 19.

[0062] Fig. 11 is an example of a histogram displayed on the display unit 19 by the frequency distribution generating unit 17. In Fig. 11, the vertical axis represents the frequency, and the horizontal axis represents the interval between sections of vector lengths predetermined by the frequency distribution generating unit 17. Fig. 11 shows an example of sections in which the length of one section is "0.2", as the sections of vector lengths predetermined by the frequency distribution generating unit 17.

[0063] The frequency distribution generation unit 17 outputs the generated frequency distribution data to the determination unit 18 (Sb9). The determination unit 18 takes in the frequency distribution data output by the frequency distribution generation unit 17. The determination unit 18 has predetermined intervals that serve as indicators for determining whether or not a gas leak exists, and a threshold value indicated by reference numeral 300 in FIGS. 11(a) and 11(b) predefined. Here, the intervals that serve as indicators for determining whether or not a gas leak exists are intervals that are equal to or longer than the length of a vector that is predefined based on empirical values, and in this case, are intervals where the length of the vector is equal to or longer than "999.8".

[0064] The determination unit 18 detects the frequency of each section in which the vector length in the imported frequency distribution data is equal to or greater than "999.8" and determines whether any of the detected frequencies is equal to or greater than a threshold. If the determination unit 18 determines that any of the detected frequencies is equal to or greater than the threshold, the determination unit 18 displays, for example, the character string "Gas Leak Present" as the determination result, superimposed on the histogram displayed on the display unit 19. If the determination unit 18 determines that none of the detected frequencies is equal to or greater than the threshold, the determination unit 18 displays, for example, the character string "Gas Leak Not Present" as the determination result, superimposed on the histogram displayed on the display unit 19.

[0065] FIG. 11( a) is an example of a histogram for an image taken at a location where no gas leak is occurring, and FIG. 11( b) is an example of a histogram for an image taken at a location where a gas leak is occurring. In FIG. 11( a), the frequency of each section where the vector length is equal to or greater than "999.8" does not exceed the threshold indicated by the reference symbol 300. In contrast, in FIG. 11( b), the frequency of at least the section from "999.8 to 1000.0" exceeds the threshold indicated by the reference symbol 300. Therefore, in the case of FIG. 11( a), the determination unit 18 determines that none of the detection thresholds are equal to or greater than the threshold, and displays "No gas leak" on the display unit 19. In the case of FIG. 11( b), the determination unit 18 determines that one of the detection thresholds is equal to or greater than the threshold, and displays "Gas leak present" on the display unit 19.

[0066] After making the determination, the determination unit 18 outputs an instruction signal indicating that processing should continue to the image processing unit 13 (Sb10). When the image processing unit 13 receives the instruction signal indicating that processing should continue from the determination unit 18, it determines whether new original image data associated with time information later than the time information of the reference original image data stored in the internal storage area is stored in the original image data storage unit 12 (Sb11).

[0067] If the image processing unit 13 determines that new original image data is stored in the original image data storage unit 12 (Sb11, Yes), it performs the process of Sb2-2 again. If the image processing unit 13 determines that new original image data is not stored in the original image data storage unit 12 (Sb11, No), it ends the process.

[0068] (Operations and Effects) In the gas leak monitoring device 10 of the above-described embodiment, the image data acquisition unit 11 generates original image data by dividing frames of a video captured by the imaging device 2 while it is moving. The image processing unit 13 performs predetermined image processing on the original image data to emphasize gas, generating processed image data. The vector generation unit 15 selects two chronologically adjacent original image data and two processed image data corresponding to the two original image data, and performs predetermined image processing to calculate the movement direction and movement speed of an object included in the image for each combination of original image data and each combination of processed image data, thereby generating a first vector for each pixel and a second vector for each pixel. The filter unit 16 uses the corresponding first vector for each pixel to filter the corresponding second vector, thereby emphasizing and extracting a vector corresponding to gas. The frequency distribution generation unit 17 classifies the extracted vectors by section and generates a frequency distribution. The determination unit 18 determines the presence or absence of a gas leak for each image using the frequency of each predetermined section, for example, a section equal to or greater than the length of a predetermined vector, and a predetermined threshold.

[0069] Because the original image data shows gas indistinctly, the first vectors generated from the original image data are primarily vectors corresponding to fixed objects such as structures. In contrast, because the processed image data emphasizes the gas, the second vectors generated from the processed image data include vectors corresponding to fixed objects such as structures and vectors corresponding to the gas. By utilizing this difference between the first vectors generated from the original image data and the second vectors generated from the processed image data, the first vectors can be used to filter the second vectors, thereby emphasizing and extracting the vectors corresponding to the gas from the second vectors. Among the vectors extracted in this manner, the vectors corresponding to the gas are emphasized and therefore have a certain length. Therefore, a frequency distribution can be generated from the extracted vectors, and the presence or absence of a gas leak can be determined based on the frequency of sections in the generated frequency distribution that include vectors corresponding to the gas.

[0070] The technology disclosed in Patent Document 1 is based on the premise that gas is captured in the captured image, and therefore, in cases where gas is not captured in the captured image, the gas leak monitoring device 10 can more accurately indicate the presence or absence of a gas leak. In the gas leak monitoring device 10, three or more image data may be used in the predetermined image processing by the image processing unit 13 to enhance gas. However, in the processing from the vector generation unit 15 onwards, only two original image data and two processed image data are used as image data, and there is no need to increase the number of image data to improve accuracy, as with the technology disclosed in Patent Document 1. Therefore, the gas leak monitoring device 10 of the above embodiment can indicate the presence or absence of a gas leak more accurately and in a shorter time than the method disclosed in Patent Document 1.

[0071] (Other configuration examples of the embodiment) The embodiment of the present disclosure has been described above in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and also includes designs within the scope that do not deviate from the gist of the present disclosure.

[0072] (When the gas is clearly visible in the original image data) When the difference between the temperature of a fixed object such as a structure and the temperature of the gas is small, the gas will be indistinctly visible in the original image data generated by capturing images with the imaging device 2. However, it is possible to extract the vector corresponding to the gas by filtering the vector corresponding to the fixed object using the gas leak monitoring device 10 of the above embodiment. On the other hand, when the difference between the temperature of the fixed object and the temperature of the gas is large, the gas will be clearly visible in the original image data generated by capturing images with the imaging device 2.

[0073] In this case, even if the image processing unit 13 does not perform predetermined image processing for emphasizing gas, the first vector generated by the vector generation unit 15 from the original image data will also include a vector corresponding to gas. Therefore, the gas leak monitoring device 10 may not include the image processing unit 13, the processed image data storage unit 14, and the filter unit 16. In this case, steps Sb1 to Sb5 in FIG. 3 are not performed. When the vector generation unit 15 detects that the image data acquisition unit 11 has stored at least two or more pieces of original image data in the original image data storage unit 12, it performs step Sb6 in FIG. 3 to generate a first vector and skips step Sb7 for generating a second vector. Step Sb8 is also skipped, and the frequency distribution generation unit 17 takes in the first vector generated by the vector generation unit 15 instead of the vector calculated and output by the filter unit 16, performs step Sb9 on the taken-in first vector, and then performs step Sb10 and subsequent steps. The processing of Sb11 is performed by the vector generation unit 15, and if the result is "Yes", the processing of Sb6 will be performed on the original image data from the time one time later than the original image data that was processed in the previous Sb6.

[0074] However, the first vectors generated by the vector generation unit 15 include vectors corresponding to fixed objects and vectors corresponding to gas. Therefore, even if the first vectors are classified into sections of predetermined vector lengths and a frequency distribution is generated in which the number of vectors in each section is used as a frequency, the frequency of this frequency distribution will include vectors corresponding to fixed objects. Therefore, if the difference between the length of the vector corresponding to the fixed object and the length of the vector corresponding to the gas is small, it becomes difficult to determine whether or not a gas leak has occurred.

[0075] The moving speed of a fixed object within an image is approximately equal to the moving speed of the image capture device 2. Therefore, if the moving speed of the image capture device 2 is constant, it is possible to identify the range of the section of the frequency distribution corresponding to that moving speed. Therefore, the determination unit 18 can determine the presence or absence of a gas leak for each image by using, for example, the frequency of each section excluding the range of the identified section and having a length equal to or greater than a predetermined vector length, and a predetermined threshold value.

[0076] Furthermore, even if the moving speed of the imaging device 2 is not constant, if there is a large difference between the moving speed of the imaging device 2 and the speed at which the gas diffuses, the difference between the length of the vector corresponding to the fixed object and the length of the vector corresponding to the gas will be large. Therefore, in the frequency distribution generated by the frequency distribution generating unit 17, there will be a large difference between the section in which the vector corresponding to the fixed object is shown and the section in which the vector corresponding to the gas is shown. In this case, if it is possible to identify the range of the section in which the vector corresponding to the gas is shown, the determining unit 18 can determine the presence or absence of gas leakage for each image using the frequency of each section within the range of the identified section and a predetermined threshold.

[0077] Therefore, when gas is clearly visible in the original image data, there is no need to perform predetermined image processing to enhance the gas by the image processing unit 13, and the time required for this processing is reduced. In the processing performed by the vector generation unit 15 and subsequent units, only two original image data are used as image data, and there is no need to increase the number of image data to improve accuracy, as in the technique disclosed in Patent Document 1. Therefore, even when gas is clearly visible in the original image data, the presence or absence of a gas leak can be indicated more accurately and in a shorter time than the technique disclosed in Patent Document 1.

[0078] (When the Imaging Device is Fixed (Part 1)) In the above embodiment, the imaging device 2 is mounted on, for example, a UAV and moved. In contrast, when the imaging device 2 is fixed and images are captured, such as for fixed-point observation, fixed objects such as structures do not move within the image unless the imaging device 2 is swayed by the wind, and therefore, even if optical flow processing is performed, no vectors are generated. Therefore, in this case, the gas leak monitoring device 10 does not need to include the filter unit 16, and the vector generation unit 15 does not need to generate the first vector. Instead of the vectors output by the filter unit 16, the second vectors generated by the vector generation unit 15 are classified into sections of predetermined vector lengths, and the frequency distribution generation unit 17 generates a frequency distribution in which the number of vectors for each section is used as a frequency. From this frequency distribution, the determination unit 18 can determine whether or not a gas leak exists for each image. In this case, only two processed image data are used as image data in the processing after the vector generation unit 15, and there is no need to increase the number of image data to improve accuracy, as in the technique disclosed in Patent Document 1. Therefore, even if the imaging device 2 is fixed, it is possible to indicate the presence or absence of a gas leak with higher accuracy and in a shorter time than the technique disclosed in Patent Document 1.

[0079] (Case 2: Imaging Device is Fixed) Assume that the imaging device 2 is fixed, and furthermore, there is a large difference in temperature between the gas and the temperature of the fixed object, such as a structure, and that the gas is clearly visible in the original image data generated by the imaging device 2. In this case, the gas leak monitoring device 10 does not need to include the image processing unit 13, the processed image data storage unit 14, and the filter unit 16, and the vector generation unit 15 may generate the first vector from the original image data stored in the original image data storage unit 12, but not the second vector. The first vector will mainly include vectors corresponding to the gas. Therefore, instead of the second vector, the first vector may be classified into sections of a predetermined vector length, and the frequency distribution generation unit 17 may generate a frequency distribution in which the number of vectors in each section is the frequency. From this frequency distribution, the determination unit 18 can determine whether or not there is a gas leak for each image.

[0080] If gas is clearly visible in the original image data, there is no need to perform predetermined image processing to enhance the gas by the image processing unit 13, thereby reducing the time required for this processing. In the processing performed by the vector generation unit 15 and subsequent processes, only two original image data are used as image data, and there is no need to increase the number of image data to improve accuracy, as in the technique disclosed in Patent Document 1. Therefore, even if the imaging device 2 is fixed and gas is clearly visible in the original image data, the presence or absence of a gas leak can be indicated more accurately and in a shorter time than the technique disclosed in Patent Document 1.

[0081] (Configuration Example Without Determination Unit) In the above embodiment, the frequency distribution generation unit 17 generates a frequency distribution, and then the determination unit 18 determines the presence or absence of a gas leak using the generated frequency distribution. Alternatively, the determination unit 18 may be omitted, and a person may determine the presence or absence of a gas leak by comparing the frequency of each predetermined section, for example, a section equal to or greater than a predetermined vector length, in the histogram displayed on the display unit 19 by the frequency distribution generation unit 17. In this case, a person may determine the presence or absence of a gas leak by comparing the frequency with a threshold, or may determine the presence or absence of a gas leak from the difference in the relative magnitude of the frequency of each section. Therefore, when determining the presence or absence of a gas leak from the difference in the relative magnitude of the frequency of each section, the frequency distribution generation unit 17 may display the histogram using the number of vectors in each section as the frequency, rather than calculating the ratio of the number of vectors in each section as the numerator and the total number of imported vectors as the denominator.

[0082] (Other Configuration Examples) In the above embodiment, the image data acquisition unit 11 associates the original image data with time information associated with the frame from which the original image data was generated and records the original image data in the original image data storage unit 12. Alternatively, instead of associating time information with the original image data, the image data acquisition unit 11 may assign consecutive numbers, starting from 1, in the order in which the original image data was divided and record the assigned numbers in the original image data storage unit 12. In this case, the subsequent processing by the image processing unit 13 and the vector generation unit 15 uses the assigned numbers instead of time information. Specifically, in the processing of Sc2 in FIG. 4, the image processing unit 13 identifies the original image data to be used to generate the processed image data based on the assigned numbers and reads them from the original image data storage unit 12. In the processing of Sb6 and Sb7 in FIG. 3, the vector generation unit 15 reads the original image data and processed image data, to which the numbers received from the image processing unit 13 are assigned, from the original image data storage unit 12 and the processed image data storage unit 14, respectively.

[0083] In the above embodiment, the imaging device 2 is configured to capture video. Alternatively, the imaging device 2 may capture still images at different times, associate time information indicating the time of capture with the still image data, and transmit the still image data to the image data acquisition unit 11. However, the interval between the different capture times must be long enough to allow optical flow processing to generate vectors corresponding to objects moving within the image, and it is preferable that the capture times be consecutive, as in video capture. Note that instead of the imaging device 2 associating time information with still image data, as described above, consecutive numbers starting from 1 may be associated with the image data in the order in which the image data acquisition unit 11 receives them.

[0084] In the above embodiment, the predetermined image processing for emphasizing gas performed by the image processing unit 13 is not limited to the image processing shown in Fig. 4, and may be, for example, image processing in which, of the brightness values ​​of the original grayscale image data from 0 to 255, the brightness values ​​in the range of gas brightness, for example, 100 to 150, are left as they are, and the brightness values ​​of the remaining pixels are set to "0" or "255." Alternatively, image processing may be used that adjusts the contrast so that the gas is emphasized.

[0085] In the above embodiment, the vector generation unit 15 selects two pieces of original image data that are adjacent in chronological order from the original image data storage unit 12 and selects two pieces of original image data that are adjacent in chronological order from the processed image data storage unit 14. However, the vector generation unit 15 does not necessarily have to select adjacent pieces. For example, if a sufficient number of pieces of original image data are stored in the original image data storage unit 12 and a sufficient number of pieces of processed image data are stored in the processed image data storage unit 14, the vector generation unit 15 may select one piece of original image data corresponding to the time information output by the image processing unit 13, and then select original image data that is a predetermined time before the selected piece of original image data as the other piece of original image data. In this case, the vector generation unit 15 selects two pieces of processed image data from the processed image data storage unit 14 that are associated with the time information of each of the two selected pieces of original image data. Note that the predetermined time may be set in advance or may be changed for each selection. However, it is preferable that the predetermined time be set so that the shooting ranges of the two pieces of original image data do not differ significantly.

[0086] In the above embodiment, the vector generation unit 15 acquires the time information output by the image processing unit 13 in the process of Sb5 in Fig. 3 , and selects and reads the original image data and processed image data corresponding to the acquired time information in the processes of Sb6 and Sb7. On the other hand, while the range captured by the imaging device 2 does not change significantly, i.e., while there is almost no change in the position of fixed objects such as structures, the original image data may not be selected based on the time information. For example, after the vector generation unit 15 performs the process of Sb6 once to calculate the first vector, the vector generation unit 15 may skip the process of Sb6 for a number of times corresponding to the period during which the range captured by the imaging device 2 does not change significantly, and when outputting the first vector to the filter unit 16 in the process of Sb7, the most recently generated first vector may be output.

[0087] In the above embodiment, optical flow processing by the Gunnar-Farneback method, which is one of the flow estimation methods, is shown as an example of the predetermined image processing performed by the vector generation unit 15 to calculate the moving direction and moving speed of an object included in an image. However, any optical flow method may be applied, or processing other than optical flow may be applied.

[0088] In the above embodiment, the filter unit 16 performs filtering using equation (2). However, instead of cos(θ / 2) in equation (2), a function other than cos(θ / 2) may be applied that calculates a proportion that is 100% when the angle θ formed by the second vector and the first vector whose pixel position is the same as that of the second vector is 0°, and whose percentage value approaches 0% as the angle θ approaches 180°.

[0089] In the above embodiment, the gas is CO2 gas, but it may be a gas such as methane or ammonia that has an infrared absorption wavelength band, or a gas that has an absorption wavelength band outside the infrared region. When a gas other than CO2 gas is to be photographed, the imaging device 2 is provided with a filter that transmits the absorption wavelength band of the gas.

[0090] In the above embodiment, when generating differential image data in the process of Sc4 in Fig. 4, if the shooting ranges of the two original image data included in the combination are different, the image processing unit 13 may align the shooting ranges so that the shooting ranges match before generating differential image data. The alignment method may be a method of detecting common calculation target points in the two original image data and changing the coordinates so that the positions of the common calculation target points match, or another method may be applied.

[0091] In the above embodiment, the image data acquisition unit 11 performs noise removal processing and grayscale conversion processing in the process of Sa2 in Fig. 2 , but noise removal processing may not be performed if the image captured by the imaging device 2 has little noise, and grayscale conversion may not be performed if the image captured by the imaging device 2 is grayscale rather than color. Also, in the process of Sa2 in Fig. 2 , the image data acquisition unit 11 converts the image data into grayscale image data to preserve the luminance of the image data and generate the original image data, but the image data may also be converted into grayscale image data to preserve the lightness of the image data and generate the original image data. In this case, "luminance" in the above embodiment would be read as "lightness."

[0092] In the above embodiment, the image processing unit 13 performs the process of binarizing the differential image data in the process of Sc5 in Fig. 4, but the binarization process may not be performed. In this case, the image processing unit 13 performs the process of Sc6 on the differential image data generated in the process of Sc4, that is, performs the process of generating processed image data by accumulating the luminance value for each pixel of the differential image data generated in the process of Sc4, and then performs the processes of Sc7 and Sc8.

[0093] In the above embodiment, the filter unit 16 is configured to exclude vectors on the outer edge of the image, but if the reliability of the vectors on the outer edge of the image is high, it is also possible not to exclude the vectors on the outer edge of the image.

[0094] In the above embodiment, the vector generation unit 15 generates the first vector and then the second vector, as shown in the processes of Sb6 and Sb7 in FIG. 3 . However, conversely, the second vector may be generated first and then the first vector, or the process of generating the first vector and the process of generating the second vector may be performed in parallel.

[0095] In the above embodiment, the determination unit 18 determines the presence or absence of gas leakage for each image using the frequency of each section that is equal to or greater than a predetermined vector length and a predetermined threshold value. On the other hand, if the length of the vector corresponding to the gas is known empirically, the determination unit 18 may determine the presence or absence of gas leakage for each image by predetermining a section that includes the length of the vector corresponding to the gas, rather than using a section that is equal to or greater than the predetermined vector length, and using the frequency of each section that is predetermining the predetermined threshold value.

[0096] In the above embodiment, the image processing unit 13 receives an instruction signal from the determination unit 18 indicating that processing should continue, and performs the processing of Sb11 in Fig. 3. Alternatively, the determination unit 18 may not output an instruction signal indicating that processing should continue, and the image processing unit 13 may perform the processing of Sb5 and then the processing of Sb11.

[0097] In the above embodiment, in the process of Sc5 in Fig. 4 and the process of Sb10 in Fig. 3, the image processing unit 13 and the determination unit 18 perform a determination process using a threshold value to determine whether the target value is equal to or greater than the threshold value. In contrast, in the processes of Sc5 and Sb10, depending on how the threshold value is defined, the process may be such that the target value exceeds the threshold value.

[0098] (Computer Configuration) FIG. 12 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 90 includes a processor 91, a main memory 92, a storage 93, and an interface 94. The gas leak monitoring device 10 described above is implemented on the computer 90. The operations of the above-described processing units, i.e., the image data acquisition unit 11, the image processing unit 13, the vector generation unit 15, the filter unit 16, the frequency distribution generation unit 17, and the determination unit 18, are stored in the storage 93 in the form of a program. The processor 91 reads the program from the storage 93, loads it into the main memory 92, and executes the above-described processing in accordance with the program. The processor 91 also allocates storage areas in the main memory 92 or the storage 93 corresponding to the above-described original image data storage unit 12 and processed image data storage unit 14 in accordance with the program. The display unit 19 is connected via the interface 94. Therefore, the display unit 19 may or may not be a component of the computer 90, i.e., a component of the gas leak monitoring device 10 as described above.

[0099] The program may be for realizing some of the functions to be performed by the computer 90. For example, the program may be combined with other programs already stored in the storage 93 or other programs implemented in other devices to perform the functions. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0100] Examples of storage 93 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. Storage 93 may be an internal medium directly connected to the bus of computer 90, or an external medium connected to computer 90 via interface 94 or a communication line. Furthermore, if this program is distributed to computer 90 via a communication line, computer 90 that receives the program may load the program into main memory 92 and execute the above-described processing. In at least one embodiment, storage 93 is a non-transitory tangible storage medium.

[0101] <Additional Notes> The gas leakage monitoring device 10 according to the embodiment of the present disclosure can be understood, for example, as follows.

[0102] (1) A gas leak monitoring device 10 according to a first aspect includes an image data acquisition unit 11 that acquires image data generated by photographing at different times as original image data, a vector generation unit 15 that selects two original image data items in chronological order from the original image data and performs predetermined image processing on the two selected original image data items to calculate the direction and speed of movement of an object included in the image, thereby generating vectors indicating the direction and speed of movement of each pixel included in the original image data, and a frequency distribution generation unit 17 that classifies the vectors generated by the vector generation unit into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring. According to this aspect and each of the following aspects, the presence or absence of a gas leak can be indicated accurately and in a shorter time.

[0103] (2) A gas leak monitoring device 10 according to a second aspect is the gas leak monitoring device 10 of (1), and includes an image processing unit 13 that performs predetermined image processing on the original image data to enhance gas and generate processed image data, and the vector generation unit uses the processed image data instead of the original image data to generate vectors indicating the direction and speed of movement for each pixel included in the processed image data. According to this aspect, even if gas is not clearly visible in the original image data, a vector corresponding to the gas can be generated from the processed image data in which the gas is enhanced.

[0104] (3) A gas leak monitoring device 10 according to a third aspect is the gas leak monitoring device 10 of (1), and includes an image processing unit 13 that performs predetermined image processing on the original image data to emphasize gas and generate processed image data, and a filter unit 16 that emphasizes and extracts vectors corresponding to gas, wherein the vector generation unit defines the vector generated for each pixel included in the original image data as a first vector, and further selects two processed image data from the processed image data in chronological order and performs predetermined image processing on the two selected processed image data to calculate the movement direction and movement speed of an object included in the image, thereby generating vectors indicating the movement direction and movement speed for each pixel included in the processed image data to be defined as second vectors, the filter unit filters each of the second vectors for each pixel using the first vectors whose pixels match each of the second vectors, and the frequency distribution generation unit generates a frequency distribution for gas leak monitoring using the vectors extracted by the filter unit instead of the vectors generated by the vector generation unit. According to this aspect, even if the imaging device that takes the photograph is moving and the gas is not clearly visible in the original image data, processed image data that emphasizes the gas is generated, and filtering is performed taking into account the difference between the first vector generated from the original image data and the second vector generated from the processed image data, thereby emphasizing and extracting the vector corresponding to the gas, and the extracted vector can be used to indicate whether or not there is a gas leak.

[0105] (4) A fourth aspect of the gas leak monitoring device 10 is the gas leak monitoring device 10 of (3), wherein the vector generation unit, when selecting the two original image data, selects two original image data that are adjacent in chronological order, and when selecting the two processed image data, selects processed image data that correspond to each of the two selected original image data. According to this aspect, the two original image data from which the first vector is generated are adjacent in chronological order, and the two processed image data from which the second vector is generated are processed image data that correspond to each of the two original image data and are adjacent in chronological order. In this way, by using image data that are adjacent in chronological order, the accuracy of the first vector and the second vector is improved. Furthermore, by using processed image data that correspond to each of the two original image data, the shooting ranges of the original image data and the processed image data match, allowing the vector corresponding to the gas to be extracted with higher accuracy.

[0106] (5) A gas leak monitoring device 10 according to a fifth aspect is the gas leak monitoring device 10 of (3) or (4), wherein the filter unit performs filtering by adding a vector having a length obtained by multiplying the length of the first vector by a percentage that is 100% when the angle formed between the second vector and the first vector, the percentage value of which approaches 0% as the angle approaches 180°, and whose direction is opposite to that of the second vector. According to this aspect, the first vector can be used to exclude vectors corresponding to fixed objects such as structures from the second vector, while emphasizing vectors corresponding to gas.

[0107] (6) A gas leak monitoring device 10 according to a sixth aspect is the gas leak monitoring device 10 of any one of (2) to (5), wherein the image processing unit performs predetermined image processing to emphasize the gas by selecting two chronologically successive combinations of the original image data to be processed from the original image data to be processed and a plurality of the original image data acquired by the image data acquisition unit at a time earlier than the original image data, generating differential image data for each selected combination, and integrating pixel values ​​of the generated differential image data for each pixel to generate the processed image data corresponding to the original image data to be processed. According to this aspect, even if the gas is unclearly depicted in the original image data, processed image data in which the gas is emphasized can be generated.

[0108] (7) A gas leak monitoring device 10 according to a seventh aspect is the gas leak monitoring device 10 of any one of (1) to (6), further comprising a determination unit 18 that determines the presence or absence of a gas leak, wherein the frequency distribution generation unit generates the frequency distribution in which the number of the vectors for each interval is used as the numerator and the ratio of the total number of the vectors is used as the denominator, and the determination unit determines the presence or absence of a gas leak based on the frequency of the interval, which is set in advance, and a predetermined threshold value. According to this aspect, the determination unit 18 can determine the presence or absence of a gas leak from the frequency distribution data generated by the frequency distribution generation unit 17, without the need for a person to determine the presence or absence of a gas leak by referring to a histogram.

[0109] (8) The gas leak monitoring device 10 according to an eighth aspect is the gas leak monitoring device 10 of any one of (1) to (7), wherein the filter unit excludes vectors corresponding to pixels on the outer edge of the image data from the vectors obtained after emphasizing and extracting vectors corresponding to gas. According to this aspect, the vectors used in generating a frequency distribution can be narrowed down to highly reliable vectors, thereby improving the accuracy of the frequency distribution and enabling the presence or absence of a gas leak to be accurately determined.

[0110] According to the gas leakage monitoring device, gas leakage monitoring method, and program of the present disclosure, it is possible to indicate the presence or absence of a gas leakage with high accuracy and in a shorter time.

[0111] DESCRIPTION OF SYMBOLS 1... Gas leak monitoring system 2... Imaging device 3... Communication network 10... Gas leak monitoring device 11... Image data acquisition section 12... Original image data storage section 13... Image processing section 14... Processed image data storage section 15... Vector generation section 16... Filter section 17... Frequency distribution generation section 18... Determination section 19... Display section

Claims

1. A gas leak monitoring device comprising: an image data acquisition unit that acquires image data generated by photographing at different times as original image data; a vector generation unit that selects two original image data items in chronological order from the original image data, and performs a predetermined image processing on the two selected original image data items to calculate a moving direction and moving speed of an object included in the image, thereby generating vectors indicating a moving direction and moving speed for each pixel included in the original image data; and a frequency distribution generation unit that classifies the vectors generated by the vector generation unit into predetermined vector length intervals, and generates a frequency distribution for gas leak monitoring.

2. A gas leak monitoring device as described in claim 1, further comprising an image processing unit which performs predetermined image processing on the original image data to emphasize gas and generate processed image data, and wherein the vector generating unit uses the processed image data instead of the original image data to generate vectors indicating a direction and speed of movement for each pixel contained in the processed image data.

3. A gas leakage monitoring device as claimed in claim 1, comprising: an image processing unit which performs predetermined image processing on the original image data to emphasize gas and generate processed image data; and a filter unit which emphasizes and extracts vectors corresponding to gas, wherein the vector generation unit: defines the vector generated for each pixel included in the original image data as a first vector, and further selects two processed image data in chronological order from the processed image data, and performs predetermined image processing on the two selected processed image data to calculate a moving direction and moving speed of an object included in the image, thereby generating vectors indicating a moving direction and moving speed for each pixel included in the processed image data as a second vector; the filter unit: filters each of the second vectors for each pixel using the first vector whose pixels match each of the second vectors; and the frequency distribution generation unit: generates the frequency distribution for gas leakage monitoring by using the vector extracted by the filter unit instead of the vector generated by the vector generation unit.

4. The gas leak monitoring device of claim 3, wherein the vector generation unit, when selecting the two original image data, selects two pieces of original image data that are adjacent in chronological order, and, when selecting the two processed image data, selects processed image data corresponding to each of the two selected original image data.

5. The gas leakage monitoring device of claim 3, wherein the filter unit performs filtering by adding the second vector to a vector having a length obtained by multiplying the length of the first vector by a rate that is 100% when the angle between the second vector and the first vector, which has the same pixel position as the second vector, and whose percentage value approaches 0% as the angle approaches 180°, and whose direction is opposite to that of the second vector.

6. A gas leakage monitoring device as claimed in any one of claims 2 to 5, wherein the image processing unit performs predetermined image processing to emphasize the gas by selecting a combination of two original image data that are successive in chronological order from the original image data to be processed and a plurality of original image data acquired by the image data acquisition unit at a time earlier than the original image data, generating differential image data for each selected combination, and accumulating pixel values ​​of the generated differential image data for each pixel, thereby generating the processed image data corresponding to the original image data to be processed.

7. A gas leakage monitoring device as described in any one of claims 1 to 5, further comprising a judgment unit that judges whether or not a gas leakage exists, wherein the frequency distribution generation unit generates the frequency distribution in which the number of the vectors for each of the sections is used as the numerator and the ratio of the total number of the vectors is used as the denominator, and the judgment unit judges whether or not a gas leakage exists based on the predetermined frequency of the sections and a predetermined threshold value.

8. A gas leakage monitoring device as claimed in any one of claims 3 to 5, wherein the filter unit excludes vectors corresponding to pixels on the outer edge of the image data from the vectors obtained after emphasizing and extracting vectors corresponding to gas.

9. A gas leak monitoring method comprising the steps of: acquiring image data generated by photographing at different times as original image data; selecting two pieces of original image data in chronological order from the acquired original image data, and performing a predetermined image processing on the two selected original image data to calculate a moving direction and moving speed of an object contained in the image, thereby generating vectors indicating a moving direction and moving speed for each pixel contained in the original image data; and classifying the generated vectors into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring.

10. A program that causes a computer to execute the steps of: acquiring image data generated by photographing at different times as original image data; selecting two pieces of original image data in chronological order from the acquired original image data, and performing a predetermined image processing on the two selected original image data to calculate the moving direction and moving speed of an object contained in the image, thereby generating vectors indicating the moving direction and moving speed for each pixel contained in the original image data; and classifying the generated vectors into intervals of predetermined vector lengths, to generate a frequency distribution for gas leak monitoring.

Images