Gas leakage monitoring device, gas leakage monitoring method, and program

Abstract

To provide a gas leakage monitoring device, a gas leakage monitoring method, and a program capable of indicating presence or absence of gas leakage accurately in a further short period of time.SOLUTION: A gas leakage monitoring device is provided with: an image data acquisition unit which acquires image data, as original image data, generated by being photographed at different time points; a vector generating unit which selects two pieces of original image data in a time series order from the original image data, and generates a vector indicating a moving direction and a moving speed for each pixel contained in the original image data by executing a prescribed image process for calculating the moving direction and moving speed of an object contained in the image to the two pieces of selected original image data; and a frequency distribution generation unit which generates a frequency distribution for gas leakage monitoring by classifying the vector generated by the vector generating unit for each segment of a previously determined length of a vector length.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a gas leakage monitoring device, a gas leakage monitoring method, and a program.

Background Art

[0002] Gases such as CO2 gas each have a specific absorption wavelength band, and a method for detecting gas leakage using the characteristics of this absorption wavelength band is known (see, for example, Patent Document 1). For example, CO2 gas has an absorption wavelength band near 4.3 μm. Therefore, by taking a photograph with an infrared camera equipped with a filter that transmits infrared rays of 4.3 μm, the following phenomenon occurs. When CO2 gas is present in a part of the region between the object to be photographed and the infrared camera, there is an intensity difference between the electromagnetic wave radiated from the object to be photographed and directly reaching the infrared camera and the electromagnetic wave passing through the CO2 gas and reaching the infrared camera due to absorption by the CO2 gas. This intensity difference of the electromagnetic wave appears as a difference in luminance values in the image data generated by photographing with the infrared camera, and it becomes possible to visualize the CO2 gas.

[0003] As a method for detecting the CO2 gas visualized in the image data, for example, Patent Document 1 discloses the following method. In the image data generated by photographing with an infrared camera, there is a difference in luminance values as described above. Using this difference in luminance values, a method is to detect, as pixels on which the gas is imaged, pixels whose change amount of luminance values per unit time is equal to or less than a preset threshold value.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Incidentally, it is desired that the determination of the presence or absence of gas leakage be performed with high accuracy in as short a time as possible. From this perspective, the process of observing the amount of change in the luminance value per unit time for each pixel, which is included in the method disclosed in Patent Document 1, is considered to have the following problems. That is, in order to improve the accuracy of the amount of change in the luminance value per unit time, it is necessary to increase the number of captured images. On the other hand, when the number of images is increased, there is a problem that the processing time becomes long.

[0006] The present disclosure has been made to solve the above problems, and an object thereof is to provide a gas leakage monitoring device, a gas leakage monitoring method, and a program that can accurately indicate the presence or absence of gas leakage in a shorter time.

Means for Solving the Problems

[0007] In order to solve the above problems, a gas leakage monitoring device according to the present disclosure includes an image data acquisition unit that acquires, as original image data, image data generated by being captured at different times, and selects two pieces of original image data in time series order from the original image data, and performs predetermined image processing for calculating the moving direction and moving speed of an object included in the image on the two pieces of original image data to be selected, thereby generating a vector indicating the moving direction and moving speed for each pixel included in the original image data. A vector generation unit, and a frequency distribution generation unit that classifies the vectors generated by the vector generation unit for each interval of a predetermined vector length to generate a frequency distribution for gas leakage monitoring.

[0008] The gas leakage monitoring method according to the present disclosure includes the steps of: obtaining, as original image data, image data generated by being photographed at different times; selecting two pieces of original image data in chronological order from the obtained original image data, and performing predetermined image processing for calculating the moving direction and moving speed of an object included in the image on the selected two pieces of original image data, thereby generating, for each pixel included in the original image data, a vector indicating the moving direction and moving speed; and classifying the generated vectors for each section of a predetermined vector length interval to generate a frequency distribution for gas leakage monitoring.

[0009] The program according to the present disclosure causes a computer to execute the steps of: obtaining, as original image data, image data generated by being photographed at different times; selecting two pieces of original image data in chronological order from the obtained original image data, and performing predetermined image processing for calculating the moving direction and moving speed of an object included in the image on the selected two pieces of original image data, thereby generating, for each pixel included in the original image data, a vector indicating the moving direction and moving speed; and classifying the generated vectors for each section of a predetermined vector length interval to generate a frequency distribution for gas leakage monitoring.

Advantages of the Invention

[0010] According to the gas leakage monitoring device, gas leakage monitoring method, and program of the present disclosure, it is possible to accurately indicate the presence or absence of gas leakage in a shorter time.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0012] Hereinafter, a gas leakage monitoring device, a gas leakage monitoring method, and a program according to embodiments of the present disclosure will be described with reference to the respective drawings. In the respective drawings, the same or corresponding configurations are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0013] (System Configuration) FIG. 1 is a block diagram showing the configuration of a gas leakage monitoring system 1 according to an embodiment of the present disclosure. The gas leakage monitoring system 1 includes an imaging device 2, a communication network 3, and a gas leakage monitoring device 10. In the gas leakage monitoring system 1, the gas to be monitored is assumed to be CO2 gas. The communication network 3 is, for example, a communication network operated by a communication carrier, and is a communication network that can be connected by wire and wirelessly.

[0014] The imaging device 2 is, for example, an infrared camera including a filter that transmits infrared rays of 4.3 μm, which is an absorption wavelength band of CO2 gas. The object to be photographed by the imaging device 2 is, for example, a field where a plant or the like is installed. The imaging device 2 is mounted on, for example, an unmanned aerial vehicle (hereinafter referred to as UAV), and while moving together with the UAV, shoots a video from above toward the field. The imaging device 2 includes a communication device that wirelessly connects to the communication network 3, and transmits the video data generated by shooting to the gas leakage monitoring device 10 via the communication network 3.

[0015] (Configuration of Gas Leakage Monitoring Device) The gas leakage monitoring device 10 can be configured using, for example, a computer such as a server, a personal computer, or a microcomputer, and peripheral devices of the computer. As a functional configuration composed of a combination of hardware such as the computer and software such as a program executed by the computer, it includes an image data acquisition unit 11, an original image data storage unit 12, an image processing unit 13, a processed image data storage unit 14, a vector generation unit 15, a filter unit 16, a frequency distribution generation unit 17, a determination unit 18, and a display unit 19.

[0016] The image data acquisition unit 11 is connected to the communication network 3 by wire or wirelessly, and receives the video data transmitted by the imaging device 2. The video data received by the image data acquisition unit 11 includes frames arranged in the order of shooting, and time information indicating the shooting time is associated with each frame. The image data acquisition unit 11 divides the frames included in the received video data in chronological order. The image data acquisition unit 11 generates original image data from each of the divided frames, associates the time information associated with the frame corresponding to the generated original image data with the generated original image data, and records it in the original image data storage unit 12.

[0017] The image processing unit 13 performs predetermined image processing for emphasizing the gas on the original image data recorded in the original image data storage unit 12, and generates processed image data corresponding to the original image data. The image processing unit 13 associates the time information associated with the original image data corresponding to the generated processed image data with the generated processed image data, and records it in the processed image data storage unit 14.

[0018] The vector generation unit 15 selects two adjacent original image data in chronological order in the original image data storage unit 12. The vector generation unit 15 selects two pieces of processed image data corresponding to each of the two pieces of original image data to be selected. Here, the two pieces of processed image data corresponding to each of the two pieces of original image data are specifically two pieces of processed image data associated with the same time information as the time information of each of the two pieces of original image data, and the two pieces of processed image data are also adjacent processed image data in chronological order.

[0019] The vector generation unit 15 performs predetermined image processing for calculating the moving direction and moving speed of the object included in the image on the two selected original image data, and generates a vector indicating the moving direction and moving speed (hereinafter, this vector is referred to as the first vector) for each pixel. The vector generation unit 15 performs predetermined image processing for calculating the moving direction and moving speed of the object included in the image on the two selected processed image data, and generates a vector indicating the moving direction and moving speed (hereinafter, this vector is referred to as the second vector) for each pixel.

[0020] As the predetermined image processing for calculating the moving direction and moving speed of the object included in the image, for example, the optical flow processing of the Gunnar Farneback method among the flow estimation methods is applied.

[0021] The filter unit 16 performs a process of emphasizing and extracting a vector corresponding to gas, which is called filtering, for each of the second vectors for each pixel using the first vector that matches the pixel. The frequency distribution generation unit 17 classifies the vectors for each pixel output by the filter unit 16 for each section of a predetermined vector length and generates a frequency distribution. The determination unit 18 determines the presence or absence of gas leakage based on the frequency of a predetermined section in the frequency distribution generated by the frequency distribution generation unit 17 and a predetermined threshold value.

[0022] The display unit 19 is, for example, a display device such as a liquid crystal display, and displays the frequency distribution generated by the frequency distribution generation unit 17 in a histogram or displays the determination result of the determination unit 18.

[0023] (Operation example of gas leakage monitoring device) An operation example of the gas leakage monitoring device 10 will be described with reference to the flowcharts of FIGS. 2 to 4.

[0024] As shown in FIG. 2, the image data acquisition unit 11 receives the moving image data transmitted by the imaging device 2 (Sa1). While continuously receiving the moving image data transmitted by the imaging device 2, the image data acquisition unit 11 sequentially performs division in order from the most recently received frame, and generates image data from the divided frames. The image data acquisition unit 11 performs noise removal processing on the generated image data. Here, the moving image data captured by the imaging device 2 is color moving image data, and the image data acquisition unit 11 converts the image data into grayscale image data so as to save the luminance of the noise-removed image data, and generates original image data. The pixel value of each pixel of the original image data is a value representing a luminance value with any value from 0 to 255 (Sa2).

[0025] The image shown in FIG. 5 is an example of the image shown by the original image data. The object indicated by reference numeral 100 in FIG. 5 is, for example, a structure fixedly installed on a plant or the like.

[0026] The image data acquisition unit 11 associates the time information associated with the frame that is the generation source of the original image data with the original image data, and records it in the original image data storage unit 12 (Sa3). The image data acquisition unit 11 determines whether the reception of the moving image data is continued (Sa4). When the image data acquisition unit 11 determines that the reception of the moving image data is continued (Sa4, Yes), the process of Sa2 is performed on the frame that has not been divided. On the other hand, when the image data acquisition unit 11 determines that the reception of the moving image data is not continued (Sa4, No), the process is terminated.

[0027] Note that the image data acquisition unit 11 may perform the determination process of Sa4 after dividing one frame and performing the processes of Sa2 and Sa3, or may divide a certain number of frames, perform the processes of Sa2 and Sa3 on each of the divided frames, and then perform the determination process of Sa4. As a result, the original image data storage unit 12 stores the original image data arranged in chronological order and associated with time information.

[0028] As shown in FIG. 3, the image processing unit 13 determines whether or not a predetermined number of original image data are recorded in the original image data storage unit 12 (Sb1). Here, the predetermined number is the number of original image data required to perform a predetermined image processing for emphasizing the gas performed by the image processing unit 13. For example, a value of about "5" or "6" is determined in advance.

[0029] When the image processing unit 13 determines that a predetermined number of original image data are not recorded in the original image data storage unit 12 (Sb1, No), it performs the process of Sb1 again. On the other hand, when the image processing unit 13 determines that a predetermined number of original image data are recorded in the original image data storage unit 12 (Sb1, Yes), it reads from the original image data storage unit 12 the original image data recorded at the predetermined number position and the time information associated with the original image data. The image processing unit 13 sets the read original image data as the reference original image data (Sb2-1). When the image processing unit 13 determines the reference original image data, it starts the subroutine of the image processing shown in FIG. 4, which is the predetermined image processing for emphasizing the gas described above (Sb3).

[0030] (Predetermined Image Processing for Emphasizing Gas by Image Processing Unit) The image processing unit 13 records in the internal storage area the time information corresponding to the reference original image data read together with the reference original image data (Sc1). The image processing unit 13 reads from the original image data storage unit 12 the original image data of the time before the time indicated by the time information corresponding to the reference original image data and (predetermined number - 1) consecutive original image data in the time series order. Since the process of Sc2 is performed after the process of Sb1, when the process of Sc2 is performed, the original image data of the time before the reference original image data in the time series order are stored in the original image data storage unit 12 by at least (predetermined number - 1) or more.

[0031] Here, the definitions of the terms "previous time" and "subsequent time" will be explained. For example, assume that there are three original image data items associated with three time information items: "00:00:01", "00:00:02", and "00:00:03". In this case, the original image data associated with the time information indicating a time before the time indicated by the time information associated with the original image data of "00:00:02" is the original image data of "00:00:01". On the other hand, the original image data associated with the time information indicating a time after the time indicated by the time information associated with the original image data of "00:00:02" is the original image data of "00:00:03". The definitions of the terms "previous time" and "subsequent time" are the same in the following as well.

[0032] The image processing unit 13 generates a combination of two original image data items that are before and after in chronological order from a predetermined number of original image data items obtained by adding the reference original image data to the read original image data (Sc3). Here, being before and after in chronological order does not necessarily mean being consecutive in chronological order, but simply being before and after in chronological order. Therefore, the image processing unit 13 所定数 will generate C2 combinations.

[0033] For each combination, the image processing unit 13 performs the following processing to generate differential image data for each combination. The image processing unit 13 reads the luminance values of the pixels at the same positions in the two original image data items included in a certain combination, and performs processing to set the absolute value of the difference between the two read luminance values as the luminance value of the pixel at the corresponding position in the differential image data. The image processing unit 13 generates the differential image data corresponding to the combination by performing this processing for all pixels (Sc4).

[0034] The image processing unit 13 performs a process of binarizing the luminance value of each pixel for each of the generated difference image data for each combination. The image processing unit 13 compares the luminance value of each of all the pixels of the difference image data with a predetermined threshold value. For example, when the luminance value is equal to or greater than the threshold value, the luminance value of the pixel is set to "1", and when the luminance value is less than the threshold value, the luminance value of the pixel is set to "0" for binarization processing (Sc5).

[0035] The image processing unit 13 integrates the luminance values for each pixel for all the binarized difference image data as shown in the following formula (1), and calculates the luminance change frequency which is the sum of the integrated luminance values (Sc6).

[0036]

Equation

[0037] In formula (1), S(x, y) is the luminance change frequency of the pixel (x, y) after integration, and l t (x, y) is the luminance value of the pixel (x, y) in the t-th difference image data. The image processing unit 13 detects the maximum value of the luminance change frequency S(x, y) in the luminance change frequency distribution indicating the luminance change frequencies of all the generated pixels, and normalizes the luminance change frequency S(x, y) of each pixel in the luminance change frequency distribution so that the maximum value becomes a predetermined value, for example, "255". Specifically, the image processing unit 13 divides the luminance change frequency S(x, y) of each pixel in the luminance change frequency distribution by the maximum value of the luminance change frequency S(x, y) in the luminance change frequency distribution and multiplies by a predetermined value to perform normalization (Sc7).

[0038] For example, when the predetermined value is "255", the pixel value of the pixel included in the normalized luminance change frequency distribution becomes a value between 0 and 255. Regarding this pixel value as a luminance value, the normalized luminance change frequency distribution becomes grayscale image data including the same number of pixels as the pixels of the original image data. Therefore, hereinafter, the normalized luminance change frequency distribution is referred to as processed image data.

[0039] The image processing unit 13 reads out the time information associated with the reference original image data stored in the internal storage area. The image processing unit 13 associates the read time information with the processed image data and records it in the processed image data storage unit 14 (Sc8), and ends the processing of the subroutine.

[0040] The image shown in FIG. 6 is an example of an image shown by the processed image data obtained when the original image data in FIG. 5 is used as the reference original image data. In FIG. 6, the pixels having luminance values shown in the area indicated by reference numeral 101 are pixels indicating gas. It can be seen that in the image of the original image data in FIG. 5, the gas that had a small difference from the background luminance value and was unclear is emphasized and made clear in the image of the processed image data shown in FIG. 6.

[0041] Returning to FIG. 3, the image processing unit 13 determines whether or not two or more pieces of processed image data are stored in the processed image data storage unit 14 (Sb4). When the image processing unit 13 determines that two or more pieces of processed image data are not stored in the processed image data storage unit 14 (Sb4, No), the original image data associated with the time information indicating the time one after the time indicated by the time information stored in the internal storage area and the time information associated with the original image data are read out from the original image data storage unit 12. The image processing unit 13 sets the read original image data as the reference original image data (Sb2-2). When the image processing unit 13 determines the reference original image data, it starts again the subroutine of the image processing shown in FIG. 4, which is the processing of Sb3.

[0042] When the image processing unit 13 determines that two or more pieces of processed image data are stored in the processed image data storage unit 14 (Sb4, Yes), it outputs the time information associated with the processed image data recorded most recently in the processed image data storage unit 14 to the vector generation unit 15 (Sb5).

[0043] The vector generation unit 15 captures the time information output by the image processing unit 13. The vector generation unit 15 reads from the original image data storage unit 12 the original image data corresponding to the captured time information and the original image data adjacent to the original image data in time series order and corresponding to a time earlier than the time indicated by the captured time information. The vector generation unit 15 performs optical flow processing on the two read original image data to generate a first vector, which is a vector indicating the moving direction and moving speed of each pixel (Sb6). Note that since the process of Sb6 is performed after the processes of Sb1 and Sb4, when the process of Sb6 is performed, at least one or more pieces of original image data from a time earlier than the original image data corresponding to the time information are stored in the original image data storage unit 12.

[0044] The image shown in FIG. 7 is a diagram in which a first vector obtained by performing optical flow processing on the original image data shown in FIG. 5 and the original image data adjacent to the original image data in time series order is superimposed on the image of the original image data shown in FIG. 5. Note that the first vector is obtained for each pixel, but in FIG. 7, for ease of viewing, the first vector for each pixel with a certain interval is shown.

[0045] The vector generation unit 15 reads from the processed image data storage unit 14 the processed image data corresponding to the captured time information and the processed image data adjacent to the processed image data in time series order and corresponding to a time earlier than the time indicated by the captured time information. In this way, by reading the processed image data, the vector generation unit 15 reads from the processed image data storage unit 14 two pieces of processed image data associated with the same time information as the time information of each of the two original image data read by itself in the process of Sb6, that is, two pieces of processed image data corresponding to each of the two original image data.

[0046] The vector generation unit 15 performs the same processing as the optical flow processing performed in the Sb6 process on the two read processed image data, and generates a second vector which is a vector indicating the moving direction and moving speed of each pixel.

[0047] The vector generation unit 15 associates information indicating the position of the corresponding pixel with each of the generated second vectors and each of the first vectors generated in the Sb6 process, and outputs them to the filter unit 16 (Sb7). Note that since the Sb7 process is performed after the Sb4 process, when the Sb7 process is performed, at least one or more processed image data at a time earlier than the processed image data corresponding to the time information are stored in the processed image data storage unit 14.

[0048] The image shown in FIG. 8 is a diagram in which a second vector obtained by performing optical flow processing on the processed image data shown in FIG. 6 and the processed image data adjacent thereto in time series order is superimposed on the image of the processed image data shown in FIG. 6. Note that the second vector is obtained for each pixel, but in FIG. 8, in order to make the second vector easier to see, the second vector for each pixel with a certain interval is shown. As can be seen from the image shown in FIG. 7 and the image shown in FIG. 8, in FIG. 7, almost no first vector indicating the moving direction and moving speed of the gas is shown, but in FIG. 8, it can be seen that many second vectors indicating the moving direction and moving speed of the gas are shown.

[0049] The filter unit 16 takes in the first vector and the second vector associated with the information indicating the position of the pixel output by the vector generation unit 15. The filter unit 16 selects the first vector and the second vector for which the pixels match, and performs a process of filtering the second vector using the first vector for each pixel. For example, as shown in FIG. 9, the first vector at the pixel (x, y) is the vector V org (x,y) indicated by the solid line arrow of reference numeral 201, and the second vector at the same pixel (x, y) is the vector V processedAssume it is (x, y). Here, vector V org (x, y)201 and vector V processed (x, y)202 form an angle represented by "θ". However, θ is an angle between 0° and 180°.

[0050] In this case, the filter unit 16 filters to calculate a vector V histogram (x, y) which is a second vector and has the same direction as vector V processed (x, y)202, namely vector V histogram (x, y)204. However, when the length calculated by formula (2) is a negative value, the direction of vector V histogram (x, y)204 is opposite to the direction of vector V processed (x, y)202.

[0051]

Equation

[0052] Explaining the filtering performed by the filter unit 16 using vectors, it is as follows. As shown in the second term on the right side of formula (2), the filter unit 16 calculates a multiplication value by multiplying the norm of the first vector, vector V org (x, y)201, by cos(θ / 2). As shown in FIG. 9, the filter unit 16 determines a vector 203 with the calculated multiplication value as the length and in the direction opposite to that of the second vector, vector V processed (x, y)202. The filter unit 16 adds the determined vector 203 and vector V processed (x, y)202 to calculate vector V histogram (x, y)204.

[0053] Since the first vector shown in FIG. 7 shows that the gas is obscurely imaged in the original image data, it is mainly a calculation target point for calculating the optical flow, and is a vector indicating the moving direction and moving speed of the calculation target point corresponding to a fixed object such as a structure. Note that the fixed object is not limited to a structure installed in a plant or the like, but also includes objects such as soil, sand, and grass. The vector of the calculation target point corresponding to this fixed object is a vector generated by the imaging device 2 performing imaging while moving. Therefore, most of the first vectors are vectors that depend on the moving direction and moving speed of the imaging device 2.

[0054] On the other hand, the second vector shown in FIG. 8 is a vector including a vector indicating the moving direction and moving speed of the calculation target point corresponding to a fixed object and a vector indicating the moving direction and moving speed of the calculation target point corresponding to the diffusing gas. The direction and speed of gas diffusion are also affected by factors such as the pressure applied to the gas, the direction in which the gas is discharged, and the strength and direction of the wind around the gas. Therefore, the vector indicating the moving direction and moving speed of the gas in the second vector is a vector obtained by synthesizing a vector that depends on the moving direction and moving speed of the imaging device 2 and a vector generated by factors other than the moving direction and moving speed of the imaging device 2.

[0055] The purpose of the filtering performed by the filter unit 16 is to remove the vector of the calculation target point corresponding to the fixed object and further emphasize the vector of the calculation target point corresponding to the gas. To achieve the purpose of removing the vector of the calculation target point corresponding to the former fixed object, it is only necessary to subtract the first vector from the second vector of the same pixel. However, in the first vector, when there is a calculation target point corresponding to a fixed object in the pixel where the gas exists, simply subtracting the first vector from the second vector has the drawback of reducing the length of the vector of the calculation target point corresponding to the gas. Therefore, in Equation (2), the norm of the vector V org (x, y) is multiplied by cos(θ / 2).

[0056] When θ = 0°, since the first vector and the second vector point in the same direction, it is presumed that both the first vector and the second vector are vectors of the calculation target point corresponding to the fixed object. Since cos(θ / 2) becomes "1" when θ = 0°, the first vector will be directly subtracted from the second vector. Therefore, the vector of the calculation target point corresponding to the fixed object can be removed from the second vector.

[0057] As the angle θ formed by the first vector and the second vector increases, it is presumed that the elements of the vector of the calculation target point corresponding to the gas in the second vector are increasing. Cos(θ / 2) decreases as θ increases and becomes "0" when θ = 180°. Therefore, as the elements of the vector of the calculation target point corresponding to the gas in the second vector increase, the influence of the first vector can be reduced, and the vector of the calculation target point corresponding to the gas included in the second vector can be emphasized. Therefore, the filter unit 16 calculates the vector V for each pixel histogram (x,y)204 can be said to be a vector obtained by emphasizing and extracting the vector of the calculation target point corresponding to the gas while removing the vector of the calculation target point corresponding to the fixed object from the second vector.

[0058] In the following description, the vector of the calculation target point corresponding to the fixed object will simply be referred to as the vector corresponding to the fixed object, and the vector of the calculation target point corresponding to the gas will simply be referred to as the vector corresponding to the gas.

[0059] The image shown in FIG. 10 is the vector V for each pixel calculated by the filter unit 16 performing filtering using Equation (2) on the first vector shown in FIG. 7 and the second vector shown in FIG. 8. histogram(x, y) 204 is an image obtained by superimposing it on the image of the processed image data shown in FIG. 6. As can be seen from the image shown in FIG. 10, it can be understood that by filtering with the filter unit 16, vectors in the vicinity of the region where the structure exists are removed, and mainly, vectors in the vicinity of the region where the gas exists are extracted.

[0060] The filter unit 16 associates information indicating the position of a pixel with the vector for each pixel calculated by filtering. In the processing of optical flow, for pixels at the outer edge of the image, there is a possibility of calculating vectors with large detection errors in the moving speed and low reliability. Also, in the image shown in FIG. 10, like the region indicated by reference numeral 110, the outer edge of the image may contain information indicating the index of the image. Therefore, the filter unit 16 excludes the vectors at the outer edge based on the information indicating the position of the pixel associated with the vector. The filter unit 16 outputs the vectors excluding the vectors at the outer edge to the frequency distribution generation unit 17 (Sb8).

[0061] The frequency distribution generation unit 17 takes in the vectors output by the filter unit 16. The frequency distribution generation unit 17 classifies the taken-in vectors for each preset interval of the vector length and counts the number of vectors for each interval. The frequency distribution generation unit 17 calculates, as the frequency for each interval, the ratio with the number of vectors in each interval as the numerator and the total number of the taken-in vectors as the denominator, and generates a frequency distribution based on the frequency for each interval. The frequency distribution generation unit 17 generates a histogram from the data of the generated frequency distribution and displays the generated histogram on the display unit 19.

[0062] FIG. 11 is an example of the histogram displayed on the display unit 19 by the frequency distribution generation unit 17. In FIG. 11, the vertical axis indicates the frequency, and the horizontal axis indicates the interval of the preset vector length in the frequency distribution generation unit 17. FIG. 11 shows an example of an interval with an interval length of "0.2" as the preset interval of the vector length in the frequency distribution generation unit 17.

[0063] The degree distribution generation unit 17 outputs the generated degree distribution data to the determination unit 18 (Sb9). The determination unit 18 takes in the degree distribution data output by the degree distribution generation unit 17. In the determination unit 18, a predetermined interval that serves as an index for determining the presence or absence of gas leakage and a threshold value indicated by reference numeral 300 in FIGS. 11(a) and 11(b) are predetermined. Here, the interval that serves as an index for determining the presence or absence of gas leakage is an interval longer than the length of a vector predetermined from empirical values or the like. Here, it is assumed that the interval is one where the length of the vector is 999.8 or more.

[0064] The determination unit 18 detects the frequency of each interval in the taken-in degree distribution data where the length of the vector is 999.8 or more, and determines whether any of the detected frequencies is equal to or greater than the threshold value. When the determination unit 18 determines that any of the detected frequencies is equal to or greater than the threshold value, for example, it superimposes and displays the character string "Gas leakage present" as the determination result on the histogram displayed on the display unit 19. When the determination unit 18 determines that none of the detected frequencies is equal to or greater than the threshold value, for example, it superimposes and displays the character string "No gas leakage" as the determination result on the histogram displayed on the display unit 19.

[0065] FIG. 11(a) is an example of a histogram in the case of an image taken at a location where no gas leakage has occurred, and FIG. 11(b) is an example of a histogram in the case of an image taken at a location where gas leakage has occurred. In FIG. 11(a), it is assumed that the frequency of each interval where the length of the vector is 999.8 or more does not exceed the threshold value indicated by reference numeral 300. In contrast, in FIG. 11(b), the frequency of at least the interval of 999.8 to 1000.0 exceeds the threshold value indicated by reference numeral 300. Therefore, in the case of FIG. 11(a), the determination unit 18 determines that none of the detected thresholds is equal to or greater than the threshold value, and displays "No gas leakage" on the display unit 19. In the case of FIG. 11(b), the determination unit 18 determines that any of the detected thresholds is equal to or greater than the threshold value, and displays "Gas leakage present" on the display unit 19.

[0066] After making a determination, the determination unit 18 outputs an instruction signal indicating to continue the process to the image processing unit 13 (Sb10). When the image processing unit 13 receives the instruction signal indicating to continue the process from the determination unit 18, it determines whether new original image data associated with time information at a time later than the time information of the reference original image data stored in the internal storage area is stored in the original image data storage unit 12 (Sb11).

[0067] When the image processing unit 13 determines that new original image data is stored in the original image data storage unit 12 (Sb11, Yes), it performs the process of Sb2-2 again. When the image processing unit 13 determines that new original image data is not stored in the original image data storage unit 12 (Sb11, No), it ends the process.

[0068] (Operation and Effect) In the gas leakage monitoring device 10 of the above-described embodiment, the image data acquisition unit 11 divides the frames of the moving image captured while the imaging device 2 is moving to generate original image data. The image processing unit 13 performs predetermined image processing for emphasizing gas on the original image data to generate processed image data. The vector generation unit 15 selects two adjacent original image data in chronological order and two processed image data corresponding to the two original image data, and performs predetermined image processing for calculating the moving direction and moving speed of the object included in the image for each combination of the original image data and each combination of the processed image data, and generates a first vector for each pixel and a second vector for each pixel. The filter unit 16 performs filtering for emphasizing and extracting the vector corresponding to gas with respect to the second vector corresponding to each pixel using the first vector corresponding to each pixel at each pixel. The frequency distribution generation unit 17 classifies the extracted vectors for each section to generate a frequency distribution. The determination unit 18 determines the presence or absence of gas leakage for each image using the frequency of each section of a predetermined section, for example, a section having a length equal to or greater than a predetermined vector length, and a predetermined threshold value.

[0069] In the original image data, since the gas is vaguely depicted, the first vector generated from the original image data mainly corresponds to vectors of fixed objects such as structures. On the other hand, in the processed image data, since the gas is emphasized, the second vector generated from the processed image data includes vectors corresponding to fixed objects such as structures and vectors corresponding to the gas. By utilizing the difference between the first vector generated from the original image data and the second vector generated from the processed image data, and filtering the second vector using the first vector, the vector corresponding to the gas can be emphasized and extracted from the second vector. In the vector extracted in this way, the vector corresponding to the gas becomes a vector having a certain length because it is emphasized. Therefore, it is possible to generate a frequency distribution from the extracted vectors and determine the presence or absence of gas leakage based on the magnitude of the frequency in the interval where the vector corresponding to the gas is included in the generated frequency distribution.

[0070] In the technique disclosed in Patent Document 1, since it is premised that the gas is depicted in the captured image, in the case where the gas is not depicted in the captured image, the gas leakage monitoring device 10 can more accurately indicate the presence or absence of gas leakage. In the gas leakage monitoring device 10, although three or more image data may be used in a predetermined image process for emphasizing the gas by the image processing unit 13, in the process after the vector generation unit 15, as image data, only two original image data and two processed image data are used, and there is no need to increase the number of image data for improving accuracy as in the technique disclosed in Patent Document 1. Therefore, the gas leakage monitoring device 10 of the above-described embodiment can more accurately indicate the presence or absence of gas leakage in a shorter time compared to the method disclosed in Patent Document 1.

[0071] (Other configuration examples of the embodiment) As described above, the embodiments of the present disclosure have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and designs and the like within the scope not departing from the gist of the present disclosure are also included.

[0072] (When gas is clearly shown in the original image data) When the difference between the temperature of a fixed object such as a structure and the temperature of the gas is small, the gas will not be clearly shown in the original image data generated by the imaging device 2 taking a picture. By using the gas leakage monitoring device 10 of the above embodiment, it is possible to filter the vector corresponding to the fixed object and extract the vector corresponding to the gas. On the contrary, when the difference between the temperature of the fixed object and the temperature of the gas is large, the gas will be clearly shown in the original image data generated by the imaging device 2 taking a picture.

[0073] In this case, even if the predetermined image processing for emphasizing the gas by the image processing unit 13 is not performed, the vector generation unit 15 will include the vector corresponding to the gas in the first vector generated from the original image data. Therefore, the gas leakage monitoring device 10 may not include the image processing unit 13, the processed image data storage unit 14, and the filter unit 16. In this case, the processes from Sb1 to Sb5 in FIG. 3 will not be performed. When the vector generation unit 15 detects that at least two or more pieces of original image data have been recorded in the original image data storage unit 12 by the image data acquisition unit 11, the vector generation unit 15 performs the process of Sb6 in FIG. 3 to generate the first vector and skips the process of Sb7 for generating the second vector. The process of Sb8 is also skipped. The frequency distribution generation unit 17 takes in the first vector generated by the vector generation unit 15 instead of the vector calculated and output by the filter unit 16, performs the process of Sb9 on the taken-in first vector, and then the processes after Sb10 are performed. The process of Sb11 is performed by the vector generation unit 15. When the determination is "Yes", the process of Sb6 is performed on the original image data at a time one later than the original image data targeted by the previous process of Sb6.

[0074] However, the first vector generated by the vector generation unit 15 includes a vector corresponding to a fixed object and a vector corresponding to the gas. Therefore, even if the first vector is classified for each interval of a predetermined vector length and a frequency distribution is generated with the number of vectors for each interval as the frequency, the frequency of the frequency distribution will include the frequency corresponding to the vector corresponding to the fixed object. Therefore, when the difference between the length of the vector corresponding to the fixed object and the length of the vector corresponding to the gas is small, it becomes difficult to determine the presence or absence of gas leakage.

[0075] By the way, the moving speed of the fixed object in the image approximately matches the moving speed of the imaging device 2. Therefore, if the moving speed of the imaging device 2 is constant, the range of the interval of the frequency distribution corresponding to the moving speed can be specified. Therefore, the determination unit 18 can determine the presence or absence of gas leakage for each image, for example, using the frequency of each interval outside the specified interval range and a predetermined threshold value, where the interval is equal to or greater than the predetermined vector length.

[0076] Also, even if the moving speed of the imaging device 2 is not constant, if there is a large difference between the moving speed of the imaging device 2 and the speed at which the gas diffuses, the difference between the length of the vector corresponding to the fixed object and the length of the vector corresponding to the gas will increase. Therefore, in the frequency distribution generated by the frequency distribution generation unit 17, a large difference will occur between the interval in which the vector corresponding to the fixed object is shown and the interval in which the vector corresponding to the gas is shown. In this case, if the range of the interval in which the vector corresponding to the gas is shown can be specified, the determination unit 18 can determine the presence or absence of gas leakage for each image using the frequency of each interval in the specified interval range and a predetermined threshold value.

[0077] Therefore, when gas is clearly shown in the original image data, since it is not necessary to perform the predetermined image processing for emphasizing the gas by the image processing unit 13, the time required for this processing is reduced. In the processing after the vector generation unit 15, as the image data, only two pieces of original image data are used, and it is not necessary to increase the number of pieces of image data for improving the accuracy as in the technique disclosed in Patent Document 1. Therefore, even when gas is clearly shown in the original image data, it is possible to indicate the presence or absence of gas leakage with higher accuracy and in a shorter time as compared with the method disclosed in Patent Document 1.

[0078] (When the imaging device is fixed (Part 1)) In the above-described embodiment, the imaging device 2 is mounted on, for example, a UAV and moved. On the other hand, when the imaging device 2 such as for fixed-point observation is fixed and imaging is performed, a fixed object such as a structure does not move in the image as long as the imaging device 2 does not shake due to wind. Therefore, even if the optical flow process is performed, no vector will be generated. Therefore, in this case, the gas leakage monitoring device 10 may not include the filter unit 16, and the vector generation unit 15 may not generate the first vector. Instead of the vector output from the filter unit 16, if the second vector generated by the vector generation unit 15 is classified for each section of a predetermined vector length and a frequency distribution having the number of vectors for each section as the frequency is generated by the frequency distribution generation unit 17, the presence or absence of gas leakage for each image can be determined by the determination unit 18 from this frequency distribution. In this case, in the processing after the vector generation unit 15, as the image data, only two pieces of processed image data are used, and it is not necessary to increase the number of pieces of image data for improving the accuracy as in the technique disclosed in Patent Document 1. Therefore, even when the imaging device 2 is fixed, it is possible to indicate the presence or absence of gas leakage with higher accuracy and in a shorter time as compared with the method disclosed in Patent Document 1.

[0079] (When the imaging device is fixed (Part 2)) The imaging device 2 is fixed, and furthermore, the difference between the temperature of a fixed object such as a structure and the temperature of the gas is large, and it is assumed that the gas is clearly shown in the original image data generated by the imaging device 2 taking a picture. In this case, the gas leakage monitoring device 10 may not include the image processing unit 13, the processed image data storage unit 14, and the filter unit 16. The vector generation unit 15 may generate the first vector from the original image data stored in the original image data storage unit 12 and may not generate the second vector. The first vector mainly includes a vector corresponding to the gas. Therefore, instead of the second vector, if the first vector is classified for each section of a predetermined vector length and the frequency distribution generation unit 17 is made to generate a frequency distribution with the number of vectors for each section as the frequency, the presence or absence of gas leakage for each image can be determined by the determination unit 18 from this frequency distribution.

[0080] When the gas is clearly shown in the original image data, since it is not necessary to perform a predetermined image process for emphasizing the gas by the image processing unit 13, the time required for this process is reduced. In the process after the vector generation unit 15, as image data, only two pieces of original image data are used, and it is not necessary to increase the number of pieces of image data for improving accuracy as in the technique disclosed in Patent Document 1. Therefore, even when the imaging device 2 is fixed and the gas is clearly shown in the original image data, it is possible to indicate the presence or absence of gas leakage with higher accuracy and in a shorter time compared to the method disclosed in Patent Document 1.

[0081] (Configuration example without a determination unit) In the above-described embodiment, after the frequency distribution generation unit 17 generates the frequency distribution, the determination unit 18 determines the presence or absence of gas leakage using the generated frequency distribution. In contrast, without providing the determination unit 18, the frequency distribution generation unit 17 may allow a person to refer to the frequency of each of the predetermined intervals, for example, intervals longer than a predetermined vector length, in the histogram displayed on the display unit 19 to determine the presence or absence of gas leakage. In this case, a person can compare the frequency with a threshold value to determine the presence or absence of gas leakage, or can also determine the presence or absence of gas leakage from the difference in the relative magnitudes of the frequencies of the respective intervals. Therefore, when determining the presence or absence of gas leakage from the difference in the relative magnitudes of the frequencies of the latter respective intervals, the frequency distribution generation unit 17 may display the histogram with the number of vectors in each interval as the frequency as it is, instead of calculating the ratio with the number of vectors in each interval as the numerator and the total number of captured vectors as the denominator as the frequency of each interval.

[0082] (Other Configuration Examples) In the above-described embodiment, the image data acquisition unit 11 associates the time information associated with the frame of the source of the original image data with the original image data and records it in the original image data storage unit 12. In contrast, the image data acquisition unit 11 may not associate the time information with the original image data, but instead may sequentially assign consecutive numbers starting from 1, for example, in the order of division, and record it in the original image data storage unit 12. In this case, for the subsequent processing by the image processing unit 13 and the vector generation unit 15, the assigned numbers will be used instead of the time information. Specifically, in the process of Sc2 in FIG. 4, the image processing unit 13 will identify the original image data for generating the processed image data based on the assigned numbers and read it from the original image data storage unit 12. The vector generation unit 15 will read the original image data and the processed image data, to which the numbers received from the image processing unit 13 are assigned, from the original image data storage unit 12 and the processed image data storage unit 14, respectively, in the processes of Sb6 and Sb7 in FIG. 3.

[0083] In the above-described embodiment, the imaging device 2 is configured to capture a moving image. In contrast, the imaging device 2 may capture still images at different times, associate time information indicating the captured time with the data of the still images, and transmit the still images to the image data acquisition unit 11. However, the interval between different times at which the imaging is performed needs to be such that vectors corresponding to objects moving within the image can be generated by optical flow processing, and it is preferably different consecutive times as in the case of capturing a moving image. Note that, instead of the imaging device 2 associating time information with the data of the still images, as described above, consecutive numbers starting from 1 may be associated with the image data in the order in which the image data acquisition unit 11 receives the image data.

[0084] In the above-described embodiment, the predetermined image processing for emphasizing the gas performed by the image processing unit 13 is not limited to the image processing shown in FIG. 4. For example, among the luminance values from 0 to 255 of the original image data in grayscale, the luminance value range of the gas, for example, 100 to 150, may be used as the original values, and the luminance values of the remaining pixels may be set to "0" or "255" in the image processing. Further, image processing for adjusting the contrast so that the gas is emphasized may also be used.

[0085] In the above-described embodiment, the vector generation unit 15 selects two adjacent original image data in chronological order from the original image data storage unit 12, and selects two adjacent original image data in chronological order from the processed image data storage unit 14. In contrast, the vector generation unit 15 does not necessarily have to select adjacent ones. For example, when a sufficient number of original image data are stored in the original image data storage unit 12 and a sufficient number of processed image data are stored in the processed image data storage unit 14, the vector generation unit 15 selects one of the original image data corresponding to the time information output by the image processing unit 13, and then selects, as the other original image data, the original image data that is a predetermined time before the selected one of the original image data in chronological order. In this case, the vector generation unit 15 selects two processed image data associated with the time information of each of the two selected original image data from the processed image data storage unit 14. Note that the predetermined time may be determined in advance or may be changed for each selection, but the predetermined time is preferably determined such that the shooting ranges of the two original image data do not vary significantly.

[0086] In the above-described embodiment, the vector generation unit 15 captures the time information output by the image processing unit 13 in the process of Sb5 in FIG. 3, and selects and reads out the original image data and the processed image data corresponding to the captured time information in the processes of Sb6 and Sb7. In contrast, while the range captured by the imaging device 2 does not change significantly, that is, while there is little change in the position of a fixed object such as a structure, the selection of the original image data according to the time information may not be performed. For example, once the vector generation unit 15 performs the process of Sb6 to calculate the first vector, it skips the process of Sb6 for the number of process times corresponding to the period during which the range captured by the imaging device 2 does not change significantly, and in the process of Sb7, when outputting the first vector to the filter unit 16, it may output the most recently generated first vector.

[0087] In the above-described embodiment, as the predetermined image processing for calculating the moving direction and moving speed of an object included in an image performed by the vector generation unit 15, the optical flow processing of the Gunnar Farneback method among the optical flow estimation methods is shown as an example. However, any optical flow method may be applied, or processing other than optical flow may be applied.

[0088] In the above-described embodiment, the filter unit 16 performs filtering according to formula (2). In contrast, instead of cos(θ / 2) in formula (2), when the angle θ formed by the second vector and the first vector having the same pixel position as the second vector is 0°, it is a ratio that becomes 100%, and as the angle θ approaches 180°, a function other than cos(θ / 2) that calculates a ratio in which the percentage value approaches 0% may be applied.

[0089] In the above-described embodiment, the gas is CO2 gas. However, for example, a gas having an infrared absorption wavelength band such as methane or ammonia may be used, or a gas having an absorption wavelength band outside the infrared region may be used. When a gas other than CO2 gas is the imaging target, the imaging device 2 is provided with a filter that transmits the absorption wavelength band of the gas.

[0090] In the above-described embodiment, in the process of Sc4 in FIG. 4, when generating the difference image data, if the imaging ranges of the two original image data included in the combination are different, alignment may be performed so that the imaging ranges match, and then the difference image data may be generated. The method of performing alignment may be a method of detecting a common calculation target point in the two original image data and changing the coordinates so that the positions of the common calculation target points match, or other methods may be applied.

[0091] In the above-described embodiment, the image data acquisition unit 11 performs noise removal processing and conversion to grayscale in the process of Sa2 in FIG. 2. However, when there is little noise in the image captured by the imaging device 2, the noise removal processing may not be performed. Also, when the image captured by the imaging device 2 is grayscale instead of color, the grayscale conversion may not be performed. Further, in the process of Sa2 in FIG. 2, the image data acquisition unit 11 converts the image data into grayscale image data so as to preserve the luminance of the image data and generate the original image data. However, the image data may be converted into grayscale image data so as to preserve the lightness of the image data and generate the original image data. In this case, in the above-described embodiment, "luminance" will be read as "lightness".

[0092] In the above-described embodiment, the image processing unit 13 performs binarization processing on the differential image data in the process of Sc5 in FIG. 4. However, the binarization processing may not be performed. In this case, the image processing unit 13 performs the process of Sc6 on the differential image data generated in the process of Sc4, that is, the process of integrating the luminance values for each pixel of the differential image data generated in the process of Sc4 to generate the processed image data. Thereafter, the processes of Sc7 and Sc8 will be performed.

[0093] In the above-described embodiment, the filter unit 16 excludes the vectors at the outer edge of the image. However, when the reliability of the vectors at the outer edge of the image is high, the vectors at the outer edge of the image may not be excluded.

[0094] In the above-described embodiment, as shown in the processes of Sb6 and Sb7 in FIG. 3, the vector generation unit 15 generates the second vector after generating the first vector. Conversely, the second vector may be generated first and then the first vector may be generated, or the process of generating the first vector and the process of generating the second vector may be performed in parallel.

[0095] In the above-described embodiment, the determination unit 18 determines the presence or absence of gas leakage for each image by using the frequency of each interval equal to or longer than a predetermined vector length and a predetermined threshold value. On the other hand, when there is a situation such that the length of the vector corresponding to the gas is empirically known, instead of setting an interval equal to or longer than the predetermined vector length, an interval including the length of the vector corresponding to the gas may be predetermined, and the presence or absence of gas leakage for each image may be determined by using the frequency of each of the predetermined intervals and a predetermined threshold value.

[0096] In the above-described embodiment, the image processing unit 13 performs the process of Sb11 in FIG. 3 upon receiving an instruction signal indicating to continue the process from the determination unit 18. On the other hand, the image processing unit 13 may perform the process of Sb11 after performing the process of Sb5 without the determination unit 18 outputting an instruction signal indicating to continue the process.

[0097] In the above-described embodiment, in the process of Sc5 in FIG. 4 and the process of Sb10 in FIG. 3, the image processing unit 13 and the determination unit 18 perform a determination process using a threshold value, and determine whether or not the target value is equal to or greater than the threshold value. On the other hand, in each of the processes of Sc5 and Sb10, depending on how the threshold value is determined, it may be a process of determining whether or not the target value exceeds the threshold value.

[0098] (Computer Configuration) FIG. 12 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. The computer 90 includes a processor 91, a main memory 92, a storage 93, and an interface 94. The above-described gas leakage monitoring device 10 is implemented in the computer 90. Then, the operations of the above-described respective processing units, that is, the image data acquisition unit 11, the image processing unit 13, the vector generation unit 15, the filter unit 16, the frequency distribution generation unit 17, and the determination unit 18 are stored in the storage 93 in the form of a program. The processor 91 reads the program from the storage 93, expands it in the main memory 92, and executes the above processing according to the program. Further, the processor 91 secures a storage area corresponding to the above-described original image data storage unit 12 and the processed image data storage unit 14 in the main memory 92 or the storage 93 according to the program. The display unit 19 is connected via the interface 94. Therefore, the display unit 19 may or may not be a component of the computer 90, that is, a component of the gas leakage monitoring device 10 as described above.

[0099] The program may be for realizing a part of the functions to be exhibited by the computer 90. For example, the program may exhibit functions in combination with other programs already stored in the storage 93 or in combination with other programs implemented in other devices. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array), and the like. In this case, part or all of the functions realized by the processor may be realized by the integrated circuit.

[0100] Examples of the storage 93 include HDD (Hard Disk Drive), SSD (Solid State Drive), magnetic disk, magneto-optical disk, CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), semiconductor memory, and the like. The storage 93 may be an internal medium directly connected to the bus of the computer 90, or may be an external medium connected to the computer 90 via the interface 94 or a communication line. Further, when this program is distributed to the computer 90 via a communication line, the computer 90 that has received the distribution may expand the program in the main memory 92 and execute the above processing. In at least one embodiment, the storage 93 is a non-transitory tangible storage medium.

[0101] <Appendix> The gas leakage monitoring device 10 described in the embodiment according to the present disclosure is understood as follows, for example.

[0102] (1) The gas leakage monitoring device 10 according to the first aspect includes an image data acquisition unit 11 that acquires, as original image data, image data generated by being photographed at different times, and selects two pieces of original image data in chronological order from the original image data, and performs a predetermined image process on the two pieces of original image data to be selected to calculate the moving direction and moving speed of an object included in the image, thereby generating a vector indicating the moving direction and moving speed for each pixel included in the original image data. A vector generation unit 15, and a frequency distribution generation unit 17 that classifies the vectors generated by the vector generation unit for each section of a predetermined vector length to generate a frequency distribution for gas leakage monitoring. According to this aspect and each of the following aspects, it is possible to accurately indicate the presence or absence of gas leakage in a shorter time.

[0103] (2) The gas leakage monitoring device 10 according to the second aspect is the gas leakage monitoring device 10 of (1), and includes an image processing unit 13 that performs predetermined image processing for emphasizing gas on the original image data to generate processed image data. The vector generation unit generates a vector indicating the moving direction and moving speed for each pixel included in the processed image data using the processed image data instead of the original image data. According to this aspect, even if the gas is not clearly shown in the original image data, a vector corresponding to the gas can be generated from the processed image data with the gas emphasized.

[0104] (3) The gas leakage monitoring device 10 according to the third aspect is the gas leakage monitoring device 10 of (1), and includes an image processing unit 13 that performs predetermined image processing for emphasizing gas on the original image data to generate processed image data, and a filter unit 16 that emphasizes and extracts a vector corresponding to the gas. The vector generation unit sets the vector generated for each pixel included in the original image data as the first vector. Further, two pieces of processed image data are selected from the processed image data in chronological order, and predetermined image processing for calculating the moving direction and moving speed of the object included in the image is performed on the two selected pieces of processed image data, thereby generating a vector indicating the moving direction and moving speed for each pixel included in the processed image data as the second vector. The filter unit filters each of the second vectors for each pixel using the first vector that matches the pixel. The frequency distribution generation unit generates the frequency distribution for gas leakage monitoring using the vector extracted by the filter unit instead of the vector generated by the vector generation unit. According to this aspect, even if the imaging device for imaging is moving and the gas is not clearly shown in the original image data, processed image data with the gas emphasized is generated, and by filtering based on the difference between the first vector generated from the original image data and the second vector generated from the processed image data, the vector corresponding to the gas is emphasized and extracted, and the presence or absence of gas leakage can be indicated using the extracted vector.

[0105] (4) The gas leakage monitoring device 10 according to the fourth aspect is the gas leakage monitoring device 10 of (3), wherein when the vector generation unit selects the two original image data, it selects two adjacent original image data in the chronological order, and when selecting the two processed image data, it selects the processed image data corresponding to each of the two selected original image data. According to this aspect, the two original image data that are the generation sources of the first vector are adjacent in the chronological order, and the two processed image data that are the generation sources of the second vector are the processed image data corresponding to each of the two original image data and are adjacent processed image data in the chronological order. Thus, by using the image data adjacent in the chronological order, the accuracy of the first vector and the second vector is improved. Further, by using the processed image data corresponding to each of the two original image data, since the shooting ranges in the original image data and the processed image data are the same, the vector corresponding to the gas can be extracted with higher accuracy.

[0106] (5) The gas leakage monitoring device 10 according to the fifth aspect is the gas leakage monitoring device 10 of (3) or (4), wherein the filter unit performs filtering by adding a vector having a length obtained by multiplying the length of the first vector by a ratio that becomes 100% when the angle formed by the second vector and the first vector having the same pixel position as the second vector is 0°, and the percentage value approaches 0% as the angle approaches 180°, and the direction of which is opposite to that of the second vector, and the second vector. According to this aspect, it is possible to emphasize the vector corresponding to the gas while excluding the vector corresponding to a fixed object such as a structure from the second vector by using the first vector.

[0107] (6) The gas leakage monitoring device 10 according to the sixth aspect is any one of the gas leakage monitoring devices 10 from (2) to (5), and the image processing unit, as predetermined image processing for emphasizing the gas, selects a combination of two pieces of the original image data that are before and after in chronological order from the original image data to be processed and a plurality of pieces of the original image data acquired by the image data acquisition unit at a time earlier than the original image data, generates difference image data for each selected combination, and generates the processed image data corresponding to the original image data to be processed by integrating the pixel values of the generated difference image data for each pixel. According to this aspect, even if the gas is not clearly shown in the original image data, it is possible to generate processed image data with the gas emphasized.

[0108] (7) The gas leakage monitoring device 10 according to the seventh aspect is any one of the gas leakage monitoring devices 10 from (1) to (6), and includes a determination unit 18 that determines the presence or absence of gas leakage. The frequency distribution generation unit generates the frequency distribution in which the ratio with the number of the vectors for each interval as the numerator and the total number of the vectors as the denominator is the frequency for each interval. The determination unit determines the presence or absence of gas leakage based on the frequency of the predetermined interval and a predetermined threshold value. According to this aspect, it is not necessary for a person to refer to the histogram to determine the presence or absence of gas leakage, and the determination unit 18 can determine the presence or absence of gas leakage from the data of the frequency distribution generated by the frequency distribution generation unit 17.

[0109] (8) The gas leakage monitoring device 10 according to the eighth aspect is any one of the gas leakage monitoring devices 10 from (1) to (7), and the filter unit excludes the vectors corresponding to the pixels at the outer edge of the image data from the vectors after emphasizing and extracting the vectors corresponding to the gas. According to this aspect, since the vectors used when generating the frequency distribution can be narrowed down to highly reliable vectors, the accuracy of the frequency distribution can be improved, and accordingly, the presence or absence of gas leakage can be accurately distinguished.

Explanation of reference numerals

[0110] 1…Gas leakage monitoring system 2…Imaging device 3…Communication network 10…Gas leakage monitoring device 11…Image data acquisition unit 12…Original image data storage unit 13…Image processing unit 14…Processed image data storage unit 15…Vector generation unit 16…Filter unit 17…Frequency distribution generation unit 18…Judgment unit 19…Display unit

Claims

1. An image data acquisition unit that acquires image data generated by taking pictures at different times as the original image data, A vector generation unit selects two original image data in chronological order from the aforementioned original image data, and performs predetermined image processing on the two selected original image data to calculate the direction and speed of movement of objects contained in the images, thereby generating a vector indicating the direction and speed of movement for each pixel contained in the original image data. A frequency distribution generation unit that classifies the vectors generated by the vector generation unit into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring, An image processing unit that performs a predetermined image processing to enhance gas on the original image data to generate processed image data, It comprises a filter section that emphasizes and extracts the vector corresponding to the gas, The aforementioned vector generation unit, The vector generated for each pixel in the original image data is designated as the first vector, and further, two processed image data are selected from the processed image data in chronological order, and a predetermined image processing is performed on the two selected processed image data to calculate the direction and speed of movement of an object contained in the image, thereby generating a vector indicating the direction and speed of movement for each pixel in the processed image data, which is designated as the second vector. The filter unit is Each of the second vectors for each pixel is filtered using the first vector that matches each pixel. The frequency distribution generation unit is, Instead of using the vector generated by the vector generation unit, the frequency distribution for gas leak monitoring is generated using the vector extracted by the filter unit. Gas leak monitoring device.

2. The system includes an image processing unit that performs a predetermined image processing to enhance gas in the original image data to generate processed image data, The aforementioned vector generation unit, Instead of the original image data, the processed image data is used to generate vectors indicating the direction and speed of movement for each pixel included in the processed image data. The gas leak monitoring device according to claim 1.

3. The aforementioned vector generation unit, When selecting two of the aforementioned original image data, two adjacent original image data are selected in chronological order, and when selecting two of the aforementioned processed image data, processed image data corresponding to each of the two original image data to be selected is selected. The gas leak monitoring device according to claim 1.

4. The filter unit is A filtering process is performed by adding the second vector and the second vector, which has a length obtained by multiplying the length of the first vector by a ratio that is 100% when the angle between the second vector and the first vector (which has the same pixel position as the second vector) is 0°, and whose percentage value approaches 0% as the angle approaches 180°, and whose direction is opposite to that of the second vector. The gas leak monitoring device according to claim 1.

5. The aforementioned image processing unit, As a predetermined image processing to enhance the gas, the image processing involves selecting a combination of two original image data that are in a sequential order from the original image data to be processed and a plurality of original image data acquired by the image data acquisition unit at a time earlier than the original image data, generating differential image data for each selected combination, and generating the processed image data corresponding to the original image data to be processed by accumulating the pixel values ​​of the generated differential image data pixel by pixel. A gas leak monitoring device according to any one of claims 1 to 4.

6. It is equipped with a determination unit that determines whether or not there is a gas leak, The frequency distribution generation unit is, A frequency distribution is generated in which the number of vectors in each interval is the numerator and the total number of vectors is the denominator, and the frequency for each interval is the ratio. The determination unit, The presence or absence of gas leakage is determined based on the predetermined frequency of the aforementioned interval and a predetermined threshold. A gas leak monitoring device according to any one of claims 1 to 4.

7. The filter unit is After extracting the vectors corresponding to the gas, the vectors corresponding to the pixels at the outer edge of the image data are excluded. A gas leak monitoring device according to any one of claims 1 to 4.

8. The steps include: acquiring image data generated by taking pictures at different times as the original image data; The process involves selecting two original image data from the acquired original image data in chronological order, and performing predetermined image processing on the two selected original image data to calculate the direction and speed of movement of objects contained in the images, thereby generating vectors indicating the direction and speed of movement for each pixel contained in the original image data. The steps include: classifying the generated vectors into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring; The steps include: generating processed image data by performing a predetermined image processing to enhance gas on the original image data, This includes the step of highlighting and extracting the vector corresponding to the gas, In the step of generating the aforementioned vector, The vector generated for each pixel in the original image data is designated as the first vector, and further, two processed image data are selected from the processed image data in chronological order, and a predetermined image processing is performed on the two selected processed image data to calculate the direction and speed of movement of an object contained in the image, thereby generating a vector indicating the direction and speed of movement for each pixel in the processed image data, which is designated as the second vector. In the extraction step, Each of the second vectors for each pixel is filtered using the first vector that matches each pixel. In the step of generating the frequency distribution, Instead of using the vector generated in the step of generating the aforementioned vector, the frequency distribution for gas leak monitoring is generated using the vector extracted in the extraction step. Gas leak monitoring method.

9. The steps include: acquiring image data generated by taking pictures at different times as the original image data; The process involves selecting two original image data in chronological order from the acquired original image data, and performing predetermined image processing on the two selected original image data to calculate the direction and speed of movement of objects contained in the images, thereby generating vectors indicating the direction and speed of movement for each pixel contained in the original image data. The steps include: classifying the generated vectors into intervals of predetermined vector lengths to generate a frequency distribution for gas leak monitoring; The steps include: generating processed image data by performing a predetermined image processing to enhance gas on the original image data, Steps include: highlighting and extracting the vector corresponding to the gas, A program that causes a computer to execute, In the step of generating the aforementioned vector, The vector generated for each pixel in the original image data is designated as the first vector, and further, two processed image data are selected from the processed image data in chronological order, and a predetermined image processing is performed on the two selected processed image data to calculate the direction and speed of movement of an object contained in the image, thereby generating a vector indicating the direction and speed of movement for each pixel in the processed image data, which is designated as the second vector. In the extraction step, Each of the second vectors for each pixel is filtered using the first vector that matches each pixel. In the step of generating the frequency distribution, Instead of using the vector generated in the step of generating the aforementioned vector, the frequency distribution for gas leak monitoring is generated using the vector extracted in the extraction step. program.

Images