Reflection component suppression image generating device, reflection component suppression inference model generating device, reflection component suppression image generating method, and program

Abstract

The present invention is provided with: an inspection image input unit that receives, as an input, a gas distribution image in which an area where gas is present in a space is visualized and which includes an image portion where an object is irradiated with light; and a reflection-component-reduced image generating unit that generates, with use of an inference model, a reflection-component-reduced image in which an image component of reflection light is reduced in the image portion of the gas distribution image received by the inspection image input unit, said inference model having been trained by machine learning using, as training data, a combination of a first image which includes an image portion where an object is irradiated with light and a second image which includes an image portion where the object is not irradiated with light and in which elements other than this image portion are equivalent to the first image.

Description

[Technical Field]

[0001] The present disclosure relates to a reflected component suppression image generation device, a reflected component suppression inference model generation device, a reflected component suppression image generation method, and a program, and in particular to the detection and suppression using machine learning of image components including high-intensity reflected light generated by flare stacks in gas facilities, etc. [Background technology]

[0002] In facilities that use gas (hereinafter referred to as "gas facilities"), such as production facilities that produce natural gas and oil, production plants that use gas to produce chemical products, gas transmission facilities, petrochemical plants, thermal power plants, steel-related facilities, etc., the risk of gas leakage due to deterioration of the facilities over time or operational errors is recognized, and gas detection devices are installed to minimize gas leakage.

[0003] In gas detection, in addition to gas detection devices that utilize the change in the electrical properties of a detection probe when gas molecules come into contact with the probe, in recent years, optical gas leak detection methods have been adopted that detect gas leaks in an inspection area by taking infrared video images using the infrared absorption properties of gas (for example, Patent Documents 1 and 2). This gas detection method using infrared video images can visualize gas using images, making it easy to detect the release state of gas flow and other factors, as well as the location of the leak. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2016 / 143754 [Patent Document 2] International Publication No. 2017 / 150565 [Patent Document 3] Japanese Patent Application Laid-Open No. 2013-121099 Summary of the Invention [Problem to be solved by the invention]

[0005] However, gas plants and petrochemical plants are generally equipped with a facility called a flare stack, which burns excess gas generated during operation to detoxify it. The flame generated by gas combustion makes the tip of the flare stack extremely hot, and a large amount of infrared radiation is emitted from that area.

[0006] Figure 17 is a schematic diagram showing the state of reflected light from a flare stack in a gas facility. When a flare stack is observed with a gas visualization imaging device, the emitted infrared light illuminates the equipment around the flare stack, and is observed as a high-brightness reflected component, as shown in Figure 17. Furthermore, because the infrared brightness and shape of the flame change from moment to moment, the illuminance of the high-brightness reflected component also changes from moment to moment.

[0007] As a result, changes in the amount of high-intensity infrared light that are different from the gas being detected are observed, making it difficult to observe changes in the amount of infrared light caused by the gas being detected, resulting in a drastic reduction in the gas detection rate.

[0008] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a reflected component suppression image generation device, a reflected component suppression inference model generation device, a reflected component suppression image generation method, and a program that reduce the effects of changes in the amount of infrared light caused by high-brightness light sources in gas equipment from the output image of a gas visualization imaging device. [Means for solving the problem]

[0009] A reflected component suppressed image generating device according to one aspect of the present disclosure includes an inspection image input unit that receives as input a gas distribution image in which a region where gas exists in a space is visualized and which includes an image portion where light is irradiated onto an object; and a reflected component suppressed image generating unit that generates a reflected component suppressed image in which image components of reflected light are suppressed in the image portion of the gas distribution image received by the inspection image input unit using an inference model that is machine-learned using as training data a combination of a first image including the image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and that is equivalent to the first image with respect to elements other than the image portion. The image component of the reflected light is a time-varying component in the image portion where the light is irradiated onto the object. It is characterized by: [Effects of the Invention]

[0010] According to one aspect of the present disclosure, the reflected component suppression image generating device, the reflected component suppression inference model generating device, the reflected component suppression image generating method, and the program, it is possible to reduce the influence of changes in the amount of infrared light caused by high-intensity light sources in gas equipment from the output image of the gas visualization imaging device, thereby contributing to improving the detection quality in gas leak detection. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic configuration diagram of a reflection component suppression image generation system according to an embodiment. [Figure 2] 1 is a schematic diagram showing the relationship between a monitoring target 300 and a gas visualization imaging device 10. FIG. [Figure 3] FIG. 2 is a diagram showing the configuration of a reflection component suppressed image generating device 20. [Figure 4] 1A is a functional block diagram of the control unit 21, and FIG. 1B is a schematic diagram showing an outline of the logical configuration of the machine learning model. [Figure 5] 10A and 10B are schematic diagrams for explaining the characteristics of image components of reflected light due to a flare stack in a gas distribution image. [Figure 6] FIG. 2 is a functional block diagram of a machine learning data generation device 30. [Figure 7]FIG. 2 is a functional block diagram of a control unit of the machine learning data generation device 30. [Figure 8] 1(a) and 1(b) are schematic diagrams showing the data structures of three-dimensional structure data and three-dimensional optical reflection image data, respectively. [Figure 9] FIG. 10 is a schematic diagram for explaining an overview of an optical reflection image calculation method in two-dimensional single-viewpoint optical reflection image conversion processing. [Figure 10] 1 is a flowchart showing an outline of a process for generating a two-dimensional optical reflection image as a training image in a machine-learning data generation device 30. [Figure 11] 10 is a flowchart showing an outline of a two-dimensional single-viewpoint reflection component image conversion process. [Figure 12] 10 is a flowchart showing the operation of the reflection component suppressed image generating device 20 in the learning phase. [Figure 13] 10 is a flowchart showing the operation of the reflection component suppressed image generating device 20 in the operation phase. [Figure 14] FIG. 10 is a process diagram showing an overview of a process for capturing a teacher image. [Figure 15] FIG. 10 is a functional block diagram of a control unit of a machine-learning data generation device 30A according to a second embodiment. [Figure 16] 10 is a flowchart showing an outline of a reflection component enhancement process in the machine learning data generation device 30A. [Figure 17] FIG. 1 is a schematic diagram showing the manner in which light is reflected from a flare stack in a gas facility. DETAILED DESCRIPTION OF THE INVENTION

[0012] First Embodiment <Configuration of reflection component suppression image generation system 1> An embodiment of the present disclosure is realized as a reflected component suppressed image generating system 1 that suppresses image components of reflected light in an inspection image that includes a background image portion in which high-brightness light from a flare stack or the like is irradiated onto an imaging target in a gas facility. The reflected component suppressed image generating system 1 according to the embodiment will be described in detail below with reference to the drawings.

[0013] 1 is a schematic diagram of a reflected component suppressed image generating system 1 according to an embodiment. As shown in Fig. 1, the reflected component suppressed image generating system 1 is composed of a plurality of gas visualization imaging devices 10, a reflected component suppressed image generating device 20, a machine learning data generating device 30, and a storage means 40, all connected to a communication network N.

[0014] The communication network N is, for example, the Internet, and the gas visualization imaging device 10, the reflected component suppression image generation device 20, multiple machine learning data generation devices 30, and the storage means 40 are connected so that they can exchange information with each other.

[0015] (Gas visualization imaging device 10, etc.) The gas visualization imaging device 10 is a device or system that uses infrared rays to capture an image of a monitored object and provides an infrared image in which the gas is visualized to the reflected component suppression image generation device 20. For example, it includes imaging means (not shown) made up of an infrared camera that detects infrared rays and captures an image, and an interface circuit (not shown) that outputs the image to a communication network N.

[0016] Images taken by infrared cameras are generally used to detect hydrocarbon gases, such as methane, ethane, ethylene, and propylene. For example, an image sensor with a sensitivity wavelength band in at least a portion of the infrared wavelength range of 3 μm to 5 μm, more preferably an infrared camera that detects and images infrared light with a wavelength of 3.2 μm to 3.4 μm, can detect hydrocarbon gases such as methane, ethane, ethylene, and propylene.

[0017] As shown in the schematic diagram of FIG. 2, the gas visualization imaging device 10 is installed so that the monitoring target 300 is included in the field of view 310 of the infrared camera. The obtained inspection image is a video signal for transmitting images at, for example, 30 frames per second. The gas visualization imaging device 10 converts the captured image into a predetermined video signal. In this embodiment, the infrared image signal acquired from the infrared camera is restored to an image and processed as a moving image consisting of multiple frames. The image is an infrared photograph of the monitoring target, and has infrared intensity as pixel values.

[0018] Note that if the size of the gas distribution image or the number of frames as a video is excessively large, the amount of calculation required for machine learning and for determining based on machine learning will increase. In the first embodiment, the number of pixels in the gas distribution image is 224 × 224 pixels, and the number of frames is 16.

[0019] The gas visualization imaging device detects the presence of gas by capturing changes in the amount of electromagnetic waves emitted from background objects with an absolute temperature of 0 (K) or higher. Changes in the amount of electromagnetic waves are mainly caused by the gas absorbing electromagnetic waves in the infrared region or by the gas itself emitting blackbody radiation. The gas visualization imaging device 10 can capture gas leaks as images by photographing the monitored space, so it can detect gas leaks earlier and pinpoint the location of the gas more accurately than conventional detection probe methods that can only monitor grid-like locations.

[0020] The visualized inspection image is temporarily stored in a memory or the like, and is transferred to and saved in the storage means 40 via the communication network N based on an operation input.

[0021] The gas visualization imaging device 10 is not limited to this, and any imaging device capable of detecting the gas to be monitored may be used, for example, a general visible light camera may be used if the gas to be monitored is a gas that can be detected with visible light, such as white smoke-like water vapor. In this specification, gas refers to gas that has leaked from a closed space such as a pipe or a tank, and is not intentionally dispersed into the atmosphere.

[0022] Returning to FIG. 1, the storage means 40 is a storage device that stores the inspection image transmitted from the gas visualization imaging device 10, and is configured to include a volatile memory such as a DRAM (Dynamic Random Access Memory) and a non-volatile memory such as a hard disk.

[0023] (Reflection component suppression image generation device 20) The reflected component suppressed image generating device 20 is a device that acquires an inspection image of a monitoring target from the gas visualization imaging device 10, reduces image components of reflected light in background image portions where high-brightness light such as a flare stack is irradiated onto the target, and provides a reflected component suppressed image in which the image components of reflected light are suppressed to a user via a display unit 24. The reflected component suppressed image generating device 20 is realized, for example, as a computer including a general CPU (Central Processing Unit), RAM (Random Access Memory), and programs executed by these. Note that, as will be described later, the reflected component suppressed image generating device 20 may further include a GPU (Graphics Processing Unit) and RAM as an arithmetic unit.

[0024] The following describes the configuration of the reflected component suppressed image generating device 20. Fig. 3 is a diagram showing the configuration of the reflected component suppressed image generating device 20. As shown in Fig. 3, the reflected component suppressed image generating device 20 includes a control unit (CPU) 21, a communication unit 22, a storage unit 23, a display unit 24, and an operation input unit 25, and is realized as a computer in which the control unit 21 executes a gas leak detection program.

[0025] The communication unit 22 transmits and receives information to and from the reflection component suppressed image generating device 20 and the storage means 40 .

[0026] The display unit 24 is, for example, a liquid crystal panel, and displays the display screen generated by the CPU 21.

[0027] The storage unit 23 stores a program 231 and the like necessary for the reflection component suppressed image generation device 20 to operate, and also functions as a temporary storage area for temporarily storing the calculation results of the CPU 21. The storage unit 23 includes a volatile memory such as a DRAM, and a non-volatile memory such as a hard disk.

[0028] The control unit 21 executes the gas leakage detection program 231 in the storage unit 23 to realize each function of the reflection component suppressed image generating device 20 .

[0029] FIG. 4( a ) is a functional block diagram of the control unit 21 .

[0030] 4(a), the reflection component suppressed image generation device 20 includes a test image input unit 211, a training image input unit 212, a correct answer image input unit 213, a machine learning unit 2141, a learning model holding unit 2142, and a determination result output unit 215. The machine learning unit 2141 and the learning model holding unit 2142 constitute a reflection component suppression unit 214. The test image input unit 211, the training image input unit 212, the correct answer image input unit 213, the reflection component suppression unit 214, and the determination result output unit 215 constitute a reflection component suppressed image generation unit 210.

[0031] The inspection image input unit 211 is a circuit that acquires an inspection image from the gas visualization imaging device 10. For example, an image capture board or other device that captures image data into a processing device such as a computer can be used. The inspection image is an infrared image captured by an infrared camera, and is an image showing the gas distribution by visualizing the gas leak part of the inspection target. The inspection image may also be a moving image including multiple frames of time-series data. When high-intensity light generated by a flare stack or the like is captured by the gas visualization imaging device 10, the inspection image may include a background image portion generated by light irradiating the imaging target, such as a gas facility. Gain adjustment, offset adjustment, image inversion, etc. may be performed as needed for subsequent processing.

[0032] The training image input unit 212 is a circuit that accepts input of a reflection component-containing image (hereinafter sometimes referred to as a "first image") that has the same format as the inspection image generated by the gas visualization imaging device 10 and includes an image component of high-intensity reflected light in an image portion where high-intensity light generated by a flare stack or the like is irradiated onto an object such as a gas facility. The first image may be a moving image including multiple frames of time-series data. The first image is output to the machine learning unit 2141 as a training image for machine learning.

[0033] If the acquired image is not in the same format as the gas distribution image generated by the test image input unit 211, the training image input unit 212 may perform processing such as cropping or scaling to make it in the same format. Also, for example, if the acquired image is three-dimensional voxel data, it may be converted into a two-dimensional image of a viewpoint from one point.

[0034] The correct image input unit 213 is a circuit that receives input of a reflection component-free image (hereinafter, sometimes referred to as a "second image") that has the same format as the inspection image generated by the gas visualization imaging device 10 and does not include image components of high-intensity reflected light in an image portion where high-intensity light generated by a flare stack or the like is irradiated onto an object such as a gas facility. The second image may also be a moving image including multiple frames of time-series data.

[0035] The second image is an image of the same object as the first image that forms a pair, and is captured or generated under the same conditions as the first image, except for the image component of reflected light. The second image is output to the machine learning unit 2141 as a correct answer image for machine learning.

[0036] The machine learning unit 2141 is a circuit that executes machine learning based on a combination of a first image received by the training image input unit 212 and a second image received by the correct image input unit 213, and generates a machine learning model. For example, a convolutional neural network (CNN) can be used for the machine learning, and known software such as PyTorch can be used.

[0037] FIG. 4(b) is a functional block diagram of the machine learning unit 2141 in the control unit 21. The machine learning model includes an input layer 51, an intermediate layer 52-1, an intermediate layer 52-2, ..., an intermediate layer 52-n, and an output layer 53, and an interlayer filter is optimized through learning. For example, if the pixel count of the image to be processed is 224 × 224 pixels and the number of frames is 16, the input layer 51 receives a 224 × 224 × 16 three-dimensional tensor into which the pixel values ​​of the image to be processed are input. The intermediate layer 52-1 is, for example, a convolutional layer, and receives a 224 × 224 × 16 three-dimensional tensor generated by a convolution operation from the data in the input layer 51. The intermediate layer 52-2 is, for example, a pooling layer, and receives a three-dimensional tensor obtained by resizing the data in the intermediate layer 52-1. The intermediate layer 52-n is, for example, a fully connected layer, and converts the data in the intermediate layer 52-(n-1) into a two-dimensional vector indicating coordinate values. The configuration of the intermediate layers is an example, and the number n of intermediate layers is about 3 to 5, but is not limited to this. Also, in FIG. 4(b), each layer is depicted as having the same number of neurons, but each layer may have any number of neurons. The machine learning unit 2141 receives a moving image as the processing target image as input, performs learning with the gas leak position as the correct answer, generates a machine learning model, and outputs the model to the learning model holding unit 2142.

[0038] In the reflection component suppression unit 214 according to this embodiment, the machine learning unit 2141 is made up of a machine learning model consisting of an input layer 51, an intermediate layer 52, and an output layer 53, and a model learning processing program. Each of the intermediate layers 52-1, 52-2, ..., 52-n is made up of multiple processing layers such as a convolution layer and a MaxPooling layer, and an image of the same scene not containing a reflection component as a ground truth image and an image containing a reflection component to which a reflection component has been added are input to the input layer 51 via the training image input unit 212 and the ground truth image input unit 213, respectively.

[0039] The output layer 53 is a part that outputs the intermediate learning results for each learning step. A machine learning model is formed through a procedure in which the model learning processing program compares the output results with the correct image and modifies the parameters (weights, gains, etc. of each node) of the intermediate layers 52-1, 52-2, ..., 52-n.

[0040] By inputting a large number of training data sets, which are pairs of a correct image that does not contain high-brightness reflection components and an image that contains high-brightness reflection components to which high-brightness reflection components have been added, into the machine learning unit 2141, the learning accuracy of the machine learning model can be improved.

[0041] Note that, if the reflection component suppressed image generation device 20 includes a GPU and a RAM as a computing device, the machine learning unit 2141 may be realized by the GPU and software.

[0042] Generally, in machine learning, a processing system is constructed that can perform processing similar to human shape recognition and recognition of changes over time by automatically adjusting parameters for convolution filter processing, etc., used in image recognition, etc., through a learning process. The machine learning model of the reflection component suppression unit 214 according to this embodiment can estimate the location of reflection components by capturing changing parts of synchronized high-luminance signals that appear in the input image, and generate a reflection component suppressed image.

[0043] Specifically, as shown in FIG. 5, a learning model for generating an image in which the high-brightness reflected light component is suppressed is estimated based on the following characteristics of the high-brightness reflected light image component.

[0044] FIG. 5 is a schematic diagram illustrating the characteristics of an image component of high-brightness reflected light due to a flare stack in a gas distribution image. The image component of high-brightness reflected light due to a flare stack is captured by the gas visualization imaging device 10 after light due to the flare stack is reflected by a structure. This high-brightness reflected light image component has the following characteristics. Specifically, (1) the position of the high-brightness reflected light image component in the gas distribution image is fixed. (2) Time changes are synchronized among multiple high-brightness reflected light image components in the gas distribution image. (3) The brightness magnitude relationship does not change among multiple high-brightness reflected light image components in the gas distribution image. (4) Even if there is a brightness fluctuation in the high-brightness reflected light image component, the shape of the periphery does not change, and the brightness distribution within the high-brightness reflected light image component does not change relatively. (5) The period of time changes among the high-brightness reflected light image components is within a predetermined range.

[0045] Therefore, the machine learning model is formed by extracting features such as the absolute value of brightness, outer shape, brightness distribution, area, position, change in position over time, change in area over time, change in brightness over time, period of change over time, synchronicity of change over time, etc., possessed by the image components of high-brightness reflected light in the image portion of the gas distribution image where light is irradiated onto the target, or a combination of these, to construct a machine learning inference model and predict the occurrence and magnitude of the image components of high-brightness reflected light.

[0046] The learning model holding unit 2142 is a circuit that holds the machine learning model generated by the machine learning unit 2141, and uses the machine learning model to generate and output a reflected component suppressed image in which the image components of high-brightness reflected light are suppressed in the gas distribution image acquired by the inspection image input unit 211, which includes an image portion where high-brightness light generated by a flare stack or the like is irradiated onto the target.

[0047] Since the learning model holding unit 2142 outputs high-brightness reflection components in the reflection-component-free image for the reflection-component-free image, the reflection component suppression unit 214 forms a high-brightness reflection-component-reduced image by reducing the high-brightness reflection components in the initial inspection image acquired from the inspection image input unit 211 based on the correspondence between the initial inspection image and the high-brightness reflection components, and calculates the error between the formed high-brightness reflection-component-reduced image and the correct image based on the machine learning model generated by the machine learning unit 2141. Then, the reflection component suppression unit 214 calculates the amount of update for the parameters (weights, gains, etc. of each node) of the intermediate layers 52-1, 52-2, ..., 52-n in the neural network to reduce the error. The formation of the high-brightness reflection-component-reduced image and the calculation of the error between the high-brightness reflection-component-reduced image and the correct image are repeated to generate a reflection component-suppressed image in which the high-brightness reflection components are suppressed. The amount of update for the parameters can be calculated using, for example, a known algorithm such as a gradient method, a nearest neighbor method, or an error backpropagation algorithm. As a result, an image in which reflection components are reduced is generated and output based on the inspection image consisting of the input gas visualization image.

[0048] As a result, the learning model holding unit 2142 generates and outputs an image in which the high-brightness reflection components of the inspection image, which is made up of the input gas visualization image, are reduced based on the machine learning model generated by the machine learning unit 2141.

[0049] The determination result output unit 215 is a circuit that generates a display image for displaying the second image output by the learning model holding unit 2142 on the display unit 24.

[0050] (Machine learning data generation device 30) The following describes the configuration of the machine learning data generation device 30. Fig. 6 is a diagram showing the configuration of the machine learning data generation device 30. As shown in Fig. 6, the machine learning data generation device 30 includes a control unit (CPU) 31, a communication unit 32, a storage unit 33, a display unit 34, and an operation input unit 35, and is realized as a computer in which the control unit 31 executes a machine learning data generation program.

[0051] The control unit 31 executes the machine learning data generation program 331 in the storage unit 33 to realize the functions of the machine learning data generation device 30.

[0052] Fig. 7 is a functional block diagram of the control unit of the machine learning data generation device 30. The condition parameters required for processing input to each functional block in Fig. 7(a) are as shown in the table below.

[0053] [Table 1]

[0054] As shown in Figure 7(a), the machine learning data generation device 30 includes a three-dimensional structure modeling unit 311, a temperature setting unit for each part 312, a three-dimensional optical lighting analysis simulation execution unit 313, and a two-dimensional single-viewpoint reflection component image conversion processing unit 314.

[0055] The three-dimensional structure modeling unit 311 designs a three-dimensional structure model based on condition parameters CP1 input from the operator to the operation input unit 35, performs three-dimensional structure modeling to lay out the structure in three-dimensional space, and outputs three-dimensional structure data DTstr to a subsequent stage. Examples of the condition parameters CP1 include parameters related to structural conditions such as the structure position and optical properties of the structure surface, such as reflectance and emissivity. The three-dimensional structure data DTstr is shape data that represents the three-dimensional shape of, for example, piping and other plant facilities. Commercially available three-dimensional CAD (Computer-Aided Design) software can be used for three-dimensional structure modeling.

[0056] 8(a) is a schematic diagram showing the data structure of the three-dimensional structure data DTstr. In this specification, the X, Y, and Z directions in each figure are defined as the width, depth, and height directions, respectively.

[0057] As shown in Figure 8(a), the three-dimensional structure data DTstr is three-dimensional voxel data representing a three-dimensional space and is composed of structure identification information Std arranged in coordinates in the X, Y, and Z directions. Because the structure identification information Std is expressed as three-dimensional shape data, it may be recorded as a binary image using 0 or 1, such as "structure present" or "structure absent." Alternatively, the three-dimensional structure data DTstr may be recorded as a multi-valued image such as 0, 1, 2, 3, etc., by assigning a value indicating the classification of the structure surface to each pixel. In this case, the three-dimensional structure data DTstr is identification data consisting of the structure's surface classification Std (Std = 0, 1, 2, 3, etc.). Here, the surface classification Std is a classification number, for example, based on the optical characteristics of the structure surface. For example, unpainted pipes may be assigned a value of 1, painted pipes a value of 2, and concrete a value of 3. This sets the structure's position and the optical characteristics of the structure surface as shown in the structural conditions.

[0058] The temperature setting unit 312 receives the three-dimensional structure data DTstr as input, and further assigns temperature conditions to each part of the structure surface based on the condition parameter CP2 input by the operator to the operation input unit 35, and outputs three-dimensional structure radiation data DTemt for the surface of the structure laid out in three-dimensional space to a subsequent stage. Examples of the condition parameter CP2 include parameters related to temperature conditions such as the structure temperature and the temperature around the structure. The temperature of the structure itself and the temperature around the structure are set, and for example, seasonal changes in the amount of infrared radiation can also be reflected in the learning.

[0059] The 3D optical lighting analysis simulation execution unit 313 receives the three-dimensional structural radiation data DTemt as input and further acquires condition parameters CP3 required for the optical lighting analysis simulation based on the operator's input of the condition parameters CP3 to the operation input unit 35. The condition parameters CP3 are parameters that determine the setting conditions required for the optical lighting analysis simulation, mainly related to lighting conditions, such as the on / off status, quantity, position, emission intensity, and time variation of high-intensity lighting sources such as flare stacks, and the time variation of the intensity of background lighting due to sunlight conditions, as shown in Table 1. Background lighting is lighting that reproduces changes in illuminance due to weather, and its intensity is sufficiently lower than that of high-intensity lighting and changes gradually over time. High-intensity lighting is lighting that generates reflected components. In addition to the time variation of intensity, the position and number are also set. By generating images by varying these condition parameters in various ways, it is possible to generate a large amount of training data.

[0060] Then, in the three-dimensional space where the three-dimensional structure modeling has been performed, a three-dimensional optical illumination analysis simulation is performed to generate three-dimensional optical reflection image data DTrf, which is output to the subsequent stage. The three-dimensional optical reflection image data DTrf is data that includes at least a three-dimensional distribution of optical reflection characteristics. The calculation is performed using commercially available software for optical illumination analysis simulation, such as ANSYS SPEOS.

[0061] 8(b) is a schematic diagram showing the data structure of the three-dimensional optical reflection image data DTrf. As shown in FIG. 8(b), the three-dimensional optical reflection image data DTrf is three-dimensional voxel data representing a three-dimensional space, and includes optical reflection surface normal vectors and optical reflection luminance data Lu (W / m 2 ) The optical reflection luminance data Lu of each voxel may have an absolute value that changes based on the viewpoint position SP (X, Y, Z) described later. The three-dimensional optical reflection image data DTrf may be a moving image including a plurality of time-series three-dimensional voxel time-series data.

[0062] The 2D single-view reflection component image conversion processor 314 inputs and acquires the 3D optical reflection image data DTrf. Furthermore, based on the condition parameters CP4 input by the operator to the operation input unit 35, it acquires the condition parameters CP4 required for conversion processing into a 2D single-view image. The condition parameters CP4 are parameters related to the imaging conditions of the gas visualization imaging device, such as the imaging device's field of view, line of sight direction, distance, and image resolution, as shown in Table 1. The 2D single-view reflection component image conversion processor 314 then converts the 3D optical reflection image data DTrf into 2D optical reflection image data DTrf2 observed from a predetermined viewpoint. Based on the 3D structure data output by the 3D structure model design and the imaging conditions, a 2D image captured by the imaging device is generated. In this case, the 2D optical reflection image data DTrf2 may also be a moving image including multiple frames of time-series data.

[0063] Then, as the two-dimensional optical reflection image data DTrf2 serving as training data, a pair of image data DTrfon containing high-brightness reflection components based on a three-dimensional optical illumination analysis simulation under the condition that the high-brightness illumination light source is on (hereinafter, sometimes referred to as "image data containing reflection components DTrfon") and image data DTrfoff not containing high-brightness reflection components based on a three-dimensional optical illumination analysis simulation under the condition that the high-brightness illumination light source is off (hereinafter, sometimes referred to as "image data not containing reflection components DTrfoff") are generated. The pair of generated two-dimensional optical reflection image data DTrf2, i.e., the set of image data DTrfon containing reflection components and image data DTrfoff not containing reflection components, is output to the reflection component suppressed image generation device 20 as training data for machine learning.

[0064] The two-dimensional optical reflection image data DTrf2 is an image equivalent to the inspection image acquired by the gas visualization imaging device 10, and is an image that represents how the object appears from the viewpoint. Furthermore, by taking into consideration the information of the structure three-dimensional data DTstr, it is possible to generate two-dimensional optical reflection image data DTrf2 that does not reflect the object parts that are blocked by the structure and cannot be observed from the viewpoint.

[0065] FIG. 9 is a schematic diagram for explaining an outline of a method for calculating two-dimensional optical reflection image data DTrf2 in two-dimensional single-viewpoint reflection component image conversion processing.

[0066] The two-dimensional single viewpoint reflection component image conversion processing unit 314 generates multiple values ​​of the optical reflection image represented by the three-dimensional optical reflection image data DTrf when observed in the line of sight direction from a preset viewpoint position (X, Y, Z) by changing the angles θ and σ of the line of sight direction, and arranges the values ​​of the obtained optical reflection image two-dimensionally to generate two-dimensional optical reflection image data DTrf2.

[0067] Specifically, as shown in FIG. 9 , an arbitrary viewpoint position SP(X, Y, Z) is set in three-dimensional space, and a virtual image plane VF is set at a position a predetermined distance from the viewpoint position SP(X, Y, Z) in the direction of the three-dimensional structure indicated by the three-dimensional optical reflection image data DTrf. At this time, the virtual image plane VF is set so that its center O intersects with a line passing through the viewpoint position SP(X, Y, Z) and the central voxel of the three-dimensional optical reflection image data DTrf. The image frame of the virtual image plane VF is set according to the angle of view of the gas visualization imaging device 10. Thus, the line of sight direction DA from the viewpoint position SP(X, Y, Z) toward a pixel of interest A(x, y) on the virtual image plane VF is inclined by an angle θ in the X direction and an angle σ in the Y direction with respect to the line of sight direction DO toward the center pixel O, i.e., the line of sight direction DO of the gas visualization imaging device.

[0068] The voxel of the 3D optical reflection image data that first intersects with the line of sight along the line of sight direction DA corresponding to this pixel of interest A(x,y) is detected. The voxel that first intersects with the line of sight starting from the viewpoint position SP(X,Y,Z) is located in the visible area as seen from the viewpoint position SP(X,Y,Z). Therefore, of the optical reflection luminance data Lu of the voxel, the optical reflection luminance data Lu emitted in the line of sight direction DA is calculated as the value of the 2D optical reflection image data DTrf2 for the pixel of interest A(x,y).

[0069] As shown in FIG. 9, for an invisible region that cannot be observed from the gas visualization imaging device 10 because it is blocked by a structure that first intersects with the line of sight, the three-dimensional position of the structure is taken into consideration, and for voxels that exist behind the structure as viewed from the viewpoint position SP(X, Y, Z), the two-dimensional optical reflection image data DTrf2 is not calculated for the pixel of interest A(x, y).

[0070] Then, while changing the angles θ and σ according to the angle of view of the gas visualization imaging device 10, the position of the pixel of interest A(x, y) is gradually moved, and the calculation of the values ​​of the two-dimensional optical reflection image data is repeated with all pixels on the virtual image plane VF as the pixel of interest A(x, y), thereby calculating the two-dimensional optical reflection image data DTrf2.

[0071] Furthermore, by using the same three-dimensional optical reflection image data DTrf and varying the viewpoint position SP (X, Y, Z) to generate two-dimensional optical reflection image data DTrf2, multiple two-dimensional optical reflection image data DTrf2 can be easily generated from a single fluid simulation.

[0072] 6, the storage unit 33 stores a program 331 and the like necessary for the machine-learning data generation device 30 to operate, and also functions as a temporary storage area for temporarily storing the calculation results of the control unit 31. The storage unit 33 includes a volatile memory such as a DRAM, and a non-volatile memory such as a hard disk.

[0073] The communication unit 32 transmits and receives information to and from the machine learning data generation device 30 and the storage means 40.

[0074] The display unit 34 is, for example, a liquid crystal panel, and displays the display screen generated by the CPU 31.

[0075] <Machine learning data generation processing> Next, as an example of a flow for generating training data for machine learning, a method for generating an image using a three-dimensional simulation, that is, an operation for generating two-dimensional optical reflection image data DTrf2 by the machine learning data generation device 30 will be described.

[0076] FIG. 10 is a flowchart showing an outline of the process of generating a two-dimensional optical reflection image as a training image in the machine-learning data generation device 30.

[0077] First, based on the operational input of the condition parameters CP1 relating to the structural conditions, the three-dimensional structure modeling unit 311 designs a three-dimensional structure model (step S101), and outputs three-dimensional structure data DTstr to the subsequent stage.

[0078] Next, based on the operational input of the condition parameter CP2 relating to the temperature conditions, the temperature setting unit 312 for each part sets the temperature of the structure and the structure surface (step S102), assigns temperature conditions to each part of the structure surface, and outputs the structure radiation three-dimensional data DTemt on the surface of the structure laid out in three-dimensional space.

[0079] Next, based on the operation input of the condition parameter CP3 related to the lighting conditions, high-intensity lighting by flare stacks / background lighting due to weather are set (step S103), and the viewpoint position and distance are set. With other condition parameters as common conditions, the 3D optical lighting analysis simulation execution unit 313 uses known optical lighting analysis simulation software to calculate the 3D reflected light and luminance on the structure surface under high-intensity lighting on / off conditions (step S104), and generates a pair of 3D optical reflection image data DTrf corresponding to high-intensity lighting on / off and outputs it to a subsequent stage.

[0080] Next, based on the operational input of the condition parameter CP4 relating to the imaging conditions, the 2D single-viewpoint reflection component image conversion processing unit 314 performs 2D single-viewpoint image conversion processing to generate an image containing / not containing a reflection component (step S105), and outputs a pair of image data DTrfon containing a reflection component and image data DTrfoff not containing a reflection component corresponding to high-brightness lighting on / off as training data for machine learning.

[0081] Next, a two-dimensional single-viewpoint reflection component image conversion processing method will be described.

[0082] 11 is a flowchart showing an outline of the two-dimensional single-viewpoint reflection component image conversion process, which is executed by the two-dimensional single-viewpoint reflection component image conversion processing unit 314, whose functions are configured by the control unit 31.

[0083] First, the 2D single-view reflection component image conversion processing unit 314 acquires the structure 3D data DTstr (step S401), acquires the 3D optical reflection image data DTrf (when there is flare / high-brightness illumination is on), and further acquires the 3D optical reflection image data DTrf (when there is no flare / high-brightness illumination is off) (step S402).

[0084] Next, based on the operation input, the system accepts input of information regarding, for example, the imaging device field angle, line of sight direction, distance, and image resolution as condition parameters CP4 (step S403). Furthermore, based on the operation input, the system sets a viewpoint position SP (X, Y, Z) corresponding to the position of the imaging portion of the gas visualization imaging device 10 in the three-dimensional space (step S404).

[0085] Next, a virtual image plane VF is set at a predetermined distance from the viewpoint position SP (X, Y, Z) in the direction of the three-dimensional structure, and the position of the image frame on the virtual image plane VF is calculated according to the angle of view of the gas visualization imaging device 10, as described above (step S405).

[0086] Next, the coordinates of the pixel of interest A(x, y) are set to initial values ​​(step S406), and the position LV on the line of sight from the viewpoint position SP(X, Y, Z) toward the pixel of interest A(x, y) on the virtual image plane VF is set to initial values ​​(step S407).

[0087] Next, it is determined whether or not the structure identification information Std of the voxel of the structure three-dimensional data DTstr that intersects with the line of sight represents "no structure" (Std=0) (step S408).

[0088] In step S408, if the three-dimensional structure data DTstr intersecting the line of sight is "structure present", the brightness value data (Lu) at the intersection voxel with the three-dimensional optical reflection image data DTrf (high-brightness illumination on condition) during flare stack is output as an image with reflection components (step S409), and further the brightness value data (Lu) at the intersection voxel with the three-dimensional optical reflection image data DTrf (high-brightness illumination off condition) during non-flare stack is output as an image with reflection components (step S410), the position of the pixel of interest A(x, y) is gradually moved (step S411), and the process returns to step S407.

[0089] On the other hand, if the result is not "structure present," it is determined whether or not calculation has been completed for the entire line of sight, which corresponds to the range where the line of sight intersects with the voxel (step S412). If not, the line of sight position LV is incremented by a unit length (step S413), and the process returns to step S408. On the other hand, if calculation has been completed, it is determined whether or not calculation has been completed for all pixels on the virtual image plane VF (step S414). If not, the position of the target pixel A(x, y) is gradually moved (step S415), and the process returns to step S407. If calculation has been completed, the process ends. The standard value set when there is no structure is determined as the brightness value data for the target pixel A. Here, the standard value is, for example, brightness value data corresponding to the ground or sky in real space. The standard value can be obtained by appropriately setting the conditions indicated by the condition parameters CP1 and CP2.

[0090] As a result, two-dimensional optical reflection image data DTrf2 is generated for all pixels on the virtual image plane VF during flare stacking and during non-flare stacking. That is, a set of image data DTrfon containing reflection components and image data DTrfoff not containing reflection components is generated for the virtual image plane VF.

[0091] Next, it is determined whether the generation of the two-dimensional optical reflection image data DTrf2 has been completed for all viewpoint positions SP (X, Y, Z) to be calculated (step S416). If not completed, the process returns to step S404 and two-dimensional optical reflection image data DTrf2 is generated for the new viewpoint position SP (X, Y, Z) input by operation; if completed, the process ends.

[0092] As described above, a 3D optical illumination analysis simulation is performed while various setting conditions are varied, and from the results, 3D optical reflection image data is obtained under high-intensity illumination-off conditions and high-intensity illumination-on conditions. Then, by converting this into 2D optical reflection image data using 2D single-viewpoint processing, it is possible to efficiently generate a large amount of training data sets consisting of pairs of image data without reflection components and image data with reflection components under the same conditions.

[0093] In gas equipment inspections, it is considered effective to use machine learning to identify the location of gas leak sources hidden deep within complex networks of equipment such as piping from inspection images. However, machine learning generally requires tens of thousands of correct answer data, and in order to achieve this, it is necessary to efficiently acquire a large amount of training data related to gas equipment.

[0094] In contrast, by using the machine learning data generation device 30, a large amount of training data sets can be generated efficiently, which can contribute to improving the learning accuracy.

[0095] <Machine learning data generation processing> The operation of the reflection component suppressed image generating device 20 according to this embodiment will be described below with reference to the drawings.

[0096] <Learning Phase> FIG. 12 is a flowchart showing the operation of the reflection component suppressed image generating device 20 in the learning phase.

[0097] First, the machine learning data generation device 30 creates a pair of image data DTrfon containing reflection components corresponding to high-intensity illumination on / off and image data DTrfoff not containing reflection components under equivalent conditions (step S10). Each pair of training images consists of multiple frames of time-series data.

[0098] As a pair of image data DTrfon containing a reflection component corresponding to high-intensity illumination on / off and image data DTrfoff not containing a reflection component under the same conditions, 3D optical reflection image data can be used, which are 2D optical reflection image data observed from a predetermined viewpoint. The 3D optical reflection image data can be based on 3D optical illumination analysis simulation. For example, a 3D conceptual model of a gas facility can be created using commercially available 3D computer-aided design (CAD) software, and a 3D optical illumination analysis simulation can be performed using commercially available 3D optical illumination analysis simulation software taking into account the structural model. The 3D optical reflection image data obtained as a result of the simulation can be converted into a 2D image observed from a predetermined viewpoint.

[0099] Next, a combination of a pair of image data DTrfon containing a reflection component corresponding to high-brightness illumination on / off and image data DTrfoff not containing a reflection component under equivalent conditions is input to the machine learning unit 2141, with the image not containing a reflection component being the correct image (step S11). The image data DTrfon containing a reflection component is input to the training image input unit 212, and the corresponding image data DTrfoff not containing a reflection component is input to the correct image input unit 213. At this time, image data that has been subjected to processing such as gain adjustment may be input as necessary.

[0100] Next, data is input into the convolutional neural network to perform machine learning (step S12). As a result, parameters are optimized by trial and error through deep learning, and a machine-learned model is formed. The formed machine-learned model is stored in the learning model storage unit 2142.

[0101] Through the above operations, a machine-learned model is created that outputs an image in which high-brightness reflected components are suppressed based on the characteristics of an image containing high-brightness reflected light.

[0102] <Operation of generating an image with suppressed reflection components> FIG. 13 is a flowchart showing the operation of the reflection component suppressed image generating device 20 in the operation phase.

[0103] First, an inspection image acquired by the gas visualization imaging device 10 is input from the inspection image input unit 211 to the trained model holding unit 2142 (step S30). The inspection image is image data in the same format as the training image and is composed of multiple frames of time-series data. The inspection image is an infrared image captured by the infrared camera of the gas visualization imaging device 10, and is a moving image showing the gas distribution by visualizing the gas leak part of the inspection target. The inspection image may be subjected to subtraction of an offset component or gain adjustment. When high-brightness light generated by a flare stack or the like is captured, the inspection image includes a background image portion where the light is irradiated onto the imaging target, such as a gas facility, as a high-brightness reflection component. A portion may be cut out from each frame of the captured image so as to include all pixels where gas is detected, and the inspection image may be generated as a frame of the gas distribution image.

[0104] Next, a reflection component suppressed image is generated using the machine-learned model (step S31). By using the machine-learned model formed in step S12, a reflection component suppressed image in which high-brightness reflection components contained in the inspection image are reduced is generated using the inspection image as input.

[0105] Next, the high brightness reflected component suppressed image is displayed on the display unit (step S32).By this process, the reflected component suppressed image is generated.

[0106] <Small summary> When a gas visualization image in an environment where a high-intensity light source such as a flare stack is present is input to the reflected component suppressed image generating device 20 according to this embodiment having the above configuration, a gas visualization image in which the reflected component has been satisfactorily removed is obtained.

[0107] As a technique for reducing the effect of brightness changes in visualizing gases from infrared video, for example, Patent Document 3 discloses a technique for inputting images captured with at least two different exposure times and removing flicker components. This is a technique for removing flicker that occurs in illumination light sources such as fluorescent lamps, but the flicker to be removed is periodic, making it difficult to remove random brightness changes such as those caused by flare stacks.

[0108] In contrast, the reflected component suppression image generating device 20 according to the present embodiment uses an image illuminated with a high-brightness light source and an image not illuminated as a learning set, and generates an image from the gas leak image in which the reflected component due to the high-brightness light source illumination has been removed using a learning model obtained by machine learning. This makes it possible to eliminate the influence of changes in the amount of infrared light due to the high-brightness light source, thereby improving the detection rate of gas leaks.

[0109] As described above, the reflected component suppression image generating device according to this embodiment can reduce the influence of changes in the amount of infrared light caused by a high-intensity light source in a gas facility from the output image of the gas visualization imaging device, thereby contributing to improving the detection quality in gas leak detection.

[0110] <Variation 1> The reflection component suppressed image generating device 20 according to the first embodiment has been described above, but the present disclosure is not limited to the above embodiment except for its essential characteristic components. Below, as an example of such a configuration, a modified example of the above embodiment will be described.

[0111] In Modification 1, an example will be described in which two-dimensional optical reflection image data DTrf2 is acquired by imaging. This shows an example in which experimental photographic data is actually acquired using a gas visualization imaging device.

[0112] FIG. 14 is a process diagram showing an outline of the imaging process of the teacher image.

[0113] First, a structure is placed in a studio or the like (step S10A). In this case, the structure placement setting involves setting the position of the structure to be photographed and the optical property conditions of the structure surface, as shown in the structural conditions in Table 1. The structure may be a simulated plant facility that imitates a plant facility, or a model structure, which allows for photography experiments under illumination by a high-intensity lighting source. To make the optical properties of the structure surface equivalent to those of the actual plant facility, the surface is treated with a coating or other surface treatment.

[0114] Next, the temperature of the structure and the temperature of the structure surface are set using a heating device (step S11A). Here, as shown in the temperature conditions in Table 1, the temperature of the structure itself and the temperature around the structure are set as shown in the temperature conditions in Table 1 to reflect changes in the amount of infrared rays due to the seasons in the learning.

[0115] Next, high-intensity lighting is set using a high-intensity lighting source with a flare stack, and background lighting due to weather is set using natural light or normal lighting (step S12A). Here, as shown in the lighting conditions in Table 1, high-intensity lighting is lighting that generates the reflected components that are the target of removal in this case. In addition to the time change in intensity, the position and number are also set. Background lighting is lighting that reproduces changes in illuminance due to weather, and its intensity is sufficiently lower than that of the high-intensity lighting and changes gradually over time.

[0116] Then, imaging conditions such as imaging position, distance, angle of view, and resolution are set for the infrared camera of the gas visualization imaging device (step S13A), and video is captured by the gas visualization imaging device with and without high-intensity illumination (step S14A). Here, imaging conditions such as the angle of view and viewpoint of the imaging device are set as shown in the imaging conditions in Table 1.

[0117] By taking images with the high-brightness lighting turned off while changing the various settings described above, an image without reflected components can be obtained, and then by taking images with the high-brightness lighting turned on, an image with reflected components can be obtained, thereby obtaining a variety of learning data.

[0118] When a gas visualization image in an environment where a high-intensity light source such as a flare stack is present is input to the reflected component removal image generating device, a gas visualization image in which the reflected component has been effectively removed is obtained.

[0119] Second Embodiment The following describes the device for generating machine-learning data 30A according to embodiment 2. Fig. 15 is a functional block diagram of the control unit of the device for generating machine-learning data 30A. The condition parameters required for processing input to each functional block in Fig. 15 are as shown in the table above. The same components as those in the device for generating machine-learning data 30 are assigned the same numbers and will not be described again.

[0120] <Configuration> The machine learning data generation device 30A differs from the machine learning data generation device 30 in that it newly includes a reflection component enhancement processing unit 315A after the two-dimensional single-viewpoint reflection component image conversion processing unit 314. The reflection component enhancement processing is an enhancement processing for a predetermined frequency of time-series images so that the behavior of image components when high-brightness light generated by a flare stack or the like is irradiated onto an object can be emphasized.

[0121] The reflection component enhancement processing unit 315A extracts specific frequency components from the high-brightness reflection component-containing image data DTrfon and applies various enhancement processes to the high-brightness reflection image components caused by flare stacks, etc., to generate various reflection component-enhanced image data DTrem.

[0122] <Operation of reflection component enhancement processing> Next, the operation of the reflection component enhancement process in the machine learning data generation device 30A will be described with reference to the drawings.

[0123] 16 is a flowchart showing an outline of the reflected component enhancement process. First, the reflected component enhancement processor 315A acquires time-series data of the high-brightness reflected component-containing image data DTrfon (in flare / high-brightness illumination ON condition) (step S201).

[0124] For this time-series pixel data, the time-series signal of the luminance of each pixel is decomposed into time-frequency components (step S202). Here, the time-frequency decomposition is performed using a method such as Fourier transform or wavelet transform.

[0125] Next, specific frequency component data is extracted, and various gain adjustments are applied to each frequency component to generate emphasis data for various frequencies (step S203).

[0126] Next, the time series signals of the luminance of each pixel are restored to generate a restored image (step S204). The restoration to the time series signals uses a method such as inverse Fourier transform or inverse wavelet transform that corresponds to the method used in the time-frequency decomposition.

[0127] Finally, the reflected component emphasized image data DTrem is output (step S205), and the process ends.

[0128] <Small summary> As described above, the machine-learning data generation device 30A can generate a variety of reflection-component-enhanced image data DTrem by applying a variety of enhancement processes to high-brightness reflection image components caused by flare stacks, etc. by varying the gain adjustment for each frequency component. The generated reflection-component-enhanced image data DTrem and reflection-component-free image data DTrfoff can be used as a set as training data for machine learning in the reflection-component-suppressed image generation device 20, allowing for the efficient generation of a large number of sets of training data and contributing to improved training accuracy.

[0129] Other variations Although the gas leak detection device according to the embodiment has been described above, the present disclosure is not limited to the above embodiment except for its essential characteristic components. For example, the present disclosure also includes various modifications that can be made to the embodiment by a person skilled in the art, and modifications that can be realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of the present invention. Below, modifications of the above embodiment will be described as examples of such modifications.

[0130] (1) In the above-described embodiment, a gas plant is used as an example of a gas facility as an inspection image. However, the present disclosure is not limited to this and may be applied to the generation of display images for gas-using equipment, devices, laboratories, research labs, factories, and offices.

[0131] (2) The present disclosure has been described based on the above embodiment, but the present disclosure is not limited to the above embodiment, and the following cases are also included in the present invention.

[0132] For example, the present invention may be a computer system having a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating in accordance with the computer program. For example, the present invention may be a computer system having a computer program for processing the reflection component suppressed image generation system 1 of the present disclosure or its components, and operating in accordance with this program (or instructing each connected component to operate).

[0133] The present invention also includes a case where all or part of the processing in the reflection component suppression image generation system 1 or its components is implemented by a computer system comprising a microprocessor, recording media such as ROM and RAM, a hard disk unit, etc. The RAM or hard disk unit stores a computer program that achieves the same operations as each of the above devices. Each device achieves its function when the microprocessor operates in accordance with the computer program.

[0134] Furthermore, some or all of the components constituting each of the above devices may be configured as a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured to include a microprocessor, ROM, RAM, etc. These may be individually integrated into single chips, or may be integrated into a single chip to include some or all of them. The RAM stores a computer program that achieves the same operations as each of the above devices. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program. For example, the present invention also includes a case where the processing of the reflection component suppression image generation system 1 or its components is stored as an LSI program, and this LSI is inserted into a computer to execute a predetermined program (gas inspection management method).

[0135] The method of integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. It is also possible to use FPGAs (Field Programmable Gate Arrays), which can be programmed after the LSI is manufactured, or reconfigurable processors, which allow the connections and settings of circuit cells inside the LSI to be reconfigured.

[0136] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derived technologies, it is of course possible to use that technology to integrate functional blocks.

[0137] Furthermore, some or all of the functions of the reflection component suppression image generation system 1 or its components according to each embodiment may be realized by a processor such as a CPU executing a program. A non-transitory computer-readable recording medium may also be used, on which a program for implementing the operations of the reflection component suppression image generation system 1 or its components is recorded. The program may be executed by another independent computer system by recording a program or a signal on a recording medium and transferring the program. It goes without saying that the program can be distributed via a transmission medium such as the Internet.

[0138] Furthermore, the reflection component suppression image generation system 1 according to the above embodiment or each of its components may be configured to be realized by a programmable device such as a CPU, a GPU (Graphics Processing Unit), or a processor, and software. These components may be a single circuit component, or may be a collection of multiple circuit components. Furthermore, multiple components may be combined to form a single circuit component, or may be a collection of multiple circuit components.

[0139] (3) The division of functional blocks is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block. Furthermore, the functions of multiple functional blocks with similar functions may be processed in parallel or time-shared by a single piece of hardware or software.

[0140] The order in which the steps are performed is merely an example for specifically explaining the present invention, and other orders may be used. Some of the steps may be performed simultaneously (in parallel) with other steps.

[0141] Furthermore, at least some of the functions of the respective embodiments and their modified examples may be combined. Furthermore, the numbers used above are all examples for specifically explaining the present invention, and the present invention is not limited to the exemplified numbers.

[0142] <Summary> As described above, the reflection component suppression image generating device according to the present embodiment includes an inspection image input unit that receives as input a gas distribution image in which a gas presence region in a space is visualized and which includes an image portion in which light is irradiated onto an object; The inspection image input unit is characterized by comprising a reflected component suppression image generation unit that generates a reflected component suppression image in which the image components of reflected light in the image portion of the gas distribution image received by the inspection image input unit are suppressed, using an inference model that has been machine-learned using as training data a combination of a first image including an image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and that is equivalent to the first image in terms of elements other than the first image.

[0143] In another aspect, in any of the above aspects, the inference model may be an inference model that is machine-learned using the second image as a correct image.

[0144] In another aspect, in any of the above aspects, the image input to the inspection image input unit, the first image, and the second image may be a moving image including multiple frames.

[0145] In another aspect, in any of the above aspects, the image component of the reflected light may be a time-varying component in an image portion where the light is irradiated onto the object.

[0146] In another aspect, in any of the above aspects, the first image may be an image obtained by amplifying a specific frequency component in the time direction.

[0147] In another aspect, in any of the above aspects, the first image may be an image obtained by simulation.

[0148] In another aspect, in any of the above aspects, the light irradiated onto the target may be light irradiated from a light source based on a flare stack.

[0149] In another aspect, in any of the above aspects, the image portion in the first image where light is irradiated onto the object includes an image component of reflected light based on a flare stack, In the second image, an image portion where light is not irradiated onto the object may be configured not to include an image component of reflected light due to the flare stack.

[0150] The reflection component suppression inference model generating device according to this embodiment includes an image input unit, The configuration may also include a machine learning unit that uses as training data a combination of a first image including an image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and which is equivalent to the first image in terms of elements other than the image portion, performs machine learning using an image including the image portion where light is irradiated onto the object as input, and generates an inference model that outputs a reflected component suppressed image in which the image component of reflected light in the image portion is suppressed.

[0151] In another aspect, in any of the above aspects, the image input to the inspection image input unit, the first image, and the second image may be a moving image including multiple frames.

[0152] In another aspect, in any of the above aspects, the image component of the reflected light may be a time-varying component of a high-intensity portion in an image portion where the light is irradiated onto an object.

[0153] In another aspect, in any of the above aspects, the first image may be an image obtained by amplifying a specific frequency component in the time direction.

[0154] Furthermore, the reflection component suppression image generation method according to the present embodiment includes receiving, as input, a gas distribution image in which a gas presence region in space is visualized and which includes an image portion in which light is irradiated onto an object; The configuration may also be such that a reflection component suppressed image in which the image components of reflected light in the image portion of the gas distribution image received by the inspection image input unit is generated using an inference model that is machine-learned using as training data a combination of a first image including an image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and that is equivalent to the first image in terms of elements other than the first image.

[0155] Further, the program according to the present embodiment is a program that causes a computer to perform a reflection component suppressed image generation process, The reflection component suppressed image generation process includes: A gas distribution image is received as an input, in which a region where gas exists in a space is visualized and which includes an image portion where light is irradiated onto the target; The configuration may also be such that a reflection component suppressed image in which the image components of reflected light in the image portion of the gas distribution image received by the inspection image input unit is generated using an inference model that is machine-learned using as training data a combination of a first image including an image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and that is equivalent to the first image in terms of elements other than the first image.

[0156] <<Additional Information>> The embodiments described above each illustrate a preferred specific example of the present invention. The numerical values, components, component placement and connection, processing method, processing order, etc. shown in the embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the embodiments, those not described in the independent claims that represent the highest concept of the present invention are described as optional components that constitute more preferred embodiments.

[0157] The order in which the above methods are performed is merely an example for specifically explaining the present invention, and other orders may be used. Also, some of the above methods may be performed simultaneously (in parallel) with other methods.

[0158] In order to facilitate understanding of the invention, the scale of the components in the drawings of the above embodiments may differ from the actual scale. Furthermore, the present invention is not limited to the description of the above embodiments, and can be modified as appropriate within the scope of the gist of the present invention. [Industrial Applicability]

[0159] The machine learning data generation device, machine learning data generation method, and learning dataset according to the embodiments of the present disclosure are widely applicable to systems used to inspect gas leaks in gas equipment. [Explanation of symbols]

[0160] 1. Reflection suppression image generation system 10 Gas visualization imaging device 20 Reflection component suppression image generation device 21 Control unit (CPU) 210 Reflection component suppression image generation unit 211 Inspection image input unit 212 Training Image Input Unit 213 Correct image input unit 214 Reflection component suppression section 2141 Machine Learning Department 2142 Learning Model Storage Unit 215 Judgment result output unit 22 Communications Department 23 Memory section 231 Programs 24 Display section 25 Operation input section 30 Machine learning data generation device 31 Control unit (CPU) 311 3D Structure Modeling Department 312 Temperature setting section 313 3D Optical Lighting Analysis Simulation Department 314 2D single-view reflection component image conversion processing unit 315 Reflection component emphasis processing unit 32 Communications Department 33 Storage section 331 Program 34 Display section 35 Operation input section 40 Memory means

Claims

1. an inspection image input unit that receives as input a gas distribution image in which a gas presence region in a space is visualized and which includes an image portion in which light is irradiated onto an object; a reflected component suppressed image generating unit that generates a reflected component suppressed image in which image components of reflected light in the image portion of the gas distribution image received by the inspection image input unit are suppressed using an inference model that is machine-learned using as training data a combination of a first image including an image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and that is equivalent to the first image with respect to elements other than the first image, The image component of the reflected light is a time-varying component in the image portion where the light is irradiated onto the object. Reflection component suppression image generation device.

2. The inference model is an inference model that is machine-learned using the second image as a correct image. The reflection component suppression image generating device according to claim 1 .

3. The image input to the inspection image input unit, the first image, and the second image are moving images including a plurality of frames.

3. The image generating device according to claim 1, wherein the reflected component is suppressed.

4. The first image is an image obtained by amplifying a specific frequency component in the time direction. The reflection component suppression image generating device according to any one of claims 1 to 3.

5. The first image is an image obtained by simulation. The reflection component suppression image generating device according to any one of claims 1 to 4.

6. The light irradiated onto the target is light irradiated from a light source based on a flare stack. The reflection component suppression image generating device according to any one of claims 1 to 5.

7. an image portion in the first image where light is irradiated onto the object includes an image component of reflected light based on a flare stack; In the second image, an image portion where light is not irradiated onto the object does not include an image component of reflected light due to the flare stack. The reflection component suppression image generating device according to any one of claims 1 to 6.

8. an image input unit; a machine learning unit that uses a combination of a first image including an image portion where light is irradiated onto an object and a second image including an image portion where light is not irradiated onto the object and equivalent to the first image in elements other than the image portion as training data, executes machine learning using an image including the image portion where light is irradiated onto the object as input, and generates an inference model that outputs a reflection component suppressed image in which image components of reflected light in the image portion are suppressed, The image component of the reflected light is a time-varying component in the image portion where the light is irradiated onto the object. Reflex component suppression inference model generator.

9. The image input to the image input unit, the first image, and the second image are moving images including a plurality of frames. The reflection component suppression inference model generating device according to claim 8 .

10. The first image is an image obtained by amplifying a specific frequency component in the time direction. The reflection component suppression inference model generating device according to claim 8 or 9.

11. A gas distribution image is received as an input, in which a region where gas exists in a space is visualized and which includes an image portion where light is irradiated onto the target; generating a reflected component suppressed image in which image components of reflected light in the image portion of the received gas distribution image are suppressed using an inference model machine-learned using as training data a combination of a first image including an image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and which is equivalent to the first image in terms of elements other than the first image; The image component of the reflected light is a time-varying component in the image portion where the light is irradiated onto the object. Reflection component suppression image generation method.

12. A program that causes a computer to perform a reflection component suppression image generation process, The reflection component suppressed image generation process includes: A gas distribution image is received as an input, in which a region where gas exists in a space is visualized and which includes an image portion where light is irradiated onto the target; generating a reflected component suppressed image in which image components of reflected light in the image portion of the received gas distribution image are suppressed using an inference model machine-learned using as training data a combination of a first image including an image portion where light is irradiated onto the object and a second image including an image portion where light is not irradiated onto the object and which is equivalent to the first image in terms of elements other than the first image; The image component of the reflected light is a time-varying component in the image portion where the light is irradiated onto the object. program.

Images