Oil leakage image generation device and oil leakage detection device

Abstract

To design a configuration for detecting oil leakage from an image to be inspected based on various situations in which oil leakage may occur.SOLUTION: An oil leakage image generation device 200 comprises a data acquisition section 221 and an image generation section 222. The data acquisition section 221 acquires an input image ImA2 in which an oil leakage area An2 in which oil leaks from a device is designated in an image including the device. The image generation section 222 generates an output image ImO2 by drawing oil leaking from the device in the oil leakage area An2 so as to match the characteristic information Fa2 of the oil leakage area An2.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present disclosure relates to an oil leakage image generating device and an oil leakage detection device. [Background technology]

[0002] Conventionally, there are known configurations for detecting oil leakage from an image of an inspection target. For example, Japanese Patent Application Laid-Open No. 2013-101474 (Patent Document 1) discloses an oil leakage detection device that divides a color image of a device into multiple small blocks and determines that oil is present in a portion of the color image where the intensity of a specific color (e.g., yellow) is equal to or greater than a threshold value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-101474 A Summary of the Invention [Problem to be solved by the invention]

[0004] The color of oil in an image of an object to be inspected may vary depending on the circumstances under which the oil leak occurs. For example, if the object to be inspected contains crushed stone or concrete, the oil adhering to these may appear to be closer to black than yellow. Therefore, with a configuration designed on the premise that the color of oil in an image of an object to be inspected is a specific color, such as the oil leak detection device of Patent Document 1, it may be difficult to detect oil of a color other than the specific color from an image of an object to be inspected.

[0005] The present disclosure has been made to solve the problems described above, and its purpose is to make it possible to design a configuration for detecting oil leakage from an image of an inspection object based on various situations in which oil leakage may occur. [Means for solving the problem]

[0006] An oil leakage image generating device according to one aspect of the present disclosure includes a data acquisition unit and an image generation unit. The data acquisition unit acquires an input image including an equipment and a specified oil leakage region where oil leaks from the equipment. The image generation unit generates an output image by drawing the oil leaking from the equipment in the oil leakage region so as to match the characteristic information of the oil leakage region.

[0007] According to another aspect of the present disclosure, there is provided an oil leakage detection device including a data acquisition unit and an inference unit. The data acquisition unit acquires an inspection image. The inference unit infers whether or not oil leaked from a device is captured in the inspection image by using an inference model trained by machine learning. Effect of the Invention

[0008] According to the present disclosure, by drawing oil leaking from equipment in an oil leakage area so as to match the characteristic information of the oil leakage area and by using machine learning, a configuration for detecting oil leakage from an image can be designed based on various situations in which oil leakage may occur. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a configuration of an analysis system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the analysis device of FIG. [Diagram 3] FIG. 13 is a diagram showing an example of an image without oil leakage, and an example of an input image (annotation image) in which an oil leakage region is specified in the image without oil leakage. [Figure 4] 13 is a diagram showing other examples of an image without oil leakage and an annotation image. FIG. [Diagram 5] 13 is a diagram showing other examples of an image without oil leakage and an annotation image. FIG. [Figure 6] FIG. 13 is a diagram showing how the color of oil leaked from equipment changes in an image of leaked oil depending on the color of the portion (background) to which the oil is attached. [Figure 7] FIG. 13 is a diagram showing how the color of oil leaking from equipment changes in an image of leaked oil depending on the area where the oil overlaps with a shadow. [Figure 8] FIG. 13 is a diagram showing how the color of oil in an oil leakage image changes depending on the darkness of the shadow covering the oil leaked from equipment. [Figure 9] 3 is a flowchart showing an example of a process flow of an oil leakage image generating function realized by a processor that executes the oil leakage image generating program of FIG. 2. [Figure 10] 3 is a flowchart showing an example of the process flow of an oil leakage detection function and an oil leakage image generation function realized by a processor that executes the oil leakage detection program and the oil leakage image generation program of FIG. 2. [Figure 11] FIG. 11 is a diagram illustrating an example of a hardware configuration of an analysis device according to a second embodiment. [Figure 12] 12 is a block diagram showing a functional configuration when the processor of FIG. 11 executes a machine learning program for generating an oil leakage image and functions as a learning device. FIG. [Figure 13] 13 is a flowchart showing an example of the flow of a learning process performed by the analysis device of FIG. 12. [Figure 14] 12 is a block diagram showing a functional configuration when the processor of FIG. 11 executes an oil leakage image generating program to function as an oil leakage image generating device. FIG. [Figure 15] 15 is a flowchart showing an example of the flow of an image generation process performed by the analysis device of FIG. 14. [Figure 16] 12 is a block diagram showing a functional configuration when the processor of FIG. 11 executes a machine learning program for oil leakage detection and functions as a learning device. FIG. [Figure 17] 17 is a flowchart showing an example of the flow of a learning process performed by the analysis device of FIG. 16. [Figure 18] 12 is a block diagram showing a functional configuration when the processor of FIG. 11 executes an oil leakage detection program and functions as an oil leakage detection device. FIG. [Figure 19] 19 is a flowchart showing an example of the flow of an oil leakage detection process performed by the analysis device of FIG. 18. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference characters, and their description will not be repeated in principle.

[0011] Embodiment 1 FIG. 1 is a diagram showing an example of the configuration of an analysis system 20 according to the first embodiment. The analysis system 20 may collect and analyze images of equipment 160 (inspection target) installed in any facility 10 such as a power plant, a substation, or a factory. The images collected by the analysis system 20 may be used for abnormality judgment of the equipment 160 by a rule-based judgment method such as pattern matching, machine learning of a model for performing abnormality judgment of the equipment 160, generation of an oil leakage image for the rule-based judgment method or machine learning, or visual inspection. The equipment 160 may include, for example, a switchgear or transformer installed in a substation, or any equipment installed in any facility. For example, cooling oil or insulating oil is used in the equipment 160. An abnormality of the equipment 160 includes oil leaking from the equipment 160.

[0012] Analysis system 20 includes analysis device 100, database 120, terminal 130, access point 140, and surveillance camera 150. Analysis device 100, database 120, terminal 130, access point 140, and surveillance camera 150 are connected to each other via local network 40. Analysis device 100, database 120, or terminal 130 may be connected to a wide area network outside facility 10. Analysis system 20 may cooperate with a terminal, an access point, and a surveillance camera that are installed in advance in facility 10, instead of terminal 130, access point 140, and surveillance camera 150.

[0013] The surveillance camera 150 captures an image of the equipment 160 in the facility 10. The surveillance camera 150 also transmits an image of the equipment 160 to the analysis device 100 via the local network 40 in the facility 10. The image of the equipment 160 can be used to detect an abnormality such as oil leakage from the equipment 160. A plurality of surveillance cameras 150 may be installed in the facility 10. The images captured by the surveillance camera 150 include still images and moving images. The surveillance camera 150 can be realized by a fixed camera, a mobile camera that runs on rails, a camera mounted on a drone, a camera mounted on a self-propelled robot, or a combination of these.

[0014] The analysis device 100 associates the image acquired from the surveillance camera 150 with information on the device 160 (e.g., manufacturer, model, type, category of the device 160, or start time of use, etc.) and stores the image in the database 120. The analysis device 100 associates the image with the position information of the oil included in the image acquired from the surveillance camera 150, color information of the periphery or background of the oil, information on the shadow overlapping with the oil (e.g., the range of the shadow or the density (darkness) of the shadow), and information on the situation when the image was taken (e.g., the image date and time or the weather) and stores the image in the database 120. The analysis device 100 may acquire information on the sunlight conditions and weather at the time when the image acquired from the surveillance camera 150 was taken from an external weather data server. The analysis device 100 may estimate the position of the shadow of the image acquired from the surveillance camera 150 from the sunlight conditions at the time when the image was taken. Analysis device 100 may have, for example, a function of extracting image features, a function of calculating a threshold for classifying images based on the features, a function of filtering images that do not contain oil leakage (images without oil leakage), etc. Database 120 may be configured as a non-volatile memory of analysis device 100.

[0015] Terminal 130 can be used, for example, by a user of facility 10 (e.g., an on-site maintenance worker or operator) to check images stored in analysis device 100 or database 120. Terminal 130 may be installed inside facility 10 or may be installed at an external monitoring station 30 or the like.

[0016] Analysis system 20 may be placed in a control room or the like within a facility. Also, a part of analysis system 20 may be placed outside the facility, and for example, analysis device 100 may be virtually constructed in a cloud environment or may be a server.

[0017] The analysis system 20 mainly has a function of collecting images of the equipment 160, an annotation function for the images of the equipment 160, an oil leakage image generation function, and an oil leakage detection function.

[0018] The function of collecting images of equipment 160 is a function of capturing images of one or more equipment 160 by one or more surveillance cameras 150 in facility 10. Surveillance cameras 150 may capture images of equipment 160 at regular intervals and transmit the images to analysis device 100. Surveillance cameras 150 may capture images of equipment 160 at predetermined time intervals (e.g., every two hours).

[0019] The annotation function for the image of the equipment 160 is a function for extracting an area where oil leakage may occur in the image of the equipment 160 stored in the database 120 based on an input operation by the user on the terminal 130. Such areas include, for example, welded parts, bolt fastening parts, radiator parts, and bushing porcelain tubes. The area where oil leakage may occur may be automatically extracted based on the shape and color information (RGB (Red / Green / Blue) or HSV (Hue / Saturation / Value Brightness)) of objects included in the image. Hereinafter, the area extracted as an area where oil leakage may occur is also referred to as an oil leakage area.

[0020] The oil leakage image generation function is a function that generates an oil leakage image by drawing oil in the oil leakage area of ​​the image of the equipment 160. The oil leakage detection function is a function that determines whether or not the image of the equipment 160 captured by the surveillance camera 150 is an oil leakage image by using a rule-based method such as pattern matching or a trained inference model. For example, in pattern matching, whether or not the image is an oil leakage image is determined based on the similarity between the image and an oil leakage image prepared in advance in the database 120 as a comparison target template.

[0021] Fig. 2 is a diagram showing an example of a hardware configuration of analysis device 100 of Fig. 1. Analysis device 100 includes a processor 101, a primary storage device 102, a secondary storage device 103, and a communication interface 104.

[0022] Processor 101 may execute programs for implementing various functions of analysis device 100. Processor 101 is configured, for example, by at least one integrated circuit. The integrated circuit may be configured, for example, by at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one field programmable gate array (FPGA), or a combination of these.

[0023] The primary storage device 102 includes a volatile memory and stores programs executed by the processor 101 and data referenced by the processor 101. The primary storage device 102 may be realized by a dynamic random access memory (DRAM) or a static random access memory (SRAM), etc.

[0024] The secondary storage device 103 may include a non-volatile memory and store a program (e.g., an OS (Operating System) program) executed by the processor 101 and data referenced by the processor 101. In this case, the processor 101 executes a program read from the secondary storage device 103 to the primary storage device 102, and references data read from the secondary storage device 103 to the primary storage device 102. The secondary storage device 103 may be realized by a hard disk drive (HDD), a solid state drive (SSD), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, or the like.

[0025] The secondary storage device 103 stores an oil leakage detection program Pad and an oil leakage image generation program Pig. The processor 101 that executes the oil leakage detection program Pad realizes an oil leakage detection function. The processor 101 that executes the oil leakage image generation program Pig realizes an oil leakage image generation function and an annotation function. That is, the analysis device 100 functions as an oil leakage image generation device and an oil leakage detection device. The oil leakage detection program Pad and the oil leakage image generation program Pig may be configured as a single program.

[0026] The communication interface 104 may be connected to any external device such as a printer, a scanner, and an external HDD. The communication interface 104 may be realized by a USB (Universal Serial Bus) terminal or the like. The communication interface 104 is also connected to a wired or wireless network device. The communication interface 104 may be realized by a wired LAN (Local Area Network) port, a Wi-Fi (registered trademark) (Wireless Fidelity) module, or the like. The communication interface 104 may transmit and receive data using communication protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) and UDP (User Datagram Protocol).

[0027] FIG. 3 is a diagram showing an example of an image without oil leakage and an input image (annotation image) in which an oil leakage region is specified in the image without oil leakage. As shown in FIG. 3, the image without oil leakage Im1 includes radiators Rd11, Rd12 and a fastening part Bf1. Since oil is likely to leak from each of the radiators Rd11, Rd12 and the fastening part Bf1, in the annotation image Im1A, oil leakage regions An11, An12, and An13 are respectively added to the lower parts of the radiators Rd11, Rd12 and the fastening part Bf1. Note that the specification of the oil leakage region is not limited to the visual specification as shown in FIGS. 3 to 5, and may be specified without visual information such as coordinates.

[0028] Fig. 4 is a diagram showing other examples of images without oil leakage and annotation images. As shown in Fig. 4, the image without oil leakage Im2 includes the radiator Rd2 and the fastening part Bf2. In the annotation image Im2A, oil leakage areas An21 and An22 are added to the lower parts of the radiator Rd2 and the fastening part Bf2, respectively.

[0029] FIG. 5 is a diagram showing other examples of the no-oil-leakage image and the annotation image. The imaging position of the no-oil-leakage image Im3 shown in FIG. 5 is the same as the imaging position of the no-oil-leakage image Im2 in FIG. 4. Therefore, the multiple objects included in the no-oil-leakage image Im3 are the same as the multiple objects included in the no-oil-leakage image Im2. On the other hand, the imaging time of the no-oil-leakage image Im3 is different from the imaging time of the no-oil-leakage image Im2. Therefore, the appearance of the shadow included in the no-oil-leakage image Im3 is different from the appearance of the shadow included in the no-oil-leakage image Im2. In the annotation image Im3A, oil leakage areas An31 and An32 are also added to the lower parts of the radiator Rd2 and the fastening part Bf2, respectively.

[0030] 6 is a diagram showing how the color of oil leaked from equipment 160 changes in an oil leakage image depending on the color of the portion (background) to which the oil is attached. As shown in FIG. 6, in an oil leakage image Im11, oil Lo1 is attached to a background Bg1. Oil leakage images Im12, Im13, and Im14 are oil leakage images in which the color of the background Bg1 of the oil leakage image Im11 is darkened in this order. If the oil Lo1 is transparent or semi-transparent, the background Bg1 can be seen through the oil Lo1, so the darker the color of the background Bg1, the darker the color of the oil Lo1.

[0031] 7 is a diagram showing how the color of oil leaked from equipment 160 changes in an oil leakage image depending on the area where the oil overlaps with a shadow. As shown in FIG. 7, oil leakage image Im21 includes oil Lo2 and a shadow Sd2. Oil leakage images Im22, Im23, and Im24 are oil leakage images in which the area where oil Lo2 overlaps with the shadow Sd2 in oil leakage image Im21 is expanded in this order. The part of oil Lo2 that overlaps with the shadow Sd1 is darker than the part of oil Lo2 that does not overlap with the shadow Sd1, so the wider the area where oil Lo2 overlaps with the shadow Sd1, the wider the relatively dark area in oil Lo2.

[0032] Fig. 8 is a diagram showing how the color of oil in an oil leakage image changes depending on the darkness of the shadow covering the oil leaked from the equipment 160. As shown in Fig. 8, in an oil leakage image Im31, the oil Lo3 is covered by a shadow Sd3. Oil leakage images Im32, Im33, and Im34 are oil leakage images in which the shadow Sd3 of the oil leakage image Im31 is darkened in that order. The darkness of the shadow Sd3 reduces the brightness of the oil Lo3, so the darker the shadow Sd3, the darker the oil Lo3.

[0033] As shown in Figs. 6 to 8, the appearance of oil contained in an oil leakage image differs depending on the situation at the time the oil leakage image was captured (for example, weather, date and time, or background, etc.). Therefore, in the analysis system 20, the oil depicted in the oil leakage area of ​​the image without oil leakage is changed depending on the situation at which the image without oil leakage was captured. The oil leakage image generated in this manner reflects various situations in which oil leakage may occur in the equipment 160. Therefore, by using the oil leakage image as a template for pattern matching or learning data for machine learning, the analysis system 20 can be designed to adapt to various situations in which oil leakage may occur.

[0034] Fig. 9 is a flowchart showing an example of the process flow of the oil leakage image generation function realized by the processor 101 (data acquisition unit and image generation unit) that executes the oil leakage image generation program Pig in Fig. 2. The process shown in Fig. 9 is called by a main routine (not shown) that comprehensively controls the analysis system 20. Below, each step is simply abbreviated as S.

[0035] As shown in FIG. 9, the processor 101 (data acquisition unit) acquires an image of the equipment 160 without oil leakage in S101 and proceeds to processing in S102. The processor 101 (data acquisition unit) extracts an oil leakage area from the image without oil leakage in S102 to generate an annotation image and proceeds to processing in S103. The oil leakage area may be extracted by a user's manual operation or automatically. The processor 101 (data acquisition unit) acquires characteristic information (e.g., background color, weather information, imaging time, shadow range, or shadow density) of the oil leakage area of ​​the annotation image in S103 and proceeds to processing in S103. The processor 101 (image generation unit) selects an oil leakage image including an oil area matching the characteristic information of the oil leakage area of ​​the annotation image from the oil leakage images stored in the database 120 in S104 and proceeds to processing in S105. In S105, the processor 101 (image generation unit) generates a new oil leakage image by superimposing an area of ​​oil from the selected oil leakage image that matches the characteristic information of the oil leakage area in the annotation image onto the oil leakage area in the annotation image, and returns processing to the main routine.

[0036] In S105, the processor 101 may obtain a plurality of pieces of feature information by changing at least one parameter (for example, background color, date and time, weather, range of shadow, or shadow density) included in the feature information of the oil leakage area of ​​the annotation image, and generate a plurality of output images by drawing oil leaking from equipment in the same oil leakage area of ​​the annotation image so as to match each of the plurality of pieces of feature information. For example, when the processor 101 changes the background color of the feature information, the processor 101 generates a plurality of output images Im11 to Im14 having the same oil leakage area but different background colors as shown in FIG. 6. When the processor 101 changes the range of shadow of the feature information, the processor 101 generates a plurality of output images Im21 to Im24 having the same oil leakage area but different shadow ranges as shown in FIG. 7. When the processor 101 changes the shade density of the feature information, the processor 101 generates a plurality of output images Im31 to Im34 having the same oil leakage area but different shade density as shown in FIG. 8.

[0037] The processing of the oil leakage detection function and the oil leakage image generation function may be performed continuously. Fig. 10 is a flowchart showing an example of the processing flow of the oil leakage detection function and the oil leakage image generation function realized by the processor 101 executing the oil leakage detection program Pad and the oil leakage image generation program Pig of Fig. 2. The flowchart shown in Fig. 10 is a flowchart in which S101 of the flowchart of Fig. 9 is replaced with S201, S202, S203, and S204. S102 to S105 of Fig. 10 are the same as S102 to S105 of Fig. 9, respectively.

[0038] As shown in FIG. 10, in S201, the processor 101 acquires an inspection image including the device 160, and proceeds to processing in S202. In S202, the processor 101 determines whether or not the inspection image includes oil leakage. If the inspection image does not include oil leakage (NO in S202), the processor 101 determines that the image is an image without oil leakage, executes S102 to S105 as in FIG. 9, and returns processing to the main routine. If the inspection image includes oil leakage (YES in S202), the processor 101 adds the inspection image to a template of an oil leakage image referred to in a rule-based anomaly detection method, or to learning data for machine learning in S203, and proceeds to processing in S204. In S204, the processor 101 extracts an oil leakage area from an area other than the oil leakage location in the inspection image, and proceeds to S103. The processor 101 executes S103 to S105 as in FIG. 9, and returns processing to the main routine.

[0039] The processor 101 as the image generating unit may receive a notification from the user including the result of oil leakage detection by the oil leakage detection function based on a comparison (for example, pattern matching) between an inspection image including the equipment 160 and an output image. For example, if the user checks the inspection image and determines that the detection of oil leakage in the inspection image is a false positive, the user inputs the false positive to the terminal 130. When the processor 101 receives a notification of a false positive from the user via the terminal 130 that oil leakage has been detected in an inspection image that does not show oil leaking from the equipment 160, the processor 101 may darken the color of the oil leakage area in the output image generated after receiving the notification than the color of the oil leakage area in the output image used for comparison (the output image generated before receiving the notification). By darkening the color of the oil leakage area compared to before the occurrence of the false positive, the difference between the oil leakage area and an area where no oil leakage has occurred is emphasized in the comparison between the inspection image including the equipment 160 and the output image. As a result, further false positives after the false positive can be suppressed. Note that the location of the oil leakage in the inspection image may be specified by the user in the notification from the user.

[0040] Furthermore, when the processor 101 as the image generating unit receives a notification of a missed detection from a user that an oil leak was not detected in an inspection image showing oil leaking from equipment 160, the processor 101 may add the inspection image to a comparison template used in oil leak detection. Since the number of oil leak patterns included in the comparison template increases, further missed detections following the notification of the missed detection can be suppressed.

[0041] As described above, according to the first embodiment, the configuration for detecting oil leakage from an image can be designed based on various situations in which oil leakage may occur.

[0042] Embodiment 2 In embodiment 2, a configuration is described in which the oil leakage image generation function and oil leakage detection function of embodiment 1 are realized by a trained model constructed by machine learning.

[0043] Fig. 11 is a diagram showing an example of a hardware configuration of the analysis device 200 according to the second embodiment. The hardware configuration of the analysis device 200 is a configuration in which the oil leakage detection program Pad and the oil leakage image generation program Pig in Fig. 2 are replaced with an oil leakage detection program Pad2 and an oil leakage image generation program Pig2, respectively, and a machine learning program Pma and an inference model Min for oil leakage detection, and a machine learning program Pmi and an oil leakage image generation model Mig for oil leakage image generation are added to a secondary storage device. Other than these, the configuration of the analysis device 200 is similar to that of the analysis device 100, and therefore the description of the similar configuration will not be repeated.

[0044] Fig. 12 is a block diagram showing a functional configuration when the processor 101 in Fig. 11 executes the machine learning program Pmi for generating oil leakage images and functions as a learning device. The following describes a case where the analysis device 200 performs supervised learning on the oil leakage image generation model Mig. Here, supervised learning is machine learning that uses a set of data of input parameters and output results (correct answers) as learning data, learns features contained in the learning data, and optimizes a model to infer the output result from the input parameters.

[0045] As shown in FIG. 12, the processor 101 that executes the machine learning program Pmi includes a data acquisition unit 211 and a model generation unit 212. The data acquisition unit 211 acquires learning data Ldg. The learning data Ldg includes an annotation image ImA1 including an oil leakage region An1, feature information Fa1 of the oil leakage region An1, and an oil leakage image ImO1. The oil leakage image ImO1 is an image in which oil that matches the feature information Fa1 is drawn in the oil leakage region An1. The learning data Ldg can be acquired, for example, from the database 120. The oil leakage image ImO1 is a correct image that the oil leakage image generation model Mig should generate from the annotation image ImA1 and the feature information Fa1.

[0046] The model generation unit 212 performs supervised learning on the oil leakage image generation model Mig using the learning data Ldg acquired by the data acquisition unit 211, and sets the oil leakage image generation model Mig as a trained model. The trained oil leakage image generation model Mig can generate an annotation image that was not used as learning data and an oil leakage image that matches the feature information of the oil leakage area of ​​the annotation image.

[0047] Fig. 13 is a flowchart showing an example of the flow of the learning process performed by the analysis device 200 of Fig. 12. With reference to Fig. 13 and Fig. 12 together, the data acquisition unit 211 acquires learning data Ldg in S201. Note that the annotation image ImA1, the feature information Fa1, and the oil leakage image ImO1 do not need to be acquired simultaneously, and may be acquired at different times and associated with each other.

[0048] In S202, the model generation unit 212 performs supervised learning on the oil leakage image generation model Mig using the learning data Ldg acquired by the data acquisition unit 211, generates a trained oil leakage image generation model Mig, and proceeds to S203. In S203, the model generation unit 212 stores the trained oil leakage image generation model Mig in the secondary storage device 103, and ends the process.

[0049] The machine learning performed by the model generation unit 212 is not limited to supervised learning. The model generation unit 212 may use, for example, reinforcement learning, unsupervised learning, or semi-supervised learning to make the inference model Min a trained model. The machine learning may also use deep learning, which learns to extract the feature amount itself. The model generation unit 212 may perform machine learning according to other known methods (for example, genetic programming, inductive logic programming, or support vector machine).

[0050] FIG. 14 is a block diagram showing a functional configuration when the processor 101 of FIG. 11 executes the oil leakage image generation program Pig2 to function as an oil leakage image generation device. As shown in FIG. 14, the analysis device 200 includes a data acquisition unit 221 (second data acquisition unit) and an image generation unit 222. The data acquisition unit 221 acquires input data Din. The input data Din includes an annotation image ImA2 including an oil leakage region An2 and feature information Fa2 of the oil leakage region An2. The image generation unit 222 generates and outputs an oil leakage image ImO2 (output image) from the input data Din using a trained oil leakage image generation model Mig. Note that the image generation unit 222 may acquire a trained oil leakage image generation model not from the secondary storage device 103 but from an external learning device.

[0051] Fig. 15 is a flowchart showing an example of the flow of the image generation process performed by the analysis device 200 of Fig. 14. As shown in Fig. 15, the data acquisition unit 221 acquires input data Din in S211. In S212, the image generation unit 222 inputs the input data Din acquired by the data acquisition unit 221 into the trained oil leakage image generation model Mig stored in the secondary storage device 103 to generate an oil leakage image ImO2, and proceeds to processing in S213. In S213, the image generation unit 222 outputs the oil leakage image ImO2 and ends the processing.

[0052] Fig. 16 is a block diagram showing a functional configuration when the processor 101 in Fig. 11 executes the machine learning program Pma for oil leakage detection and functions as a learning device. In the following, a case where the analysis device 200 performs supervised learning on the inference model Min will be described.

[0053] As shown in FIG. 16, the processor 101 that executes the machine learning program Pma includes a data acquisition unit 231 and a model generation unit 232. The data acquisition unit 231 acquires learning data Ldi. The learning data Ldi includes an image of the equipment 160 and a label (correct answer data) indicating whether or not the image includes an oil leakage. The image included in the learning data Ldi may be an oil leakage image generated by the analysis device 100 of the first embodiment or the analysis device 200 of FIG. 14. The learning data Ldi can be acquired from the database 120, for example.

[0054] The model generation unit 232 performs supervised learning on the inference model Min using the learning data Ldi acquired by the data acquisition unit 231, and sets the inference model Min as a trained model. The trained inference model Min can determine whether or not an oil leak is included in an image of the equipment 160 that was not used as learning data.

[0055] Fig. 17 is a flowchart showing an example of the flow of the learning process performed by analysis device 200 of Fig. 16. With reference to Fig. 17 and Fig. 16 together, data acquisition section 231 acquires learning data Ldi in S221.

[0056] In S222, the model generation unit 232 performs supervised learning on the inference model Min using the learning data Ldi acquired by the data acquisition unit 231, generates a trained inference model Min, and proceeds to S223. In S223, the model generation unit 232 stores the trained inference model Min in the secondary storage device 103, and ends the process.

[0057] The machine learning performed by the model generation unit 232 is not limited to supervised learning. The model generation unit 232 may use, for example, reinforcement learning, unsupervised learning, or semi-supervised learning to make the inference model Min a trained model. The machine learning may also use deep learning, which learns to extract the feature amount itself. The model generation unit 232 may perform machine learning according to other known methods (for example, genetic programming, inductive logic programming, or support vector machine).

[0058] Fig. 18 is a block diagram showing a functional configuration when the processor 101 of Fig. 11 executes the oil leakage detection program Pad2 and functions as an oil leakage detection device. As shown in Fig. 18, the analysis device 200 includes a data acquisition unit 241 (second data acquisition unit) and an inference unit 242. The data acquisition unit 241 acquires an inspection image Im40 including an equipment 160. The inference unit 242 infers whether or not the inspection image Im40 contains oil leakage using a trained inference model Min, and outputs the presence or absence of oil leakage in the inspection image Im40 as an inference result Ri. Note that the inference unit 242 may acquire the trained inference model not from the secondary storage device 103 but from an external learning device.

[0059] Fig. 19 is a flowchart showing an example of the flow of oil leakage detection processing performed by the analysis device 200 of Fig. 18. As shown in Fig. 19, the data acquisition unit 241 acquires an inspection image Im40 in S241. The inference unit 242 inputs the inspection image Im40 acquired by the data acquisition unit 241 to a trained inference model Min stored in the secondary storage device 103 in S242, and advances the processing to S213. The inference unit 242 outputs an inference result of the inference model Min (presence or absence of oil leakage) in S243, and ends the processing.

[0060] As described above, according to the second embodiment, the configuration for detecting oil leakage from an image can be designed based on various situations in which oil leakage may occur.

[0061] The embodiments disclosed herein are also intended to be combined appropriately within a range that is not inconsistent. The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is indicated by the claims, not the above description, and is intended to include all modifications within the meaning and scope of the claims.

[0062] Various aspects of the present disclosure are summarized below as appendices.

[0063] [Appendix 1] a data acquisition unit that acquires an input image including an equipment and having an oil leakage region where oil leaks from the equipment specified therein; and an image generating unit that generates an output image by drawing the oil leaking from the equipment in the oil leakage area so as to match the characteristic information of the oil leakage area.

[0064] [Appendix 2] Further comprising a database for storing at least one oil spill image; In the database, each of the at least one oil leakage image is associated with color information of the oil reflected in the oil leakage image; the characteristic information includes the color information, The oil leakage image generating device of claim 1, wherein the image generating unit generates the output image using an oil leakage image among the at least one oil leakage image that is associated with color information that matches the color information of the oil leakage area.

[0065] [Appendix 3] The oil leakage image generating device according to claim 1 or 2, wherein the image generating unit generates a plurality of output images by varying the feature information to obtain a plurality of pieces of feature information and drawing the oil leaking from the equipment in the oil leakage area so as to match each of the plurality of pieces of feature information.

[0066] [Appendix 4] The oil leakage image generating device according to any one of appendices 1 to 3, wherein when the image generating unit receives a notification that an oil leakage has been detected in the inspection image in an oil leakage detection based on a comparison between an inspection image that does not show oil leaking from the equipment and the output image, the image generating unit makes the color of the oil leakage area darker than the color of the oil leakage area in the output image used in the comparison.

[0067] [Appendix 5] The oil leakage image generating device according to any one of Appendices 1 to 4, wherein when the image generating unit receives a notification that the oil leakage was not detected in the inspection image in the oil leakage detection based on a comparison between the inspection image showing oil leaking from the equipment and the output image, the image generating unit adds the inspection image to a comparison template used in the oil leakage detection.

[0068] [Appendix 6] The image generation unit generates the output image using an image generation model that has been trained by machine learning, The oil leakage image generating device according to any one of appendices 1 to 5, wherein the learning data used in the machine learning includes an input image in which an area where oil leaks from equipment is specified, feature information of the oil leakage area, and a correct answer image in which oil leaking from the equipment is depicted in the oil leakage area.

[0069] [Appendix 7] A data acquisition unit for acquiring an inspection image; An oil leakage detection device comprising an inference unit that uses an inference model that has been trained by machine learning to infer whether or not the inspection image shows oil leaking from an equipment.

[0070] [Appendix 8] The oil leak detection device described in Appendix 7, wherein the learning data used in the machine learning includes an oil leak image generated by an oil leak image generating device described in any one of Appendixes 1 to 6. [Explanation of symbols]

[0071] 10 Facility, 20 Analysis system, 40 Local network, 100, 200 Analysis device, 101 Processor, 102 Primary storage device, 103 Secondary storage device, 104 Communication interface, 120 Database, 130 Terminal, 140 Access point, 150 Surveillance camera, 160 Equipment, 211, 221, 231, 241 Data acquisition unit, 212, 232 Model generation unit, 222 Image generation unit, 242 Inference unit, An1, An2, An11 to An13, An21, An22, An31, An32 Oil leakage area, Bf1, Bf2 Fastening part, Bg1 Background, Fa1, Fa2 Feature information, Im1 to Im3 Image without oil leakage, Im1A, Im2A, Im3A, ImA1, ImA2 Annotation image, Im11~Im14, Im21~Im24, Im31~Im34 output image, Im40 inspection image, Ldg, Ldi learning data, Lo1~Lo3 oil, Mig oil leak image generation model, Min inference model, Pad, Pad2 oil leak detection program, Pig, Pig2 oil leak image generation program, Pma, Pmi machine learning program, Rd2, Rd11, Rd12 radiator, Ri inference result, Sd1~Sd3 shadow.

Claims

1. a data acquisition unit that acquires an input image including an equipment and having an oil leakage region where oil leaks from the equipment specified therein; and an image generating unit that generates an output image by drawing the oil leaking from the equipment in the oil leakage area so as to match the characteristic information of the oil leakage area.

2. Further comprising a database for storing at least one oil spill image; In the database, each of the at least one oil leakage image is associated with color information of oil reflected in the oil leakage image; the characteristic information includes the color information, The oil leakage image generating device according to claim 1 , wherein the image generating unit generates the output image using an oil leakage image, among the at least one oil leakage image, that is associated with color information that matches color information of the oil leakage region.

3. The oil leakage image generating device according to claim 1, wherein the image generating unit generates a plurality of output images by varying the feature information to obtain a plurality of pieces of feature information and drawing the oil leaking from the equipment in the oil leakage area so as to conform to each of the plurality of pieces of feature information.

4. The oil leakage image generating device of claim 1, wherein when the image generating unit receives a notification that an oil leakage has been detected in the inspection image in an oil leakage detection based on a comparison between an inspection image that does not show oil leaking from the equipment and the output image, the image generating unit makes the color of the oil leakage area darker than the color of the oil leakage area in the output image used for the comparison.

5. The oil leakage image generating device of claim 1, wherein when the image generating unit receives a notification that the oil leakage was not detected in the inspection image in an oil leakage detection based on a comparison between the inspection image showing oil leaking from the equipment and the output image, the image generating unit adds the inspection image to a comparison template used in the oil leakage detection.

6. The image generation unit generates the output image using an image generation model that has been trained by machine learning, The oil leakage image generating device of claim 1, wherein the learning data used in the machine learning includes an input image in which an area where oil leaks from equipment is specified, feature information of the oil leakage area, and a correct answer image in which oil leaking from the equipment is depicted in the oil leakage area.

7. A data acquisition unit for acquiring an inspection image; An oil leakage detection device comprising an inference unit that uses an inference model that has been trained by machine learning to infer whether or not the inspection image shows oil leaking from an equipment.

8. The oil leakage detection device according to claim 7, wherein the learning data used in the machine learning includes an oil leakage image generated by an oil leakage image generating device according to any one of claims 1 to 6.

Images