Oil leak detection system

Abstract

It automatically detects oil leaks in various types of machinery. [Solution] The control unit recognizes the model of the heavy machinery based on external input, etc. (S11). This allows the control unit to recognize areas prone to oil leaks. The control unit controls the excitation light source and cameras so that these areas are properly irradiated with excitation light and fluorescence can be easily captured (S12). The control unit causes each camera to take an image (S13), and images to be analyzed are obtained from each camera. The image analysis unit can recognize the areas recognized as prone to oil leaks in the image, and the region including these areas can be recognized as a region where oil leaks are particularly likely to be detected (S14). The image analysis unit determines whether or not a fluorescence pattern emitted by the fluorescent substance has been recognized within this region (S15). If this pattern is recognized (S15: Yes), it is recognized that an oil leak exists at this location (S16).

Description

Technical Field

[0001] The present invention relates to an oil leak detection system for detecting oil leaks in construction machinery and the like.

Background Art

[0002] When using a construction machine having a hydraulic drive part, it is required to quickly detect an oil leak from the viewpoints of safety or proper performance of work. For this reason, a technique for making it easy to recognize an oil leak is used.

[0003] For example, Patent Document 1 describes a technique of adding a fluorescent material that emits fluorescence (visible light) having a wavelength different from that of excitation light to oil used in equipment by irradiating the oil with excitation light (ultraviolet light). The fluorescent material is added to the oil so as not to have an adverse effect when this oil is used. Thereby, the oil leaked in this equipment emits fluorescence by irradiation with excitation light. The state of generation of this fluorescence can be recognized by visual observation or an image captured by a camera, and thereby, an oil leak can be easily detected.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, in a construction machine having a complex three-dimensional shape, oil leaks can occur from various locations for each type. For this reason, for example, various modes in which fluorescence from the leaked oil can be confirmed also become various, and it has not been easy to actually detect the presence or absence of an oil leak. For this reason, finally, the presence or absence of an oil leak was determined by human eyes. For this reason, a technique for automatically detecting oil leaks in various machines has been desired.

[0006] This invention has been made in view of these circumstances, and its purpose is to automatically detect oil leaks in various machines. [Means for solving the problem]

[0007] The present invention provides an oil leak detection system for detecting oil leaks in machinery that uses oil, wherein the oil contains a fluorescent material that emits fluorescence of a different wavelength from the excitation light when irradiated with excitation light, and comprises an excitation light source that irradiates various parts of the machinery with the excitation light, a camera that images the machinery irradiated with the excitation light and recognizes the fluorescence, a control unit that recognizes areas in the machinery where oil leaks are likely to occur according to the type of machinery and adjusts the position of the excitation light source and the direction of irradiation of the excitation light so that these areas are irradiated with the excitation light, and an image analysis unit that recognizes oil leaks by recognizing the fluorescence pattern in the image obtained by the camera. The system may also include a memory unit that stores areas where oil leaks are likely to occur, depending on the type of machine. The areas prone to oil leakage, depending on the type of machine, are determined in the initial state according to the components of the machine, and the image analysis unit may recognize these areas in the image using semantic segmentation technology. The areas prone to oil leakage, depending on the type of machine, may be updated by machine learning using the fluorescence patterns obtained in the past as training data. The image analysis unit may recognize the fluorescence pattern by evaluating the difference between the image obtained by the camera and an image estimated to correspond to the image in the case where there is no oil leakage. In the aforementioned machine, the oil is used in multiple components, and different fluorescent substances may be added to the oil depending on the component. The aforementioned machine may be a hydraulic construction machine. [Effects of the Invention]

[0008] According to the present invention, oil leaks in various machines can be automatically detected. [Brief explanation of the drawing]

[0009] [Figure 1] This is a side view showing the structure of an example of heavy machinery that is subject to inspection by the oil leak detection system according to an embodiment of the present invention. [Figure 2] This is a simplified top view showing how the oil leak detection system according to an embodiment of the present invention is used. [Figure 3] This is a block diagram showing the configuration of an oil leak detection system according to an embodiment of the present invention. [Figure 4] This flowchart shows an example of operation in an oil leak detection system according to an embodiment of the present invention. [Modes for carrying out the invention]

[0010] Next, embodiments for carrying out the present invention will be specifically described with reference to the drawings.

[0011] Figure 1 shows a side view of a hydraulic excavator 100, which is an example of a construction machine (heavy equipment) to be inspected in the oil leak detection system of this embodiment. The structure of this hydraulic excavator 100 is the same as that described in, for example, Japanese Patent Application Publication No. 2017-172174. In this hydraulic excavator 100, a slewing device (main body) 102 that can rotate horizontally relative to the crawler unit 101 is fixed on the crawler unit 101 for traveling on the ground. A cab 103 with an operator inside is fixed to the slewing device 102. An arm-shaped boom 104 is mounted on the slewing device 102 so as to be rotatable around a pivot point S1 at its base, and an arm 105 is mounted on the tip side of the boom 104 so as to be rotatable around a pivot point S2 at its base. A bucket 106 is mounted on the tip side of the arm 105 so as to be rotatable around a pivot point S3.

[0012] The angles of the boom 104 relative to the slewing device 102, the angle of the arm 105 relative to the boom 104, and the angle of the bucket 106 relative to the arm 105 are controlled by the extension and retraction movements of the hydraulic cylinders 107, 108, and 109, respectively. In this process, each hydraulic jug is also supported at both ends by pivot points. By controlling the operation of the hydraulic cylinders 107, 108, and 109, the bucket 106, located at the very front, can be made to perform the desired movement.

[0013] Inside the hydraulic cylinders 107, 108, and 109, there is operating oil, which controls the movement of the piston inside the hydraulic cylinder, and thereby controls the movement (protrusion amount) of the rod fixed to the piston, thereby controlling the extension and retraction operation. Therefore, if oil leakage occurs in the hydraulic cylinders 107, 108, and 109, the above operation will be impaired. For this reason, in this case, it is required that the presence or absence of oil leakage can be detected individually for each of the hydraulic cylinders 107, 108, and 109. In addition, in each hydraulic cylinder, grease with a composition similar to that of the driving oil is applied to the sliding part of the rod. Also, grease supplied to the movable parts of pin joints and turntable bearings may leak and adhere to the surrounding area. In this case, it is necessary to distinguish between this grease and the leaked driving oil.

[0014] The oil leak detection system according to an embodiment of the present invention is particularly used for machines having such a complex three-dimensional structure. Therefore, first, a fluorescent material that emits fluorescence upon irradiation with excitation light is mixed into the oil to be detected in the heavy machinery to be inspected, similar to the technology described in Patent Document 1. Thus, by irradiating the heavy machinery using this oil with this excitation light and recognizing the corresponding fluorescence pattern, the oil leak can be detected. In this case, if the fluorescent material is not added to the grease, which is different from the oil to be detected, the leaked oil and grease can be easily distinguished. In other words, this method allows for easy identification of the oil and grease to be detected. Furthermore, not only grease, but also water (rainwater), which is not easily distinguishable from oil by visual inspection, can be easily identified as oil.

[0015] This oil leak detection system 1 is installed in the area housing the heavy machinery 200 to be inspected (here, it is assumed to be the hydraulic excavator 100). Figure 2 is a plan view showing the configuration when the heavy machinery 200 is inspected within this oil leak detection system 1. In this figure, the configuration of the hydraulic excavator 100 from Figure 1 is schematically shown from above. Figure 3 is a block diagram showing the configuration of this oil leak detection system 1. In Figure 2, the light source drive unit 12 and the camera drive unit 22 are fixed to the excitation light source 11 and the camera 21, respectively, and the descriptions of the light source drive unit 12 and the camera drive unit 22 are omitted in Figure 2.

[0016] As shown in Figure 1, the heavy machinery 200 moves from right to left in the figure, enters the inspection area X, which is a planar area, and stops at a predetermined position. In the inspection area X, the multiple excitation light sources 11 that emit the excitation light can irradiate the heavy machinery 200 with excitation light from multiple locations. In this case, as shown in Figure 3, each excitation light source 11 is fixed to a light source drive unit 12, and its horizontal and vertical position and the direction of irradiation of the excitation light are controlled by the light source drive unit 12. This allows each part of the heavy machinery 200 to be irradiated with excitation light from various directions. The number of excitation light sources 11 and light source drive units 12 is set appropriately so that each part of the heavy machinery 200 can be irradiated with excitation light from various directions.

[0017] Furthermore, multiple cameras 21 and a camera drive unit 22 are provided to control the position and imaging range (field of view) of the cameras 21, similar to the light source drive unit 12, so that each part of the heavy machinery 200 can be imaged. This allows each part of the heavy machinery 200 to be imaged from various directions. The number of cameras 21 and camera drive units 22 are set appropriately so that each part of the heavy machinery 200 can be imaged from various directions. In addition, each part of the heavy machinery 200 is imaged by the cameras 21, and optical filters are appropriately provided to particularly focus on detecting the fluorescence emitted by the fluorescent material. In Figure 2, the cameras 21 are provided adjacent to the excitation light source 11, but the positional relationship between them is arbitrary as long as fluorescence emitted by the excitation light emitted from the excitation light source 11 can be imaged.

[0018] In order to cause the above operation to be performed, a control device 30 is provided. As shown in FIG. 3, the control device 30 includes a control unit 31 that performs overall control, an image analysis unit 32 that analyzes an image (video) obtained from each camera 21 to recognize the presence or absence of an oil leak by image analysis, a display unit 33 that includes a display for displaying this image and the like, and a storage unit 34 that is a non-volatile memory or a hard disk for storing data used for control by the control unit 31 and analysis performed by the image analysis unit 32. In FIG. 2, the control device 30 is provided outside the inspection area X, but its installation position is arbitrary as long as the same operation can be performed. At this time, it is preferable that the display on the display unit 33 can be visually observed from both inside and outside the area X.

[0019] Also, in the inspection area X, an alarm unit 40 that issues an alarm by a sound signal (alarm sound) or a light signal when an oil leak is recognized is provided. This operation is controlled by the control unit 31. Thereby, the operator of the heavy machine 200 and the like can recognize that an oil leak has been recognized in the heavy machine 200.

[0020] For example, in the hydraulic excavator 100 of FIG. 1, oil leaks can occur in the hydraulic cylinders 107, 108, 109 and their surroundings. Similarly, depending on the type and structure of the heavy machine 200, the locations where oil leaks can occur are limited. For this reason, in the storage unit 34, the positions (horizontal position, vertical position) where oil leaks can occur (are likely to occur) for each type (model) of the heavy machine 200 are registered. However, for example, in the hydraulic excavator 200 of FIG. 1, since the positions of each hydraulic cylinder vary according to the operation, a certain range of positions that can be taken during this operation is registered.

[0021] Furthermore, for example, in Figure 1, if an oil leak occurs on the upper side of the hydraulic cylinder 107, it is difficult to irradiate this area with excitation light from above, or to recognize the oil leak from above, because the boom 104 is located above it. In this case, it is preferable to irradiate this area with excitation light from both the left and right sides, and to image this area from diagonally above. For this reason, the position and irradiation direction of each excitation light source 11, the position and field of view of the camera 21 are variable, allowing excitation light to be irradiated and images to be taken from various directions. This makes it possible to detect oil leaks more reliably.

[0022] The operation of the control device 30 will now be explained. Here, the control unit 31 controls the position and irradiation direction of the excitation light source 11, and also causes the image analysis unit 32 to perform an analysis of oil leak locations using AI image analysis technology. Figure 4 is a flowchart of this operation in the control device 30. First, the control unit 31 recognizes the model of the heavy machinery 200 based on external input, etc. (S11). Here, the model may be automatically recognized by pattern recognition of the overall shape by the camera 21 without external input. As a result, the control unit 31 can recognize locations in the heavy machinery 200 where oil leaks are likely to occur (hydraulic cylinders 107, 108, and 109 in Figure 1) from the data stored in the memory unit 34. As a result, the control unit 31 controls the excitation light source 11 (light source drive unit 12) and the camera 21 (camera drive unit 22) so that these are appropriately irradiated with excitation light and fluorescence can be easily captured (S12). At this time, general image quality adjustments for the camera 21 can be performed for each camera 21.

[0023] As a result, the control unit 31 causes each camera 21 to take an image (S13). This allows the camera 21 to obtain an image to be analyzed. Subsequent processing is performed for each image obtained in this way. Here, the image analysis unit 32 can recognize areas that are prone to oil leaks (hydraulic cylinders 107, 108, and 109 in Figure 1) in the image using semantic segmentation technology, and recognize the region including these areas as a region where oil leaks are particularly likely to be detected (S14). For this reason, the image analysis unit 32 determines whether or not a fluorescence pattern emitted by the fluorescent substance (for example, a two-dimensional pattern with a certain extent in a two-dimensional image) has been recognized in this region (S15). Here, the recognition of fluorescence in a pixel in the image means that the light intensity recognized at this wavelength in that pixel exceeds a certain threshold. This threshold is set appropriately based on measured values.

[0024] If the control unit 31 recognizes this pattern (S15:Yes), it recognizes that there is an oil leak at this location (S16). Therefore, the control unit 31 controls the alarm unit 40 and outputs a message indicating that an oil leak has been detected (S17). At this time, it is preferable to display the location where the oil leak was detected (the hydraulic cylinders 107, 108, and 109 mentioned above) on the display unit 33. After that, the pattern shape recognized here and the imaging information of the image from which this pattern was obtained (position of camera 21, field of view direction) are registered in the storage unit 34 (S18). The above analysis is repeated for all images obtained by each camera 21 and then terminated (S19:Yes).

[0025] This concludes the oil leak inspection of the heavy machinery 200. If an oil leak (fluorescent pattern) is detected, its shape and other data are stored in the memory unit 34, but this data is managed separately for each model of heavy machinery 200. Therefore, after acquiring many of these patterns, the control unit 31 can use this as training data and update the data of locations stored in the memory unit 34 as places where oil leaks are likely to occur using machine learning. This makes oil leak detection more efficient in subsequent inspections. The values ​​of the various parameters used in this machine learning are stored in the memory unit 34.

[0026] In the flowchart of Figure 4, it is assumed that data regarding locations prone to oil leaks is pre-stored in the memory unit 34 (S15), and the presence or absence of a fluorescence pattern is recognized based on this premise (S16). However, when performing the machine learning operation as described above, it is possible to recognize oil leaks in the same way by acquiring a large number of images with various changes, for example, in the direction of excitation light irradiation and the imaging direction of the camera 21, without initially recognizing such specific areas (S14), and recognizing fluorescence patterns in all ranges of all images. However, in this case, image acquisition (S13) and image analysis (S15) take time.

[0027] However, once numerous inspections are conducted and a large amount of fluorescence pattern data is accumulated, machine learning can be used to store in the storage unit 34 the areas prone to oil leaks (especially the areas that should be targeted for image analysis) for each type of heavy machinery. Therefore, it is not necessary to pre-store data on areas prone to oil leaks in the storage unit 34 at the beginning. Even in this case, after many inspections, the data on areas prone to oil leaks will be updated, allowing for more efficient inspections thereafter.

[0028] Therefore, the oil leak detection system 1 described above can efficiently detect oil leaks in many types of heavy machinery.

[0029] Furthermore, as described above, different fluorescent substances may be added to the oil for each of the hydraulic cylinders 107, 108, and 109. In this case, fluorescent substances with the same excitation light wavelength but different fluorescence wavelengths, or fluorescent substances with nearly the same fluorescence wavelength but different excitation light wavelengths can be used. In this case, oil leaks in each of the hydraulic cylinders 107, 108, and 109 can be more easily detected. In this case, the excitation light source 11 and camera 21 may be set up to correspond to these wavelengths, and data can be obtained in the same manner as described above by installing switchable optical filters on the excitation light source 11 and camera 21 and switching them as appropriate.

[0030] Generally, heavy machinery, such as the hydraulic excavator 100 shown above, has a complex three-dimensional shape, and oil leaks often occur in various locations. In such cases, as described above, it is particularly effective to recognize areas where oil leaks may occur and control the position and direction of the excitation light source accordingly. Furthermore, as described above, a configuration that updates these areas where oil leaks may occur using machine learning is particularly effective for efficiently detecting oil leaks in various types of machinery. This configuration is especially effective when oil leaks can occur in areas other than those that can be clearly separated and recognized as components, such as the hydraulic cylinder in the hydraulic excavator 100 shown in Figure 1.

[0031] In the example above, the control unit 31 used semantic segmentation technology to recognize areas prone to oil leaks (hydraulic cylinders 107, 108, 109, etc.) in the image (S14), and checked for the presence or absence of a two-dimensional pattern of fluorescent material within those areas (S15). In addition to this, various AI technologies can be used to similarly recognize oil leaks. One such technology is "Patch Core," an anomaly detection method in images, which was presented at CVCR (Computer Vision and Pattern Recognition Conference) 2022, "Towards Total Recall in Industrial Anomaly Detection," by Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Scholkopf, Thomas Brox, and Peter Gehler (June 2022).

[0032] In "Patch Core," only images without abnormalities (no oil leaks) are used as training data. Feature quantities of localized regions (patches) within these images are optimized and extracted. The difference between these features and the resulting image is evaluated to recognize abnormalities. Each part of heavy machinery with a complex structure becomes such a patch, and the aforementioned two-dimensional fluorescent patterns are easily recognized as such differences. Therefore, "Patch Core" is particularly effective for recognizing oil leaks. In this case, oil leak recognition using "Patch Core" replaces the recognition of areas prone to oil leaks (S14) and the recognition of the presence or absence of two-dimensional fluorescent patterns within those areas (S15).

[0033] In other words, in this case, the fluorescence pattern (oil leak pattern) is recognized by evaluating it as the difference between the image obtained by the camera and the image that is presumed to correspond to the image in the case where there is no oil leak. This recognition may be performed for each part individually.

[0034] The present invention has been described above based on the embodiments. These embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible in the combination of these components, and that such modifications also fall within the scope of the present invention. [Explanation of symbols]

[0035] 1. Oil leak detection system 11 Excitation light source 12 Light source drive unit 21 Cameras 22 Camera drive unit 30 Control device 31 Control Unit 32 Image Analysis Department 33 Display section 34 Storage section 40. Reporting Department 100 Hydraulic Excavator (Heavy Equipment: Construction Machinery) 101 Crawler Unit 102 Swivel device (main body) 103 Cab 104 Boom 105 Arm 106 buckets 107-109 Hydraulic Cylinder 200 heavy machinery S1~S3 fulcrum X Inspection Area

Claims

1. An oil leak detection system for detecting oil leaks in machinery that uses oil, The oil is mixed with a fluorescent material that emits fluorescence at a wavelength different from the excitation light when irradiated with excitation light. An excitation light source that irradiates each part of the machine with the excitation light, A camera that images the machine irradiated with the excitation light and recognizes the fluorescence, A control unit that recognizes areas in the machine where oil leakage is likely to occur according to the type of machine, and adjusts the position of the excitation light source and the direction of irradiation of the excitation light so that such areas are irradiated with the excitation light, An image analysis unit that recognizes oil leakage by recognizing the fluorescence pattern in the image obtained by the camera, An oil leak detection system characterized by comprising the following:

2. The oil leak detection system according to claim 1, further comprising a storage unit that stores areas where oil leaks are likely to occur according to the type of machine.

3. The areas where oil leakage is likely to occur, depending on the type of machine, are determined in the initial state according to the components of the machine. The oil leak detection system according to claim 2, characterized in that the image analysis unit recognizes the region in the image using semantic segmentation technology.

4. The oil leak detection system according to claim 1 or 2, characterized in that the areas where oil leaks are likely to occur, depending on the type of machine, are updated by machine learning using the fluorescence patterns obtained in the past as training data.

5. The oil leak detection system according to claim 1, characterized in that the image analysis unit recognizes the fluorescence pattern by evaluating the difference between the image obtained by the camera and an image estimated to correspond to the image in the case where there is no oil leak.

6. The oil leak detection system according to claim 1 or 2, characterized in that the oil is used in a plurality of components, and different fluorescent substances are added to the oil depending on the component.

7. The oil leak detection system according to claim 1 or 2, characterized in that the machine is a hydraulic construction machine.

Images