An oil leakage detection method and system for equipment based on infrared thermal imaging

Abstract

The present invention discloses a method and system for detecting oil leakage of equipment based on infrared thermal imaging. In the present invention, an infrared thermal imaging camera is installed above the front of the equipment where oil leakage may occur, and an ROI oil leakage detection area is preset in the monitoring image of the camera; the infrared thermal imaging signal data of the camera is obtained in real time, and then the ROI area is subjected to oil leakage detection to find suspicious oil stain connected areas; finally, the detected suspicious oil stain connected areas are analyzed, and whether to output an oil leakage detection alarm information box on the monitoring image is determined according to the analysis result. The present invention combines oil leakage detection with infrared thermal imaging to realize automatic oil leakage monitoring. Compared with the existing manual inspection method, there is no need to go to the site for close detection, which makes the equipment oil leakage detection more efficient and convenient.

Description

Technical Field

[0001] The present invention belongs to the technical field of equipment oil leakage detection, and particularly relates to a method and system for detecting equipment oil leakage based on infrared thermal imaging. Background Art

[0002] With the continuous acceleration of the industrialization process and the rapid increase in population, all walks of life have become increasingly dependent on energy, especially oil-based energy. This has increased the workload in all aspects of oil-based energy transportation, storage, distribution, etc. Inevitably, the resulting negative problems have also increased. Among them, equipment oil leakage is a frequent and extremely negative problem. The equipment oil leakage problem not only wastes a large amount of oil but also pollutes the environment. In severe cases, it may even cause safety accidents and affect production. Therefore, it is very necessary to detect equipment oil leakage in advance to prevent problems before they occur.

[0003] In traditional technologies, the detection of equipment oil leakage mainly adopts manual inspection or the method of installing visible light cameras at positions where oil leakage may occur. If the manual inspection method is adopted, it means that manpower needs to be arranged to continuously check the equipment and its surrounding positions where oil leakage may occur for 24 hours. When the inspection personnel find that there is oil leakage in the equipment, they need to notify relevant personnel for repair. Obviously, this detection method is difficult to detect abnormal conditions in a timely manner, has a certain lag, and brings a series of safety hazards. If the method of installing visible light cameras is adopted, although this method has better real-time performance than manual inspection, it is only applicable to sunny days. Because once it is at night or in rainy weather, it is very difficult to distinguish the difference between oil and water by color. Therefore, this method is also not advisable. Currently, some latest technologies have also been used for equipment oil leakage detection. For example, deep learning technology is used to perform target recognition training on picture samples of oil leakage blocks of various shapes to generate an oil leakage block recognition model, and then this model is used to perform real-time detection on the video images obtained by the visible light camera. Although this method has good timeliness, due to the fact that the shapes of oil leakage blocks are irregular, it is impossible to include all shape samples of oil leakage blocks in the training data, which makes the robustness of the model not high, and there is a high probability of missed detection or false alarm.

[0004] Therefore, in view of the above problems, there is an urgent need to provide a method and system for detecting equipment oil leakage with high robustness and high accuracy. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems existing in the prior art and provide a method and system for detecting equipment oil leakage based on infrared thermal imaging.

[0006] The specific technical solutions adopted by the present invention are as follows:

[0007] In a first aspect, the present invention provides a method for detecting oil leakage of a device based on infrared thermal imaging, which includes the following steps:

[0008] S1. Real-time obtain infrared monitoring image frames of a target monitoring area through an infrared thermal imaging camera with a fixed viewing angle, and extract a first ROI area image covering the range where the oil stain block appears from the infrared monitoring image frames according to the preset ROI area coordinate frame;

[0009] S2. Conduct gray value statistics on the first ROI area image obtained in S1 to obtain a gray histogram of the first ROI area image, and then perform a point-throwing process on the first ROI area image to obtain a second ROI area image; the point-throwing process is to remove the extreme pixels with pixel values at both ends of the value range of the gray histogram in the first ROI area image;

[0010] S3. Cluster the pixel values of the second ROI area image obtained in S2. If all pixel values are clustered into one class, it is regarded that there is no oil leakage at the moment when the second ROI area image is obtained, and continue to execute S1-S3 for the next frame of infrared monitoring image frame; if all pixel values are clustered into multiple classes, it is regarded that there may be oil leakage in the current target monitoring area. Use the average value of all cluster center pixel values as the first threshold to binarize the second ROI area image. If the pixel value is less than or equal to the first threshold, set it to 1, otherwise set it to 0, so as to establish a binary image containing suspected oil stain blocks; then remove isolated points from the binary image and screen based on the minimum area, and update all the connected regions obtained by screening to the target tracking list; the update process is as follows:

[0011] Traverse each connected region obtained by screening, and perform target matching for the currently traversed connected region in the target tracking list. If the same tracking target cannot be matched, add the currently traversed connected region as a new tracking target in the target tracking list, and initialize the target appearance times of this new tracking target to 1 and the consecutive disappearance times to 0; if the same tracking target is matched, add 1 to the appearance times of this tracking target in the target tracking list and set the consecutive disappearance times of this tracking target to 0; after traversing all the connected regions obtained by screening, judge whether there are original tracking targets in the target tracking list whose target appearance times have not been incremented. If so, increment their consecutive disappearance times by 1;

[0012] S4. Continuously iterate through S1 to S3. After each iteration and updating the target tracking list, traverse all the tracked targets in the target tracking list. If the consecutive disappearance times of a tracked target exceed the second threshold, delete the tracked target from the target tracking list. If the appearance times of a tracked target exceed the third threshold and the average pixel fluctuation within the connected region of the tracked target is less than the fourth threshold, then consider the tracked target as an oil stain block caused by oil leakage and trigger an oil leakage alarm for the device.

[0013] As a preference of the above first aspect, the infrared thermal imaging camera supports RTSP and ONVIF protocols. After inputting the device IP, port number, username, and password in the corresponding SDK, the acquired infrared thermal imaging signal data is displayed.

[0014] As a preference of the above first aspect, during the throwing point processing, the 5% pixels with the largest gray values and the 5% pixels with the smallest gray values within the value range of the gray histogram need to be removed.

[0015] As a preference of the above first aspect, the clustering uses K - means clustering, and the method is as follows:

[0016] S31. Randomly select any two pixel points in the second ROI region image as the initial clustering centers;

[0017] S32. Calculate the distances between each of the remaining pixel points in the second ROI region image and the two clustering centers, where the distance is the absolute value of the difference in gray values between the pixel point and the clustering center;

[0018] S33. Divide each of the remaining pixel points in the second ROI region image into the cluster with the closest distance value, then calculate the average gray value of all the pixel points in each cluster and use the average value as the new clustering center;

[0019] S34. Continuously iterate through S32 and S33 until the clustering centers converge to obtain the final clustering result.

[0020] As a preference of the above first aspect, the method for removing outliers is as follows:

[0021] Traverse each pixel point in the binary image. If the sum of the pixel values of the 9 pixels within the 3×3 range centered on this pixel point is less than 9, then determine this point as an outlier and set the pixel value of the outlier in the binary image to 0.

[0022] As a preference of the above first aspect, the method for screening based on the minimum area is as follows:

[0023] Screen the connected regions in the binary image after removing isolated points according to the preset minimum area threshold of the oil stain block region, extract the connected regions with an area larger than the minimum area threshold of the oil stain block region, and filter out the connected regions with an area not larger than the minimum area threshold of the oil stain block region.

[0024] As a preference of the above first aspect, the minimum area threshold of the oil stain block region is 80 to 120 pixels.

[0025] As a preference of the above first aspect, when performing target matching for the currently traversed connected region in the target tracking list, the matching rule is as follows:

[0026] Calculate the centroid of the currently traversed connected region, and determine whether the centroid falls within the latest outer bounding rectangle of a tracking target in the target tracking list and the area change rate of the two connected regions is less than 50%. If so, it is considered that the currently traversed connected region and the corresponding tracking target belong to the same tracking target. If not, it is considered that the currently traversed connected region is a new tracking target.

[0027] As a preference of the above first aspect, when triggering the device oil leakage alarm, display the outer bounding rectangle of the oil stain block on the infrared monitoring image frame, and send an alarm signal through one or more of sound, light, electricity, image, and text.

[0028] In a second aspect, the present invention provides a device oil leakage detection system based on infrared thermal imaging, which includes:

[0029] A real-time image acquisition module, configured to continuously acquire infrared monitoring image frames of a target monitoring area through an infrared thermal imaging camera with a fixed viewing angle, and extract a first ROI area image covering the range where the oil stain block appears from the infrared monitoring image frames according to the preset ROI area coordinate frame;

[0030] A preprocessing module, configured to perform gray value statistics on the first ROI area image acquired by the real-time image acquisition module to obtain a gray histogram of the first ROI area image, and then perform dot-throwing processing on the first ROI area image to obtain a second ROI area image; the dot-throwing processing is to remove the extreme value pixels with pixel values at both ends of the value range of the gray histogram in the first ROI area image;

[0031] The target tracking module is used to cluster the pixel values of the second ROI area image obtained in the preprocessing module. If all pixel values are clustered into one class, it is regarded that there is no oil leakage at the moment when the second ROI area image is obtained, and S1 - S3 are continued to be executed for the next frame of infrared monitoring image. If all pixel values are clustered into multiple classes, it is regarded that there may be oil leakage in the current target monitoring area. The average value of all cluster center pixel values is used as the first threshold to binarize the second ROI area image. If the pixel value is less than or equal to the first threshold, it is set to 1, otherwise it is set to 0, so as to establish a binary image containing suspected oil stain blocks. After removing isolated points from the binary image, screening is performed based on the minimum area, and all connected regions obtained by screening are updated to the target tracking list. The update process is as follows:

[0032] Traverse each connected region obtained by screening, and perform target matching for the currently traversed connected region in the target tracking list. If the same tracking target cannot be matched, add the currently traversed connected region as a new tracking target in the target tracking list, and initialize the number of times the new tracking target appears to 1 and the number of consecutive disappearances to 0. If the same tracking target is matched, add 1 to the number of times the tracking target appears in the target tracking list and set the number of consecutive disappearances of this tracking target to 0. After traversing all the connected regions obtained by screening, judge whether there is an original tracking target in the target tracking list whose number of times of appearance has not been incremented. If it exists, increment its number of consecutive disappearances by 1.

[0033] The device oil leakage alarm module is used to continuously iterate and execute the real - time image acquisition module, the preprocessing module, and the target tracking module. After updating the target tracking list after each iteration, traverse all the tracking targets in the target tracking list. If the number of consecutive disappearances of a tracking target exceeds the second threshold, delete the tracking target from the target tracking list. If the number of times a tracking target appears exceeds the third threshold and the average pixel fluctuation within the connected region of this tracking target is less than the fourth threshold, it is considered that this tracking target is an oil stain block caused by oil leakage, and the device oil leakage alarm is triggered.

[0034] The present invention has the following beneficial effects compared with the prior art:

[0035] 1) The present invention combines oil leakage detection with infrared thermal imaging, completely getting rid of the influence of weather on device oil leakage detection. Compared with the existing manual inspection method, there is no need to go to the site for close - range detection, which makes device oil leakage detection more efficient and convenient.

[0036] 2) The present invention uses K-means clustering for equipment oil leakage detection. After preprocessing, the infrared image is input into the K-means model, which can automatically identify the equipment oil leakage defect information in the infrared image. This process does not require human participation, thus reducing human errors, improving the detection accuracy and work efficiency, and expanding the application field of infrared thermal imaging.

[0037] 3) The present invention not only realizes the real-time intelligent detection of equipment oil leakage, but also improves the detection efficiency and accuracy. In addition, when it is determined that there is an oil leakage in the equipment, an alarm message can be output in a timely manner, greatly improving the safety of production. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of the equipment oil leakage detection method based on infrared thermal imaging provided by an embodiment of the present invention.

[0039] Figure 2 It is an image of the ROI area to be detected provided by an embodiment of the present invention;

[0040] Figure 3 It is a binary image of the suspected oil stain in the ROI area after detection provided by an embodiment of the present invention;

[0041] Figure 4 It is a binary image of the suspected oil stain in the ROI area after removing scatter points provided by an embodiment of the present invention.

[0042] Figure 5 It is the final output alarm block diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given with reference to the accompanying drawings. Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below. The technical features in the various embodiments of the present invention can be combined correspondingly without conflict.

[0044] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features.

[0045] The inventive concept of the present invention is: install an infrared thermal imaging camera above the front of the equipment where oil leakage may occur, and preset an ROI oil leakage detection area in the monitoring image of the camera; then obtain the infrared thermal imaging signal data of the camera in real time through the network or RTSP protocol, and perform oil leakage detection on the ROI area to find the suspicious oil stain connection area; finally, analyze the detected suspicious oil stain connection area and decide whether to output an oil leakage detection alarm information box on the monitoring image according to the analysis result.

[0046] In the above solution, the present invention combines oil leakage detection with infrared thermal imaging, judges whether there is oil leakage in the target area by analyzing the infrared radiation data in different bands in the target area, uses the video image obtained by the infrared thermal imaging camera for real-time intelligent detection, and can output alarm information in time when it is judged that there is oil leakage in the equipment. This not only improves the detection efficiency and accuracy, but also greatly improves the production safety. The following will elaborate on the specific implementation of the present invention in detail.

[0047] As Figure 1 shown, in a preferred embodiment of the present invention, a method for detecting oil leakage of equipment based on infrared thermal imaging is provided, which includes the following steps:

[0048] S1. Obtain the infrared monitoring image frame of the target monitoring area in real time through an infrared thermal imaging camera with a fixed viewing angle, and extract the first ROI area image covering the range where the oil stain block appears from the infrared monitoring image frame according to the preset ROI area coordinate frame.

[0049] As a preferred implementation manner of the embodiment of the present invention, when extracting the first ROI area image, a ROI area where oil leakage may occur can be drawn in the real-time video image according to the coordinate frame given by the upper-layer instruction first, and the gray value of this area and its corresponding coordinates are cached, and in the original video image, all parts except the ROI area are filled with 0, as Figure 2 shown.

[0050] It should be noted that the target monitoring area of the infrared camera should cover the entire equipment and the area around the equipment where oil leakage may occur, and its viewing angle needs to be fixed. The specific form of the infrared camera is not limited, as long as it can obtain accurate infrared images in real time. As a preferred implementation manner of the embodiment of the present invention, the infrared camera preferably supports RTSP and ONVIF protocols, and the obtained infrared thermal imaging signal data can be displayed after inputting the device IP, port number, username and password in the corresponding SDK.

[0051] S2. Perform grayscale value statistics on the obtained first ROI region image in S1 to obtain the grayscale histogram of the first ROI region image, and then perform outlier removal processing on the first ROI region image to obtain the second ROI region image; the outlier removal processing is to remove the extreme value pixels at both ends of the value range of the grayscale histogram of the first ROI region image.

[0052] As a preferred implementation manner of the embodiment of the present invention, when obtaining the grayscale histogram, the grayscale distribution histogram of the ROI region image can be statistically calculated according to the real-time infrared thermal imaging signal data in the target monitoring area. After the statistics are completed, the grayscale histogram of the ROI region image is saved to an array for subsequent calling during the oil leakage detection process.

[0053] As a preferred implementation manner of the embodiment of the present invention, when performing outlier removal processing, the pixels in this region can be first subjected to outlier removal according to the grayscale histogram of the cached ROI region image to exclude the interference of some useless pixels or extreme value pixels and improve the detection efficiency. The outlier removal ratio can be set to 10%, that is, it is necessary to remove 5% of the pixels with the largest grayscale value and 5% of the pixels with the smallest grayscale value within the value range of the grayscale histogram. For example, if the value range of the grayscale histogram is [x min ,x max , then it is necessary to remove the pixel points within the two regions where the pixel grayscale values fall within [x min ,x min +0.05(x max -x min )] and [x max ,x max -0.05(x max -x min )].

[0054] S3. Cluster the pixel values of the second ROI region image obtained in S2. If all pixel values are clustered into one class, it is regarded that there is no oil leakage situation at the moment when the second ROI region image is obtained, and S1 to S3 are continued to be executed for the next frame of infrared monitoring image; if all pixel values are clustered into multiple classes, it is regarded that there may be an oil leakage situation in the current target monitoring area. The average value of all cluster center pixel values is used as the first threshold to binarize the second ROI region image. If the pixel value is less than or equal to the first threshold, it is set to 1, otherwise it is set to 0, thereby establishing a binary image containing suspected oil stain blocks; then, after removing the isolated points from the binary image, screening is performed based on the minimum area, and all connected regions obtained by the screening are updated to the target tracking list.

[0055] As a preferred implementation manner of the embodiment of the present invention, the clustering method can adopt K-means clustering, and its approach is as follows:

[0056] S31. Randomly select any two pixel points in the second ROI region image as the initial clustering centers;

[0057] S32. Calculate the distances between each of the remaining pixel points in the second ROI region image and the two clustering centers, where the distance is the absolute value of the difference in gray values between the pixel point and the clustering center;

[0058] S33. Divide each of the remaining pixel points in the second ROI region image into the cluster with the closest distance value, then calculate the average gray value of all pixel points in each cluster, and use the average value as the new clustering center;

[0059] S34. Continuously iterate S32 and S33 until the clustering centers converge and no longer change, then the final clustering result can be obtained.

[0060] In addition, the purpose of removing isolated points is to delete the discrete points that obviously do not belong to the oil stain blocks in the binary image, reduce the computational amount of subsequent oil leakage determination, and improve the computational efficiency. As a preferred implementation manner of the embodiment of the present invention, the method for removing isolated points is as follows: Traverse each pixel point in the binary image. If the sum of the pixel values of 9 pixels within a 3×3 range centered on this pixel point is less than 9, then determine that this point is an isolated point and set the pixel value of the isolated point in the binary image to 0.

[0061] In addition, the purpose of screening based on the minimum area is to remove the connected regions whose areas are significantly smaller than the normal area size of the oil stain block region, reduce the computational amount of subsequent oil leakage determination, and improve the computational efficiency. As a preferred implementation manner of the embodiment of the present invention, the method for screening based on the minimum area is as follows:

[0062] Screen the connected regions in the binary image after removing isolated points according to a preset minimum area threshold of the oil stain block region, extract the connected regions with areas larger than the minimum area threshold of the oil stain block region, and filter out the connected regions with areas not larger than the minimum area threshold of the oil stain block region. Among them, the minimum area threshold of the oil stain block region needs to be determined according to the actual size of the detection region and the common area size of the oil stain block in this region. In one embodiment, the minimum area threshold of the oil stain block region can be set to 80 - 120 pixels, preferably 100 pixels. That is, when the number of pixels in a certain connected region is less than 100, it is considered that the area of this connected region is small and it is impossible to belong to the oil stain block region.

[0063] It should be noted that the purpose of clustering in the present invention is to determine whether there is an oil leakage situation. The principle is that the appearance of oil stain blocks will cause the gray values of pixels in the oil stain block area to be significantly different from those in other normal areas. Therefore, based on this principle of gray value change, it is possible to determine whether there is an oil leakage situation by clustering the pixel gray values. If the pixel values in the ROI area are clustered into one category, it means that there is no oil leakage situation in this area. At this time, the above K-means clustering operation is continued for the next frame of infrared thermal imaging image. If the pixel values in the ROI area are clustered into multiple categories (generally, the number of clusters can be set to two), it means that there may be an oil leakage situation in this area. At this time, the pixel values of the cluster centers are saved. Next, the entire ROI area is traversed, and the average value of all pixel values of the cluster centers is used as a threshold to convert this area into a binary image, that is, when the pixel value in the ROI area is less than or equal to the threshold, the pixel value is set to 1, otherwise it is 0. In one example, through this process, a binary image containing the oil leakage information of the ROI area is obtained, as Figure 3 shown. And as can be seen from Figure 3 , there are many scattered points in the binary image of the ROI area image extracted, which will affect the subsequent detection accuracy. Therefore, it is necessary to remove the scattered points in this binary image next. If the matrix A = {a ij} m×n represents this binary image, then for any point a ij in this matrix, if it satisfies , it means that this point is an isolated point, that is, a scattered point. At this time, let a ij = 0 to remove this scattered point. The binary image after removing the scattered points is as Figure 4 shown.

[0064] In the present invention, the core purpose of updating all the connected regions obtained by screening to the target tracking list is to track the appearance and disappearance of the connected regions of each suspected oil stain block in different image frames, so as to facilitate the judgment of whether there is really an oil leakage. The specific update process of the target tracking list in the present invention is as follows:

[0065] Traverse each connected region obtained by screening, and perform target matching for the currently traversed connected region in the target tracking list. If the same tracking target cannot be matched, add the currently traversed connected region as a new tracking target in the target tracking list, and initialize the number of times the new tracking target appears to 1 and the number of consecutive disappearances to 0; if the same tracking target is matched, add 1 to the number of times this tracking target appears in the target tracking list, and set the number of consecutive disappearances of this tracking target to 0; after traversing all the connected regions obtained by screening, judge whether there is an original tracking target in the target tracking list whose number of times of appearance has not been incremented. If there is, increment its number of consecutive disappearances by 1.

[0066] It should be noted that, for the convenience of the computer to automatically execute the above update process of the target tracking list, for the connected regions of the suspicious oil stain blocks screened out in the binary image after removing the scattered points, the connected regions can be counted and recorded by means of line scanning, and each connected region can be marked with a unique and non-repeating number. During the line scanning process, if there are obvious irregularities in the same connected region, it may be recorded multiple times. Therefore, after the scanning is completed, the areas of all the recorded connected regions can be compared. If two connected regions with exactly the same area are found, they are regarded as the same connected region, and the corresponding numbers need to be merged. Moreover, since the number of connected regions in a general image is small, generally not exceeding 254, considering the algorithm efficiency, when the number of marked connected regions reaches 254, the number of the remaining uncounted connected regions can be no longer counted, and the pixel values of the remaining uncounted ones can be assigned 0 to avoid excessive computational load of the algorithm in some abnormal situations and affecting the real-time performance of detection. In addition, for the convenience of subsequent target matching, the coordinates of the circumscribed rectangles of each connected region can be obtained according to the final numbers and saved for calling during the target matching process.

[0067] In the above update process of the target tracking list, the prerequisite is to determine whether there is a corresponding connected region in the target tracking list for the connected region found in the current real-time infrared monitoring image frame. The matching rule between the two can be designed according to the geometric transformation of the connected region of the oil stain block.

[0068] Therefore, in a preferred implementation manner of the embodiment of the present invention, for the convenience of description, when traversing each connected region obtained by screening, each connected region calculated currently during each traversal process is called the currently traversed connected region. The specific matching rule when performing target matching of the currently traversed connected region in the target tracking list is as follows:

[0069] Calculate the centroid of the currently traversed connected region, and determine whether the centroid falls within the latest circumscribed rectangle of a tracking target in the target tracking list and the area change rate of the two connected regions is less than 50%. If so, it is regarded that the currently traversed connected region and the corresponding tracking target belong to the same tracking target; if not, the currently traversed connected region is regarded as a new tracking target.

[0070] It should be noted that since each tracking target in the target tracking list actually corresponds to a connected region when it appears in each frame of the infrared monitoring image, a tracking target records the connected regions at different times. In this preferred method, when matching, it is necessary to call the most recently recorded connected region, that is, the outermost rectangle of the tracking target mentioned above is the outermost rectangle of the most recently recorded connected region of this tracking target. In addition, in the "area change rate of two connected regions", the two connected regions refer to the most recently recorded connected region of the tracking target and the currently traversed connected region, and its area change rate can be calculated by dividing the difference between the areas of the two by the maximum value of the areas of the two. The area of the connected region can be replaced by the number of pixels it contains.

[0071] It should be noted that after traversing all the connected regions obtained by screening, the purpose of the above-mentioned "judging whether there is an original tracking target in the target tracking list whose number of target appearances has not been incremented by 1, and if so, incrementing its consecutive disappearance count by 1" is to record the disappearance situation of the tracking target in each frame of the image. Because if a connected region caused by an oil stain block appears, then this oil stain block will surely appear continuously in different frames of the infrared monitoring image. However, if it is a black shadow region caused by other lighting reasons or sundries, it will not continuously exist in different frames of the infrared monitoring image. Therefore, for a tracking target, after obtaining each new frame of the infrared monitoring image, it is necessary to judge whether this tracking target appears in this frame. If it appears, increment the number of target appearances by 1. If it does not appear, increment the consecutive disappearance count by 1.

[0072] S4. Continuously iterate and execute S1 to S3. After iterating once and updating the target tracking list, traverse all the tracking targets in the target tracking list. If the consecutive disappearance count of a tracking target exceeds the second threshold, delete the tracking target from the target tracking list. If the number of appearances of a tracking target exceeds the third threshold and the average pixel fluctuation within the connected region of this tracking target is less than the fourth threshold, it is considered that this tracking target is an oil stain block caused by oil leakage, and the device oil leakage alarm is triggered.

[0073] It should be noted that the above-mentioned second threshold, third threshold, and fourth threshold can all be optimized and adjusted according to the actual situation, with the aim of achieving the best device oil leakage detection accuracy.

[0074] In a preferred implementation manner of the embodiment of the present invention, the second threshold can be set to 20% of the total number of discriminations. That is, if a certain tracking target disappears continuously for multiple frames and the number of disappearances is 20% of the total number of discriminations, then this tracking target is determined to have disappeared, and subsequent tracking of this tracking target will no longer be performed.

[0075] In a preferred implementation manner of the embodiment of the present invention, the third threshold is set to the minimum number of frames N for target alarm. If the total number of appearances of a certain tracking target from the first appearance to the current frame is greater than or equal to N, and the average pixel fluctuation within the connected region of the target is less than the average fluctuation threshold, then it is considered that the target meets the alarm condition. Generally speaking, the value of the above N can be adjusted according to actual needs. For example, N = 40 or N = 60 can be set, etc. In one example, the value of N can be set to 40.

[0076] In a preferred implementation manner of the embodiment of the present invention, the fourth threshold is an average pixel fluctuation threshold M, which needs to be optimally designed according to the change of the average pixel fluctuation within the connected region of the target in the actual scenario. In one example, the value is 60.

[0077] It should be noted that the average pixel fluctuation within the connected region of the tracking target refers to the fluctuation of the maximum pixel value in the connected region of the tracking target in all frames. This kind of fluctuation can be characterized by different calculation methods. In a preferred implementation manner of the embodiment of the present invention, assume that the number of appearances of a tracking target i is N i , then it is necessary to extract the maximum gray value x within the region for each of these N i connected regions respectively, and then sort these N i connected regions according to the recorded time, calculate the difference between the corresponding maximum gray values x of each pair of adjacent connected regions in the sequence, and take the average of the absolute values of these N i -1 differences, which is the average pixel fluctuation within the connected region of the tracking target.

[0078] In one example, after analysis, if it is confirmed that the current target meets the alarm condition, that is, there is an oil leakage situation, then an oil leakage alarm information box is output on the monitoring image, as Figure 5 shown.

[0079] In addition, in order to avoid a single form of alarm, a corresponding alarm module can be set at the SDK video display end, and the forms of alarm signals that can be emitted can be selected from one or more of sound, light, electricity, image, and text. These alarm signals can be sent to an external terminal, and then the external terminal conveys the alarm information to relevant staff. Among them, the external terminal includes an audible and visual alarm, a mobile phone, a cloud platform, a server, etc. The main function is to attract the attention of relevant personnel in the first time, so that relevant personnel can quickly respond to this potential safety hazard, thereby avoiding the occurrence of dangerous accidents.

[0080] In addition, based on the same inventive concept as the method for detecting oil leakage of equipment based on infrared thermal imaging provided in the above embodiment, in another preferred embodiment of the present invention, a system for detecting oil leakage of equipment based on infrared thermal imaging is provided, and the system includes the following functional modules:

[0081] A real-time image acquisition module, which is used to acquire infrared monitoring image frames of a target monitoring area in real time through an infrared thermal imaging camera with a fixed viewing angle, and extract a first ROI area image covering the range where the oil stain block appears from the infrared monitoring image frames according to the preset ROI area coordinate frame;

[0082] A preprocessing module, which is used to perform gray value statistics on the first ROI area image acquired in the real-time image acquisition module to obtain a gray histogram of the first ROI area image, and then perform outlier removal processing on the first ROI area image to obtain a second ROI area image; the outlier removal processing is to remove the extreme value pixels whose pixel values are at both ends of the value range of the gray histogram in the first ROI area image;

[0083] A target tracking module, which is used to cluster the pixel values of the second ROI area image obtained in the preprocessing module. If all pixel values are clustered into one class, it is regarded that there is no oil leakage at the moment when the second ROI area image is acquired, and S1-S3 are continued to be executed for the next frame of infrared monitoring image frame; if all pixel values are clustered into multiple classes, it is regarded that there may be oil leakage in the current target monitoring area. The average value of all cluster center pixel values is used as a first threshold to binarize the second ROI area image. If the pixel value is less than or equal to the first threshold, it is set to 1, otherwise it is set to 0, so as to establish a binary image containing suspected oil stain blocks; then, after removing isolated points from the binary image, screening is performed based on the minimum area, and all connected regions obtained by screening are updated to the target tracking list; the update process is as follows:

[0084] Traverse each connected region obtained by screening, and perform target matching of the current traversed connected region in the target tracking list. If the same tracking target cannot be matched, the current traversed connected region is added as a new tracking target in the target tracking list, and the target appearance times of the new tracking target are initialized to 1, and the consecutive disappearance times are set to 0; if the same tracking target is matched, the appearance times of this tracking target in the target tracking list are incremented by 1, and the consecutive disappearance times of this tracking target are set to 0; after traversing all the connected regions obtained by screening, judge whether there are original tracking targets in the target tracking list whose target appearance times have not been incremented. If so, increment their consecutive disappearance times by 1;

[0085] The equipment oil leakage alarm module is used to continuously iterate and execute the real-time image acquisition module, the preprocessing module, and the target tracking module. After iterating once and updating the target tracking list, it traverses all the tracking targets in the target tracking list. If the consecutive disappearance times of a tracking target exceed the second threshold, the tracking target is deleted from the target tracking list. If the appearance times of a tracking target exceed the third threshold and the average pixel fluctuation within the connected region of the tracking target is less than the fourth threshold, then the tracking target is considered to be an oil stain block caused by oil leakage, and the equipment oil leakage alarm is triggered.

[0086] Since the principle of solving problems of the above-mentioned infrared thermal imaging-based equipment oil leakage detection method is similar to that of the infrared thermal imaging-based equipment oil leakage detection system in the above embodiment of the present invention, the specific implementation forms of the modules in the system in this embodiment that are not fully described can also refer to the specific implementation forms of the method part shown in the above S1 to S4, and the repeated parts will not be elaborated.

[0087] In addition, it should be noted that in the system provided in the above embodiment, each module is equivalent to an ordered program module when being executed, so it essentially executes a data processing process. And those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the foregoing method embodiment, which will not be elaborated here. In each embodiment provided in the present application, the division of steps or modules in the method and system is only a logical function division, and there may be other division methods in actual implementation. For example, multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0088] In summary, the infrared thermal imaging-based equipment oil leakage detection method and system of the present invention use K-means clustering to perform real-time detection on the video images obtained by an infrared thermal imaging camera, and can automatically identify the oil leakage defect information existing in the infrared images. This process does not require human participation, thus reducing human errors. The present invention not only realizes the real-time intelligent detection of the equipment oil leakage situation, but also improves the detection efficiency and accuracy. In addition, when it is determined that there is an oil leakage situation in the equipment, an alarm message can be output in a timely manner, greatly improving the safety of production.

[0089] The above-described embodiments are only a preferred solution of the present invention, but they are not intended to limit the present invention. Those of ordinary skill in the relevant technical fields can still make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all technical solutions obtained by adopting equivalent replacement or equivalent transformation fall within the protection scope of the present invention.

Claims

1. An oil leakage detection method for equipment based on infrared thermal imaging, characterized in that, The steps include the following: S1. Obtain the infrared monitoring image frames of the target monitoring area in real time through an infrared thermal imaging camera with a fixed perspective, and extract the first ROI area image covering the range where the oil stain block appears from the infrared monitoring image frames according to the preset ROI area coordinate frame; S2. Conduct gray value statistics on the first ROI area image obtained in S1 to obtain the gray histogram of the first ROI area image, and then perform a point throwing process on the first ROI area image to obtain the second ROI area image; the point throwing process is to remove the extreme value pixels with pixel values at both ends of the value range of the gray histogram in the first ROI area image; S3. Cluster the pixel values of the second ROI area image obtained in S2. If all pixel values are clustered into one class, it is considered that there is no oil leakage at the moment when the second ROI area image is obtained, and continue to execute S1 - S3 for the next frame of infrared monitoring image frame; if all pixel values are clustered into multiple classes, it is considered that there may be oil leakage in the current target monitoring area. Use the average value of all cluster center pixel values as the first threshold to binarize the second ROI area image. If the pixel value is less than or equal to the first threshold, set it to 1, otherwise set it to 0, so as to establish a binary image containing suspected oil stain blocks; then remove the isolated points from the binary image and perform screening based on the minimum area, and update all the connected regions obtained by screening to the target tracking list; The update process is as follows: Traverse each connected region obtained by screening, and perform target matching for the currently traversed connected region in the target tracking list. If the same tracking target cannot be matched, add the currently traversed connected region as a new tracking target in the target tracking list, and initialize the target appearance times of this new tracking target to 1 and the consecutive disappearance times to 0; If the same tracking target is matched, add 1 to the appearance times of this tracking target in the target tracking list, and set the consecutive disappearance times of this tracking target to 0; After traversing all the connected regions obtained by screening, judge whether there are original tracking targets in the target tracking list whose target appearance times have not been incremented. If there are, increment their consecutive disappearance times by 1; S4. Continuously iterate and execute S1 - S3. After iterating once and updating the target tracking list, traverse all the tracking targets in the target tracking list. If the consecutive disappearance times of a tracking target exceed the second threshold, delete this tracking target from the target tracking list. If the appearance times of a tracking target exceed the third threshold and the average pixel fluctuation within the connected region of this tracking target is less than the fourth threshold, it is considered that this tracking target is an oil stain block caused by oil leakage, and trigger an equipment oil leakage alarm.

2. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 1, characterized in that, The infrared thermal imaging camera supports RTSP and ONVIF protocols, and the obtained infrared thermal imaging signal data is displayed after inputting the device IP, port number, username and password in the corresponding SDK.

3. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 1, wherein, During the point throwing process, it is necessary to remove the 5% pixels with the largest gray values and the 5% pixels with the smallest gray values within the value range of the gray histogram.

4. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 1, wherein, The clustering adopts K - means clustering, and its method is as follows: S31. Randomly select any two pixel points in the second ROI region image as the initial clustering centers; S32. Calculate the distances between each of the remaining pixel points in the second ROI region image and the two clustering centers, where the distance is the absolute value of the difference in gray values between the pixel point and the clustering center; S33. Divide each of the remaining pixel points in the second ROI region image into the cluster with the closest distance value, then calculate the average gray value of all pixel points in each cluster, and use the average value as the new clustering center; S34. Continuously iterate S32 and S33 until the clustering centers converge to obtain the final clustering result.

5. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 1, wherein The method for removing isolated points is as follows: Traverse each pixel point in the binary image. If the sum of the pixel values of 9 pixels within a 3×3 range centered on this pixel point is less than 9, then determine that this point is an isolated point and set the pixel value of the isolated point in the binary image to 0.

6. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 1, characterized in that, The method for screening based on the minimum area is as follows: Screen the connected regions in the binary image after removing isolated points according to the preset minimum area threshold of the oil stain block region, extract the connected regions with an area greater than the minimum area threshold of the oil stain block region, and filter out the connected regions with an area not greater than the minimum area threshold of the oil stain block region.

7. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 6, wherein, The minimum area threshold of the oil stain block region is 80 to 120 pixels.

8. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 1, wherein, When performing target matching for the currently traversed connected region in the target tracking list, the matching rule is as follows: Calculate the centroid of the currently traversed connected region, and determine whether the centroid falls within the latest bounding rectangle of a tracking target in the target tracking list and the area change rate of the two connected regions is less than 50%. If so, it is considered that the currently traversed connected region and the corresponding tracking target belong to the same tracking target. If not, it is considered that the currently traversed connected region is a new tracking target.

9. The method for detecting oil leakage of equipment based on infrared thermal imaging according to claim 1, characterized in that, When triggering the device oil leakage alarm, display the bounding rectangle of the oil stain block on the infrared monitoring image frame, and send an alarm signal through one or more of sound, light, electricity, image, and text.

10. An oil leakage detection system for equipment based on infrared thermal imaging, characterized in that, It includes: A real-time image acquisition module, which is used to continuously acquire infrared monitoring image frames of the target monitoring area through an infrared thermal imaging camera with a fixed perspective, and extract the first ROI region image covering the range where the oil stain block appears from the infrared monitoring image frames according to the preset ROI region coordinate frame; A preprocessing module, which is used to perform gray value statistics on the first ROI region image acquired by the real-time image acquisition module to obtain the gray histogram of the first ROI region image, and then perform point throwing processing on the first ROI region image to obtain the second ROI region image; the point throwing processing is to remove the extreme value pixels with pixel values at both ends of the value range of the gray histogram in the first ROI region image; The target tracking module is used to cluster the pixel values of the second ROI region image obtained in the preprocessing module. If all pixel values are clustered into one class, it is regarded that there is no oil leakage at the moment when the second ROI region image is acquired, and S1 - S3 are continued to be executed for the next frame of infrared monitoring image. If all pixel values are clustered into multiple classes, it is regarded that there may be oil leakage in the current target monitoring area. The second ROI region image is binarized with the average value of all cluster center pixel values as the first threshold. If the pixel value is less than or equal to the first threshold, it is set to 1, otherwise it is set to 0, so as to establish a binary image containing suspected oil stain blocks. Then, after removing isolated points from the binary image, screening is performed based on the minimum area, and all connected regions obtained by screening are updated to the target tracking list. The update process is as follows: Traverse each connected region obtained by screening, and perform target matching for the currently traversed connected region in the target tracking list. If the same tracking target cannot be matched, the currently traversed connected region is added as a new tracking target in the target tracking list, and the target appearance times of this new tracking target are initialized to 1, and the consecutive disappearance times are set to 0. If the same tracking target is matched, the appearance times of this tracking target in the target tracking list are incremented by 1, and the consecutive disappearance times of this tracking target are set to 0. After traversing all the connected regions obtained by screening, check whether there are original tracking targets in the target tracking list whose target appearance times have not been incremented. If there are, increment their consecutive disappearance times by 1. The device oil leakage alarm module is used to continuously iterate and execute the real-time image acquisition module, the preprocessing module, and the target tracking module. After iterating once and updating the target tracking list, traverse all the tracking targets in the target tracking list. If the consecutive disappearance times of a tracking target exceed the second threshold, delete this tracking target from the target tracking list. If the appearance times of a tracking target exceed the third threshold and the average pixel fluctuation within the connected region of this tracking target is less than the fourth threshold, it is considered that this tracking target is an oil stain block caused by oil leakage, and the device oil leakage alarm is triggered.

Images