A steam turbine lubricating oil leakage identification method, system, device and medium based on real-time video data

Abstract

A kind of steam turbine lubricating oil drip identification method, system, device and medium based on real-time video data, method includes obtaining lubricating oil drip video information by high-definition camera at steam turbine lubricating oil tank, the weighted average of RGB three channel components of color image in video information is used as the gray value of gray image, the picture space transformation of gray scale is carried out, the visual effect is improved using image enhancement method, image feature extraction is carried out using LBP, experimental data is collected in different scenes to generate material database, real-time video and material database are compared, it is judged whether feature vector matching is successful, if matching is successful, it proves that drip occurs, the result is pushed to operating personnel, if matching fails, it indicates that there is no drip problem;System, device and medium are used to realize the steam turbine lubricating oil drip identification method based on real-time video data;The present application realizes full autonomous intelligent monitoring at steam turbine lubricating oil main tank, and the requirement of field environment is low.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of steam turbine monitoring, and in particular to a steam turbine lubricating oil dripping identification method, system, device and medium based on real-time video data. BACKGROUND

[0002] The operation and management of thermal power plants is an important part of production management, and its central task is to reasonably arrange human and material resources to ensure the safe, stable and efficient operation of power generation equipment. As one of the three cores of thermal power units, the oil system of the steam turbine is prone to leakage. Although the ignition point of the fire-resistant oil is relatively high and is not easy to ignite, the ignition point of the lubricating oil is generally only 250℃, so it is easy to ignite. According to the data, before the accident of the steam turbine, there is often oil leakage, even smoke or small fire, but it has not been taken seriously and measures have not been taken to solve it quickly, so the dripping identification of the lubricating oil main tank is one of the key factors to ensure the stable operation of the steam turbine. However, because the operation of the thermal power plant relies on the regular inspection of the operation personnel, because of the time interval of the inspection, there are gaps in the inspection of some equipment, and it cannot be guaranteed to be problem-free for 24 hours. The dripping of the lubricating oil occurs at an indefinite time, so it is necessary to identify the dripping of the lubricating oil main tank to remind the operation personnel to handle it at the first time of dripping. If the dripping of the lubricating oil is not handled in time, it is likely to cause a fire accident, which seriously affects the safe and economic operation of the thermal power plant. The dripping identification of the lubricating oil main tank is the need to improve the reliability of the thermal power unit, the need to improve the economic efficiency of the power generation equipment, and the need to create a first-class thermal power plant.

[0003] The patent CN114283372A, entitled "Run-off, leakage and dripping image recognition method based on convolution neural network", collects video materials of the monitoring site through a camera device; the video materials are decomposed to obtain a plurality of picture materials, the picture materials are labeled through an online labeling tool, a motion target tracking algorithm is applied, and the regions of the run-off, leakage and dripping targets are labeled according to the motion target and expression and similarity measurement; an AI model is trained and arranged in the monitoring device, and the monitoring device with the AI model is used to decompose and identify the video materials collected on site; when there is a run-off, leakage and dripping event, the picture materials obtained by decomposition are labeled and an alarm and event linkage are performed. This method can automatically identify the run-off, leakage and dripping of liquids and gases and give an alarm, but the accuracy and real-time performance of the detection results of this patent are high only under the premise that the light in the video collection site is sufficient, the illumination angle is reasonable, and the pixel of the camera device is sufficient. SUMMARY

[0004] In view of the deficiencies of the prior art, the purpose of the present application is to provide a turbine lubricating oil dripping identification method, system, device and medium based on real-time video data, which installs a high-definition camera at the turbine lubricating oil main tank, collects real-time video data, uses an LBP local binary pattern method for feature extraction, obtains target information required by the system, realizes identification of dripping at the lubricating oil main tank, realizes all-weather and fully autonomous intelligent monitoring of the turbine lubricating oil main tank, and has low requirements on the site environment.

[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0006] A turbine lubricating oil dripping identification method based on real-time video data, comprising the following steps:

[0007] Step 1: Obtain lubricating oil dripping video information through a high-definition camera at the turbine lubricating oil tank, and perform weighted average on RGB three-channel components of a color image in the video information according to the importance of RGB three primary colors, so as to obtain a weighted average value as a gray value of a gray image;

[0008] Step 2: Perform spatial transformation on the grayed image in step 1 through an opencv cross-platform computer vision and machine learning software library, and the transformation mode is shearing, which converts the originally rectangular image into a square image through shearing;

[0009] Step 3: According to the image obtained in step 2, an image enhancement method is used to improve the visual effect of the image;

[0010] Step 4: Use an LBP local binary pattern to extract features from the image obtained in step 3 to obtain the required target information, so that the image is converted into information;

[0011] Step 5: If a feature vector is generated through LBP local binary pattern analysis in step 4, the analysis is successful, and step 6 is entered; if not, the analysis fails, and the failed image is analyzed again, if the number of analysis times is less than 9, the gray processing in step 1 is performed again, and if the number of analysis times reaches 9 and the analysis is still unsuccessful, the image is considered as a dirty image and is discarded;

[0012] Step 6: According to steps 1-5, a large amount of experimental data under different state scenes at the turbine lubricating oil main tank, and according to the signal timing of the high-definition camera at the turbine lubricating oil main tank, collect data of real-time video to generate a material database;

[0013] Step 7: According to the histogram of each small area, generate the LBP feature vector of the whole image, and analyze and compare the real-time video and the material database;

[0014] Step 8, judge whether the feature vector matching is successful, if the matching is successful, prove that the dripping occurs, push the result to the operator, if the matching fails, indicate that there is no dripping problem.

[0015] The step 1 weights and averages the RGB three-channel components of the color image, and the obtained weighted average value is taken as the gray value of the gray image, and the specific process is as follows:

[0016] According to the different sensitivities of human eyes to R, G and B three primary colors, a simulation experiment is carried out by using MATLAB, when the weighting coefficients of RGB three channels meet 0.299:0.587:0.114, the white color matched is the standard illuminant, that is:

[0017] f(x, y) = 0.299R(x, y) + 0.587G(x, y) + 0.114B(x, y).

[0018] The image enhancement method in step 3 adopts Fourier transform for frequency domain analysis, and the result of the analysis is used to enhance the local features of the image by high-pass filtering, and the specific process is as follows:

[0019] Step 3.1, the image is divided into N*N small areas, the filter function is H(u,v), the filling parameters are P=Q=2N, a new image f p (x,y) of size 4*N*N is formed;

[0020] Step 3.2, by performing Fourier inverse transform on f p (x,y), F(u,v) is obtained;

[0021] Step 3.3, the filter function H(u,v) is multiplied by F(u,v), and a real part about the image is obtained by using discrete inverse Fourier transform, and the real part is selected as a new image;

[0022] Step 3.4, by high-pass filtering, the image with enhanced local features is obtained, and the image is selected as a new image.

[0023] In step 4, LBP local binary pattern is used for feature extraction of the image, and the specific process is as follows:

[0024] Step 4.1, first divide the image into M*M small areas;

[0025] Step 4.2, for a pixel in each small area, compare the gray values of the adjacent 8 pixels with it, if the surrounding pixel values are greater than the center pixel value, the position of the pixel point is marked as 1, otherwise as 0, 8-bit binary number can be generated by comparing the 8 points in the 3*3 neighborhood, that is, the LBP local binary pattern value of the center pixel point of the window is obtained;

[0026] Step 4.3, calculate the histogram of each small area, that is, the frequency of each number, and then normalize the histogram;

[0027] Step 4.4, finally, the obtained statistical histogram of each small area is connected to form a feature vector, that is, the LBP local binary pattern texture feature vector of the whole image.

[0028] A steam turbine lubricating oil dripping identification system based on real-time video data, comprising:

[0029] A data acquisition layer, experimental data under different state scenarios at the steam turbine lubricating oil tank, real-time video is collected according to the signal time sequence of the high-definition camera at the lubricating oil tank, and the collected video image data is transmitted to a material database;

[0030] A preprocessing layer for optimizing and improving the image to achieve better input quality and prepare for video image analysis, forming the data required in the analysis stage through grayscale, spatial transformation and image enhancement;

[0031] A data analysis layer for analyzing the preprocessed image, extracting features from the image, obtaining the target information required by the system, and converting the image into information;

[0032] An information mining layer for analyzing and comparing the features of the targets in the image and the mutual relationship between the targets by constructing a model and training different state data;

[0033] An alarm pushing layer for building a corresponding alarm pushing interface according to the result identified by the information mining layer and directly pushing to the operation monitoring personnel when the dripping situation is found.

[0034] A device for identifying steam turbine lubricating oil dripping based on real-time video data, comprising:

[0035] A memory for storing a computer program for implementing a method for identifying steam turbine lubricating oil dripping based on real-time video data;

[0036] A processor for implementing a method for identifying steam turbine lubricating oil dripping based on real-time video data when executing the computer program.

[0037] A computer readable storage medium,

[0038] The computer readable storage medium stores a computer program, and the computer program can implement a method for identifying steam turbine lubricating oil dripping based on real-time video data when executed by a processor.

[0039] Compared with the prior art, the beneficial effects of the present application are:

[0040] 1. The present application combines the collected video data with image processing, deep learning network technology for processing, analysis and mining, realizes the turbine lubricating oil drip recognition function based on real-time video data, and improves the prediction accuracy.

[0041] 2. In step 4, the LBP local binary pattern method is used for feature extraction to obtain the target information required by the system, so that the image is converted into information, the LBP texture feature vector of the normal condition and a large amount of experimental data is established, the difference between the normal condition and the drip condition is found out through machine learning, the drip recognition of the lubricating oil main tank is realized, the environmental restriction is less, and the adaptation range is wide.

[0042] 3. The maximum number of comparisons in step 5 is 8, when it is greater than 8, the image is discarded, all operations are processed by the computer, if the maximum number of comparisons is 16, the processing time and the calculation ability will be geometrically improved, and the delay time will be greatly increased, therefore, the maximum 8 comparison method is used to realize real-time and ensure as few discarded images as possible to achieve as high accuracy as possible.

[0043] 4. In the early stage, a large amount of experimental data is obtained under different conditions, and the experimental comparison image is obtained, so that the influence of light and other factors on the turbine lubricating oil drip recognition is as small as possible, and the recognition accuracy is improved.

[0044] 5. The turbine lubricating oil drip recognition based on real-time video data uses image analysis method, and a large amount of data is used to analyze the turbine lubricating oil main tank, the analysis result is mined, the lubricating oil drip recognition is realized, when the drip occurs, the corresponding mode is used to inform the running monitoring personnel to make treatment, which provides guarantee for the safe operation of the power plant turbine, the reliability of the data sample is greatly improved, and the data quality is improved.

[0045] In summary, the present application realizes the turbine lubricating oil drip recognition of the thermal power plant, reasonably and correctly recognizes the drip occurrence, timely issues an alarm, can comprehensively improve the operation safety of the turbine side, improves the reliability of the equipment, treats the accident in the early stage, reduces the influence of the accident to the minimum, and finally improves the production and operation stability of the thermal power plant. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0047] The present application will be further described in detail below with reference to the drawings.

[0048] A steam turbine lubricating oil leakage identification method based on real-time video data, comprising the following steps:

[0049] Step 1, through the high-definition camera at the steam turbine lubricating oil tank, obtain the lubricating oil leakage video information, according to the importance of RGB three primary colors, the RGB three channel components of the color image in the video information are weighted and averaged, and the weighted average value is taken as the gray value of the gray image;

[0050] The weighted average method is used for gray processing, and the specific process is as follows:

[0051] According to the different sensitivities of human eyes to R, G and B three primary colors, MATLAB is used for simulation experiment, when the weighted coefficients of RGB three channels meet 0.299:0.587:0.114, the white prepared is the standard illuminant, that is:

[0052] f(x, y) = 0.299R(x, y) = = + 0.587G(x, y) + 0.114B(x, y)

[0053] Step 2, through using the open source computer vision and machine learning software library opencv, the space transformation is carried out on the already grayed picture, the transformation mode is cutting, through using cutting, the originally rectangular image is converted into a square, the purpose is to process the picture later;

[0054] Step 3, according to the picture obtained in step 2, the image enhancement method is adopted to improve the visual effect of the image; the high-pass filtering method in the frequency domain method is adopted, which mainly enhances the high-frequency signal of the image, makes the blurred picture clear, and enhances the local features of the image, and the specific process is as follows:

[0055] Step 3.1, the image is divided into N*N small areas, the filter function is H(u,v), the filling parameters P=Q=2N, a new image f p (x,y) with a size of 4*N*N is formed;

[0056] Step 3.2, by performing Fourier inverse transform on f p (x,y), F(u,v) is obtained;

[0057] Step 3.3, the filter function H(u,v) is multiplied by F(u,v), and a real part about the image is obtained by using discrete Fourier inverse transform, and the real part is selected as a new image;

[0058] Step 3.4, through the high-pass filtering method, the image with enhanced local features is obtained, and the image is selected as a new image;

[0059] Step 4, since the on-site light illumination is uncertain, the LBP local binary pattern method is used to extract features of the image to obtain the target information required by the system, so that the image is converted into information, and the specific process is as follows:

[0060] Step 4.1, first, the image is divided into M*M small regions;

[0061] Step 4.2, for a pixel in each small region, the gray values of the adjacent 8 pixels are compared with it, if the values of the surrounding pixels are greater than the value of the center pixel, then the position of the pixel point is marked as 1, otherwise as 0. 8 points in a 3*3 neighborhood can generate 8-bit binary number, that is, the LBP value of the center pixel point of the window is obtained;

[0062] Step 4.3, then the histogram of each small region is calculated, that is, the frequency of each number (assuming it is a decimal LBP value) appearing; then the histogram is normalized.

[0063] Step 4.4, finally, the statistical histogram of each small region obtained is connected to form a feature vector, that is, the LBP texture feature vector of the whole image;

[0064] Step 5, if the feature vector is generated by the LBP local binary pattern analysis in step 4, the analysis is successful, and step 6 is entered; if not, the analysis fails, and the image that fails to analyze is analyzed again, if the number of analysis times is less than 9 times, the gray scale processing is restarted in step 1, when the 9th time is reached and it is still unsuccessful, the image is considered as a dirty image and is discarded;

[0065] Step 6, according to steps 1-5, a large number of experimental data under different state scenarios at the main oil tank of the steam turbine lubricating oil, and according to the signal timing of the high-definition camera at the main oil tank of the lubricating oil, the real-time video is collected to generate a material database;

[0066] Step 7, according to the histogram of each small region, the LBP feature vector of the whole image is generated for the real-time video and the material database, and analysis and comparison are performed;

[0067] Step 8, it is judged whether the feature vector matching is successful, if the matching is successful, it is proved that dripping occurs, and the result is pushed to the operator, if the matching fails, it indicates that there is no dripping problem.

[0068] A steam turbine lubricating oil dripping identification system based on real-time video data, comprising a data acquisition layer, a preprocessing layer, a data analysis layer, an information mining layer, and an alarm pushing layer.

[0069] The data source of the data acquisition layer is two, one is a large number of experimental data in different state scenarios at the main oil tank of the steam turbine lubricating oil, the experimental scenario elements include light, dripping amount, etc., and the other is to collect real-time video according to the signal time sequence of the high-definition camera at the main oil tank of the lubricating oil, and transmit the collected video image data to the material database.

[0070] The preprocessing layer is to optimize and improve the data of the acquisition layer to achieve better input quality and prepare for video image analysis. Through grayscale, geometric transformation and image enhancement, the data required by the analysis stage is finally formed, and the implementation steps are as follows:

[0071] 1. Grayscale processing using weighted average method;

[0072] The grayscale method used is weighted average method, which weights the RGB three-channel components of the color image according to the importance of each channel, and then takes the weighted average value as the grayscale value of the grayscale image (Python can be used for programming calculation here), the calculation formula is as follows:

[0073] f(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y)

[0074] 2. Geometric transformation using opencv;

[0075] Through the use of open source computer vision and machine learning software library opencv, geometric transformation is performed on the already grayscale image, including expansion, reduction, translation, rotation, and flipping.

[0076] 3. Image enhancement processing of video image using frequency domain method;

[0077] Image enhancement is a distortion process, and image enhancement is mainly used to improve the visual effect of the image and highlight the parts needed in the image. Then, using filtering method, the original image is restored or reconstructed. The frequency domain method regards the image as a two-dimensional signal, and through Gaussian high-pass filtering, the image is smoothed or sharpened. Through the Gaussian high-pass filtering formula, Python is used for programming calculation, and finally the result after image enhancement is obtained.

[0078] The data analysis layer is to analyze the image through the preprocessed data, extract the features of the image, obtain the target information required by the system, and convert the image into information. Since the light intensity of the scene is uncertain, the feature extraction method is LBP local binary pattern method, and the specific steps are as follows:

[0079] 1. First, divide the image into M*M small areas;

[0080] 2. For each pixel in each small area, compare the gray value of the adjacent 8 pixels with it, if the value of the surrounding pixels is greater than the value of the center pixel, the position of the pixel is marked as 1, otherwise 0, 8 points in the 3*3 neighborhood can generate 8-bit binary number, that is, the LBP value of the center pixel of the window is obtained;

[0081] 3. Then calculate the histogram of each small area, that is, the frequency of each number (assuming that it is a decimal LBP value) appears; then the histogram is normalized.

[0082] 4. Finally, the obtained statistical histogram of each small area is connected to form a feature vector, that is, the LBP texture feature vector of the whole image.

[0083] When the image analysis is successful and the feature vector is generated, the information mining feature matching is carried out, if the analysis fails, the preprocessing is carried out again, when the preprocessing reaches 8 times and is still unsuccessful, the image is considered as a dirty image, and the image is discarded.

[0084] The information mining layer learns the features of the target in the image and the mutual relationship of each target by constructing a model training for different state data, and obtains the information contained. The principle is to establish the LBP texture feature vector of the normal situation and the LBP texture feature vector of a large amount of experimental data, find out the difference between the normal situation and the dripping situation through model training and machine learning, and realize the recognition of the dripping of the main oil tank of the lubricating oil.

[0085] The alarm pushing layer is to build a corresponding alarm pushing interface according to the result, and when the dripping situation is found, it is directly pushed to the running monitor personnel.

[0086] In summary, the present application correctly identifies the dripping situation, and timely issues an alarm. The method can comprehensively improve the operation safety of the steam turbine side, improve the reliability of the equipment, handle the accident in the early stage, minimize the impact of the accident, and finally improve the production and operation stability of the thermal power plant.

[0087] A kind of steam turbine lubricating oil dripping identification equipment based on real-time video data, comprising:

[0088] Memory: for storing the computer program for realizing a kind of steam turbine lubricating oil dripping identification method based on real-time video data;

[0089] Processor, for realizing a kind of steam turbine lubricating oil dripping identification method based on real-time video data when the computer program is executed.

[0090] A computer readable storage medium stores a computer program, and the computer program can realize a steam turbine lubricating oil dripping identification method based on real-time video data when executed by a processor.

Claims

1. A method for identifying turbine lubricating oil dripping based on real-time video data, characterized in that, Includes the following steps: Step 1: Obtain video information of lubricating oil dripping through a high-definition camera at the turbine lubricating oil tank. According to the importance of the three primary colors of RGB, the RGB three-channel components of the color image in the video information are weighted and averaged. The weighted average value is used as the gray value of the grayscale image. In step 1, the RGB three-channel components of the color image are weighted and averaged, and the resulting weighted average value is used as the grayscale value of the grayscale image. The specific process is as follows: Based on the different sensitivities of the human eye to the three primary colors R, G, and B, a simulation experiment was conducted using MATLAB. When the weighting coefficients of the RGB channels satisfy 0.299: 0.587: 0.114, the resulting white is a standard illuminant, i.e.: ； Step 2: The image that has been converted to grayscale in Step 1 is spatially transformed using the OpenCV cross-platform computer vision and machine learning software library. The transformation method is cropping, which transforms the original rectangular image into a square. Step 3: Based on the image obtained in Step 2, use image enhancement methods to improve the visual effect of the image; The image enhancement method in step 3 uses Fourier transform for frequency domain analysis, and the analysis results are used to enhance the local features of the image using high-pass filtering. The specific process is as follows: Step 3.1: Divide the image into N*N small regions, set the filtering function to H(u,v), and the padding parameter P=Q=2N, to form a new image f of size 4*N*N. p (x,y); Step 3.2, by adjusting f p Perform an inverse Fourier transform on (x,y) to obtain F(u,v); Step 3.3: Multiply the filter function H(u,v) and F(u,v), and use the inverse discrete Fourier transform to obtain a real part of the image. Select this real part as the new image. Step 3.4: Using high-pass filtering, obtain an image with enhanced local features, and select this image as the new image; Step 4: Use LBP local binary mode to extract features from the image obtained in step 3 to obtain the required target information, thus converting the image into information; Step 5: If the LBP local binary mode analysis in step 4 produces a feature vector, the analysis is successful and proceeds to step 6; otherwise, the analysis fails and the failed image is analyzed again. If the number of analysis attempts is less than 9, the process returns to step 1 to re-encode the image. If the analysis fails after the 9th attempt, the image is considered a dirty image and is discarded. Step 6: Based on the experimental data from various scenarios conducted at the main lubricating oil tank of the steam turbine in steps 1-5, and the signal timing data from the high-definition camera at the main lubricating oil tank, collect real-time video data to generate a material database. Step 7: For the real-time video and material database, generate the LBP feature vector of the whole image based on the histogram of each small region, and perform analysis and comparison. Step 8: Determine if the feature vector matching is successful. If the matching is successful, it proves that a leak has occurred, and the result is pushed to the operators. If the matching fails, it means that there is no leak problem.

2. The method for identifying turbine lubricating oil dripping based on real-time video data according to claim 1, characterized in that, Step 4 uses LBP (Local Binary Pattern) to extract features from the image. The specific process is as follows: Step 4.1: First, divide the image into M*M small regions; Step 4.2: For a pixel in each small region, compare the gray values ​​of the 8 adjacent pixels with it. If the value of the surrounding pixels is greater than the value of the center pixel, the position of the pixel is marked as 1; otherwise, it is marked as 0. The 8 points in the 3*3 neighborhood can generate an 8-bit binary number by comparison, which is the LBP local binary mode value of the center pixel of the region. Step 4.3: Calculate the histogram for each small region, i.e., the frequency of each number, and then normalize the histogram. Step 4.4: Connect the statistical histograms of each small region to form a feature vector, which is the LBP local binary pattern texture feature vector of the entire image.

3. A turbine lubricating oil leak identification system based on real-time video data, based on the method of claim 1, characterized in that, include: The data acquisition layer collects experimental data under different conditions at the turbine lubricating oil tank. It collects real-time video based on the signal timing of the high-definition camera at the lubricating oil tank and transmits the collected video image data to the material database. The preprocessing layer is used to optimize and improve the image to achieve better input quality and prepare it for video image analysis. Through grayscale conversion, spatial transformation, and image enhancement, it forms the image required for the analysis stage. The data analysis layer is used to analyze the pre-processed images, extract features from the images, and obtain the target information required by the system, thus converting the images into information. If the analysis fails, the failed image will be analyzed again. If the number of analysis attempts is less than 9, the process will return to the preprocessing layer and be reprocessed. If the analysis fails after the 9th attempt, the image is considered a dirty image and will be discarded. The information mining layer analyzes and compares the features of targets in the image and the relationships between them by building and training models on data from different states. The alarm push layer is used to build a corresponding alarm push interface based on the results identified by the information mining layer. When a leak is detected, the alarm is directly pushed to the operation monitoring personnel.

4. A device for identifying turbine lubricating oil leaks based on real-time video data, characterized in that, include: Memory: Used to store the computer program that implements a method for identifying turbine lubricating oil leaks based on real-time video data; A processor is configured to execute the computer program to implement the turbine lubricating oil leakage identification method based on real-time video data as described in any one of claims 1-2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the turbine lubricating oil leakage identification method based on real-time video data as described in any one of claims 1-2.

Images