An Image Processing Method and System for the Surface of a Pipeline Pump for Ice Detection

Abstract

The present application provides a method and system for processing images of the surface of a pipeline pump for ice detection. The method includes: collecting visible light images and infrared images of the surface of the pipeline pump through a visible light device and an infrared device, and performing preprocessing to obtain a grayscale visible light image and a grayscale infrared image, and collecting the real-time time; performing infrared uniformity analysis based on the grayscale infrared image to obtain an infrared uniformity parameter, and performing visible light uniformity analysis based on the grayscale visible light image to obtain a visible light uniformity parameter; performing visible light uniformity prediction of the pipeline pump based on the real-time time to obtain a visible light interference coefficient; performing ice formation probability prediction based on the infrared uniformity parameter and the visible light uniformity parameter to obtain an infrared ice formation probability and a visible light ice formation probability, and performing processing and annotation using the visible light interference coefficient to obtain a processing result. The present application solves the technical problems of low accuracy of ice detection of pipeline pumps and poor environmental adaptability in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to an image processing method and system for the surface of a pipeline pump for ice detection. Background Art

[0002] Ice formation on a pipeline pump can cause equipment failures and safety hazards. In the prior art, the ice detection of a pipeline pump is mainly carried out through manual inspections or single-sensor detections (such as only using a temperature sensor or visible light imaging detection). However, the efficiency of manual inspections is low, and single-sensor detections are extremely vulnerable to environmental interference, resulting in low accuracy and poor environmental adaptability of the ice detection method. Summary of the Invention

[0003] In view of the technical problems of low accuracy and poor environmental adaptability in ice detection of pipeline pumps in the prior art, the present invention provides an image processing method and system for the surface of a pipeline pump for ice detection.

[0004] The technical solutions for the present invention to solve the above technical problems are as follows:

[0005] In a first aspect, the present invention provides an image processing method for the surface of a pipeline pump for ice detection, including:

[0006] Collecting visible light images and infrared images of the surface of the pipeline pump through a visible light device and an infrared device, and performing preprocessing to obtain a grayscale visible light image, a grayscale infrared image, and sensing the real-time acquisition time;

[0007] Performing infrared uniformity analysis based on the grayscale infrared image to obtain an infrared uniformity parameter, and performing visible light uniformity analysis based on the grayscale visible light image to obtain a visible light uniformity parameter;

[0008] Performing visible light uniformity prediction of the pipeline pump based on the real-time time to obtain a predicted visible light uniformity parameter, and verifying it with the visible light uniformity parameter to obtain a visible light interference coefficient;

[0009] Performing ice formation probability prediction based on the infrared uniformity parameter and the visible light uniformity parameter respectively to obtain an infrared ice formation probability and a visible light ice formation probability, and processing and marking them with the visible light interference coefficient to obtain an ice detection image processing result.

[0010] In a second aspect, the present invention provides an image processing system for the surface of a pipeline pump for ice detection, including:

[0011] A data acquisition module for collecting visible light images and infrared images of the surface of the pipeline pump through a visible light device and an infrared device, and performing preprocessing to obtain a grayscale visible light image, a grayscale infrared image, and sensing the real-time acquisition time;

[0012] The uniformity analysis module is used to perform infrared uniformity analysis based on the grayscale infrared image to obtain infrared uniformity parameters, and perform visible light uniformity analysis based on the grayscale visible light image to obtain visible light uniformity parameters;

[0013] The interference analysis module is used to perform prediction of the visible light uniformity of the pipeline pump according to the real-time time to obtain predicted visible light uniformity parameters, and verify them with the visible light uniformity parameters to obtain a visible light interference coefficient;

[0014] The fusion output module is used to perform icing probability prediction according to the infrared uniformity parameters and visible light uniformity parameters respectively to obtain an infrared icing probability and a visible light icing probability, and perform processing and annotation using the visible light interference coefficient to obtain an icing detection image processing result.

[0015] The beneficial effects of the present invention are as follows:

[0016] This application first collects visible light images and infrared images on the surface of the pipeline pump with time series characteristics, providing reliable multi-modal data support for icing detection; then calculates the standard deviation of the image grayscale value, intuitively reflecting the icing situation of the pipeline pump through the quantified value; and fully considers the adverse impact of the reflection situation on the icing detection result, calculates the visible light interference coefficient to reflect the probability of the current visible light uniformity parameters being affected by reflection; finally, dynamically adjusts the visible light weight and infrared weight through the visible light interference coefficient to obtain a fused icing probability and output an accurate and anti-environmental interference icing detection result.

[0017] Through the above technical solution, this application collects visible light images and infrared images on the surface of the pipeline pump with time series characteristics, based on the significant difference in the standard deviation of the image grayscale value before and after icing, and fully considers the adverse impact of the reflection situation on the icing detection result, dynamically adjusts the visible light weight and infrared weight through the visible light interference coefficient to obtain a fused icing probability. In this way, the accuracy and robustness of icing detection are improved. Description of the Drawings

[0018] Figure 1 It is a schematic flowchart of a method for processing images on the surface of a pipeline pump for icing detection provided by the present invention;

[0019] Figure 2 It is a schematic structural diagram of a system for processing images on the surface of a pipeline pump for icing detection provided by the present invention.

[0020] In the drawings, the components represented by each reference numeral are as follows:

[0021] The data acquisition module 11, the uniformity analysis module 12, the interference analysis module 13, and the fusion output module 14. Specific Embodiments

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0023] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0024] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration, or explanation". Any embodiment described as "for example" in the present invention is not necessarily construed as being more preferred or having more advantages than other embodiments. In order for any person skilled in the art to implement and use the present invention, the following description is given. In the following description, details are set forth for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be implemented without the use of these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessary details from obscuring the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope that conforms to the principles and features disclosed in the present invention.

[0025] Embodiment 1, as Figure 1 shown, the embodiment of the present invention provides a tool life evaluation method combined with tool taking logs, including:

[0026] S10: Collect visible light images and infrared images on the surface of the pipeline pump through a visible light device and an infrared device, and perform preprocessing to obtain grayscale visible light images and grayscale infrared images, and sense the real-time acquisition time;

[0027] In the prior art, the icing detection of pipeline pumps is mainly carried out through manual inspections or single-sensor detections. However, the efficiency of manual inspections is low, and single-sensor detections are extremely vulnerable to environmental interference. For example, visible light imaging methods are extremely vulnerable to interference from changes in light. Therefore, traditional icing detection methods have a lagging response, poor environmental adaptability, and a high false alarm rate.

[0028] To address the above problems, the present application collects visible light images (reflecting surface optical characteristics) and infrared images (reflecting temperature distribution), records the time stamps at the collection moments, and then performs preprocessing to obtain grayscale visible light images and grayscale infrared images with temporal characteristics, which are used as reliable bases for ice detection.

[0029] Specifically, step S10 in the method includes:

[0030] Collect visible light images and infrared images of the surface of the pipeline pump through visible light devices and infrared devices;

[0031] Crop and perform grayscale processing on the visible light images and infrared images, retain the images of the pipeline pump area, and obtain grayscale visible light images and grayscale infrared images;

[0032] Sense the real-time acquisition time.

[0033] In the embodiments of the present application, first, visible light images and infrared images of the surface of the pipeline pump are collected through visible light devices and infrared devices, and the time stamps at the collection moments are recorded. Among them, the visible light images show significant differences in optical characteristics before and after the pipeline pump freezes. Specifically: when not frozen, the surface of the pump body mainly presents a dull color of the metal material. Due to the influence of surface roughness, the reflected light distribution is uneven and there may be miscellaneous color patches such as rust or oil stains; after freezing, the formation of the ice layer will greatly enhance the specular reflection effect, resulting in a high-brightness white feature in the covered area, and the surface texture tends to be smooth. Experimental data shows that the gray value of the frozen area is usually 30%-50% higher than that of the unfrozen area. The infrared images show significant differences in thermal distribution characteristics before and after the pipeline pump freezes. Specifically: when not frozen, due to the influence of the heat generated during operation and environmental thermal radiation of the pump body, the temperature distribution shows uneven characteristics; after freezing, the latent heat of phase change effect of the ice layer will make the surface temperature distribution tend to be uniform. Experimental data shows that the standard deviation of the infrared images of the frozen area will decrease by more than 40%, and there will be a temperature step of more than 5°C compared with the non-frozen area. Therefore, through the appearance of high-brightness features in the visible light images and the tendency to be uniform in the infrared images after the pipeline pump freezes, complementary criteria are formed to provide a reliable basis for ice detection.

[0034] Exemplarily, the visible light device uses an ordinary optical camera to collect visible light images in the wavelength range of 380-780nm to reflect the optical reflection characteristics of the surface of the pipeline pump. The infrared device uses a thermal imaging camera to collect infrared radiation with wavelengths of 3-5μm or 8-14μm to reflect the temperature distribution characteristics of the object surface.

[0035] Secondly, the collected visible light images and infrared images are cropped and grayscale processed to obtain grayscale visible light images and grayscale infrared images. Among them, in the cropping process, the redundant parts are cropped off, and only the images of the pipeline pump area are retained, so as to highlight the key points and reduce the amount of information for image processing. The grayscale processing is to convert the visible light images and infrared images into grayscale visible light images and grayscale infrared images. Specifically, converting the color visible light images into black and white grayscale images can eliminate color interference, highlight the high reflectivity characteristics of the ice layer (specifically manifested as bright white areas), and at the same time reduce the data dimension and improve the processing efficiency. By using grayscale processing methods such as adaptive histogram equalization for infrared images, the contrast of the icing area can be enhanced, and the accuracy of feature recognition can be improved. In this way, the data formats of the visible light images and infrared images are unified, facilitating subsequent feature extraction and detection analysis.

[0036] Furthermore, the real-time time at the acquisition moment is recorded. Among them, the real-time time is the acquisition timestamp accurate to minutes when the visible light device and the infrared device acquire the visible light images and infrared images of the surface of the pipeline pump. By acquiring the real-time time, grayscale visible light images and grayscale infrared images with temporal sequence characteristics are obtained. Exemplarily, the acquisition real-time time of a certain visible light image is: 2024-05-05 14:00.

[0037] In summary, compared with the prior art, the present application acquires visible light images and infrared images of the surface of the pipeline pump, simultaneously integrates the optical characteristics and thermodynamic characteristics of the surface of the pipeline pump, and cooperates with temporal sequence analysis, which can provide reliable multi-modal data support for subsequent pipeline pump icing detection.

[0038] S20: According to the grayscale infrared image, perform infrared uniformity analysis to obtain an infrared uniformity parameter, and according to the grayscale visible light image, perform visible light uniformity analysis to obtain a visible light uniformity parameter;

[0039] Icing will significantly change the thermodynamic and optical characteristics of the outer surface of the pipeline pump: After icing, the overall temperature distribution on the outside of the pipeline pump tends to be uniform, reducing the degree of dispersion of the grayscale values of the infrared grayscale image (i.e., the standard deviation decreases); moreover, most areas on the outside of the pipeline pump are wrapped by the ice layer, and the high reflectivity of the ice layer will make its surface show a uniform bright white, reducing the degree of dispersion of the grayscale values of the visible light grayscale image (i.e., the standard deviation decreases).

[0040] Therefore, based on the obvious differences in the standard deviations of the grayscale values of the collected grayscale infrared images and grayscale visible light images before and after icing, the present application calculates all the grayscale values of all pixel points of the grayscale infrared image and the grayscale visible light image respectively, and then calculates their standard deviations to obtain the infrared uniformity parameter and the visible light uniformity parameter.

[0041] Specifically, step S20 in the method includes:

[0042] Based on all the gray values in the grayscale infrared image, perform infrared uniformity analysis and calculation to obtain infrared uniformity parameters;

[0043] Based on all the gray values in the grayscale visible light image, perform visible light uniformity analysis and calculation to obtain visible light uniformity parameters.

[0044] In the embodiment of the present application, first, based on all the gray values in the grayscale infrared image, perform infrared uniformity analysis and calculation to obtain infrared uniformity parameters. Among them, the infrared uniformity parameter is the standard deviation of all the gray values of all the pixel points in the grayscale infrared image. After icing, since the overall temperature outside the pipeline pump is relatively consistent, the standard deviation is small.

[0045] Secondly, based on all the gray values in the grayscale visible light image, perform visible light uniformity analysis and calculation to obtain visible light uniformity parameters. Among them, the visible light uniformity parameter is the standard deviation of all the gray values of all the pixel points in the grayscale visible light image. After icing, since most of the outside of the pipeline pump is the transparent white of ice, the standard deviation is small.

[0046] Further, the "based on all the gray values in the grayscale infrared image, perform infrared uniformity analysis and calculation to obtain infrared uniformity parameters" includes:

[0047] Extract all the gray values of all the pixel points in the grayscale infrared image to obtain an infrared gray value matrix;

[0048] Calculate the standard deviation of all the infrared gray values in the infrared gray value matrix to obtain infrared uniformity parameters.

[0049] In the embodiment of the present application, first, extract all the gray values of all the pixel points in the grayscale infrared image to obtain an infrared gray value matrix. Exemplarily, based on the preprocessed grayscale infrared image, automate the extraction through a scientific computing library (such as OpenCV or NumPy in Python): call the image processing interface to convert the grayscale infrared image into a two-dimensional array matrix, where each matrix element image[y,x] exactly corresponds to the pixel gray value at the image coordinate (x,y), thereby obtaining the infrared gray value matrix of the grayscale infrared image.

[0050] Secondly, obtain infrared uniformity parameters by calculating the standard deviation of all the infrared gray values in the infrared gray value matrix. Specifically, the standard deviation is calculated by the following formula:

[0051] ;

[0052] where N is the total number of pixels, x iis the gray value of the i-th element in the matrix, and u is the arithmetic mean of the matrix (i.e., the average gray value of the gray infrared image).

[0053] Exemplarily, the gray values of a 640×480 resolution gray infrared image are extracted through OpenCV of Python, 311040 pixel points are traversed therein to obtain a 640×480 infrared gray value matrix, and then its standard deviation is calculated as the infrared uniformity parameter.

[0054] In summary, compared with the prior art, according to the present application, the standard deviation of the gray values of the infrared gray image and the visible light gray image of the pipeline pump after icing decreases. First, according to the gray infrared image, the standard deviation of the gray values of all pixel points in the image is calculated to obtain the infrared uniformity parameter, and then according to the gray visible light image, the standard deviation of the gray values of all pixel points in the image is calculated to obtain the visible light uniformity parameter. The icing condition of the pipeline pump is intuitively reflected by the quantified value, and effective support is provided for subsequent icing detection.

[0055] S30: According to the real-time time, perform the visible light uniformity prediction of the pipeline pump, obtain the predicted visible light uniformity parameter, verify it with the visible light uniformity parameter, and obtain the visible light interference coefficient;

[0056] Although the foregoing visible light uniformity parameter can reflect the icing condition on the surface of the pipeline pump from the perspective of optical characteristics, it is extremely susceptible to the change of the solar incident angle. Especially when the surface of the pipeline pump is completely reflected under strong reflection conditions, the obtained gray visible light image will show a highly consistent gray value distribution, which is similar to the image characteristics in the icing state, resulting in misjudgment. For example, at noon in summer, the sun shines directly on the pipeline pump, and its metal surface is completely reflected, and the gray values of the obtained gray visible light image are very uniform, but in fact, it is not frozen.

[0057] Therefore, in view of the above problems, the present application considers the influence of reflection on the visible light uniformity parameter. By constructing a visible light uniformity predictor, the real-time time is input, the predicted visible light uniformity parameter is output, and then the similarity between the predicted visible light uniformity parameter and the visible light uniformity parameter is analyzed to obtain the visible light interference coefficient. The greater the visible light interference coefficient, the greater the probability that the current visible light uniformity parameter is affected by reflection, thereby reducing the misjudgment caused by the influence of reflection.

[0058] Specifically, step S30 in the method includes:

[0059] According to the monitoring data of the pipeline pump in the non-icing state within the historical time, collect the set of sample real-time times, and collect the visible light uniformity parameters of the visible light image of the pipeline pump under the influence of reflection at different sample real-time times, and label to obtain the set of sample visible light uniformity parameters;

[0060] A multi-layer feedforward neural network is used to construct a visible light uniformity predictor;

[0061] Using the sample real-time time set and the sample visible light uniformity parameter set, the network parameters of the visible light uniformity predictor are trained and optimized until convergence;

[0062] The real-time time is input into the visible light uniformity predictor to predict and output the predicted visible light uniformity parameter;

[0063] According to the predicted visible light uniformity parameter and the visible light uniformity parameter, verification calculation is performed to obtain the visible light interference coefficient.

[0064] In the embodiment of the present application, first, a sample real-time time set at different time periods (accurate to minutes) when the pipeline pump is in an unfrozen state is collected, and the visible light uniformity parameters of visible light images at different real-time times under the influence of specular reflection are collected, and the sample visible light uniformity parameter set is obtained through annotation. Exemplarily, a certain pipeline pump is directly irradiated by the sun at noon in summer (from 11:00 to 14:00), and there is a specular reflection phenomenon on its surface. Therefore, the sample real-time time set is continuously collected at 15-minute intervals, and then the visible light uniformity parameters of the visible light images of the pipeline pump at the sample real-time times are collected correspondingly, and a set in which the real-time time and the visible light uniformity parameter correspond one by one is obtained through annotation.

[0065] Secondly, a multi-layer feedforward neural network is used to construct a visible light uniformity predictor. Exemplarily, the visible light uniformity predictor is constructed by a three-layer fully connected neural network (MLP). The input layer receives a normalized time feature vector (including year, month, day, hour, and minute, such as 2024-05-05 14:00). The hidden layer adopts a progressive design of 64-32 nodes, and each layer is configured with a ReLU activation function and Dropout (p = 0.2) regularization. The output layer predicts and outputs the predicted visible light uniformity parameter.

[0066] Thirdly, using the sample real-time time set and the sample visible light uniformity parameter set, the network parameters of the visible light uniformity predictor are trained and optimized until convergence. Exemplarily, the sample real-time time set and the sample visible light uniformity parameter set are divided into a training set, a validation set, and a test set according to a ratio of 7:1.5:1.5. After continuous training, optimization, and passing the test, a visible light uniformity predictor with convergence (the convergence condition can be set as accuracy > 95%) is obtained. Further, the model training process can be realized through the following technical path: using the mean squared error loss function and L2 regularization, preventing overfitting through early stopping, and dynamically adjusting the learning rate. The model convergence criterion is that the test set MSE < 0.5, corresponding to a prediction error less than 0.7 gray levels, ensuring that it can accurately predict and output the predicted visible light uniformity parameter.

[0067] Finally, input the real-time time into the visible light uniformity predictor, and predict and output the predicted visible light uniformity parameter. Exemplarily, input the real-time time 2024-05-05 14:00 into the visible light uniformity predictor, and the predicted visible light uniformity parameter is predicted and output as 13.8.

[0068] Further, according to the predicted visible light uniformity parameter and the visible light uniformity parameter, perform a verification calculation to obtain a visible light interference coefficient. The visible light interference coefficient reflects the degree to which the visible light uniformity parameter is affected by reflection. The larger the visible light interference coefficient, the greater the probability that the current visible light uniformity parameter is affected by reflection.

[0069] Specifically, the "performing a verification calculation according to the predicted visible light uniformity parameter and the visible light uniformity parameter to obtain a visible light interference coefficient" includes:

[0070] Calculate the absolute difference between the predicted visible light uniformity parameter and the visible light uniformity parameter;

[0071] Calculate the ratio of the absolute difference to the predicted visible light uniformity parameter, and subtract this ratio from 1 to obtain the visible light interference coefficient.

[0072] In the embodiment of the present application, first calculate the absolute difference between the predicted visible light uniformity parameter and the visible light uniformity parameter. Among them, the absolute difference reflects the similarity between the predicted visible light uniformity parameter and the visible light uniformity parameter. The smaller the absolute difference, the higher the probability that there is a reflection situation at the current moment, and the larger the absolute difference, the lower the probability that there is a reflection situation at the current moment. Exemplarily, at noon on a sunny day, the predicted visible light uniformity parameter is 13.8, and the actually detected visible light uniformity parameter is 13.5, and the absolute difference is 0.3; at the same time on a cloudy day, the predicted visible light uniformity parameter is still 13.8, and the actually detected visible light uniformity parameter is 16.1, and the absolute difference is 2.3 at this time.

[0073] Secondly, calculate the ratio of the absolute difference to the predicted visible light uniformity parameter, and subtract this ratio from 1 to obtain the visible light interference coefficient. The visible light interference coefficient reflects the probability that the current visible light uniformity parameter is affected by reflection. The larger the visible light interference coefficient, the greater the probability that the current visible light uniformity parameter is affected by reflection, and the smaller the visible light interference coefficient, the smaller the probability that the current visible light uniformity parameter is affected by reflection. Exemplarily, at noon on a sunny day, the predicted visible light uniformity parameter is 13.5, the actually detected visible light uniformity parameter is 13.8, the absolute difference is 0.3, and the visible light interference coefficient is 0.98; at the same time on a cloudy day, the predicted visible light uniformity parameter is still 13.5, the actually detected visible light uniformity parameter is 16.1, at this time the absolute difference is 2.3, and the visible light interference coefficient is 0.83.

[0074] In summary, compared with the prior art, the present application uses a multi-layer feedforward neural network to construct a visible light uniformity predictor. By inputting the real-time time, the predicted visible light uniformity parameter is output. Then, based on the predicted visible light uniformity parameter and the actually detected visible light uniformity parameter, the visible light interference coefficient is calculated, and this coefficient reflects the probability that the current visible light uniformity parameter is affected by reflection. In this way, the misjudgment of the result caused by the influence of reflection is reduced.

[0075] S40: According to the infrared uniformity parameter and the visible light uniformity parameter, perform icing probability predictions respectively to obtain the infrared icing probability and the visible light icing probability, and use the visible light interference coefficient for processing and marking to obtain the icing detection image processing result.

[0076] When fusing and predicting the icing condition of the pipeline pump based on the foregoing infrared uniformity parameter and visible light uniformity parameter, since the visible light uniformity parameter is extremely susceptible to the influence of reflection, the visible light weight and the infrared weight cannot be fixed during the prediction process. Specifically, when the visible light interference coefficient increases (indicating that the current probability of being affected by reflection is relatively large), the visible light weight should be reduced and the infrared weight should be increased to reduce the adverse effect of the reflection situation on the icing prediction result.

[0077] To address the above problems, the present application first constructs an icing predictor based on a multi-layer feedforward neural network, inputs the infrared uniformity parameter and the visible light uniformity parameter respectively, and outputs the infrared icing probability and the visible light icing probability. Then, according to the ratio of the average visible light interference coefficient to the current visible light interference coefficient, the preset visible light weight is corrected and calculated to obtain the visible light weight, and thus the fused icing probability is calculated. Finally, the visible light interference coefficient is used to mark the fused icing probability to obtain the icing detection image processing result.

[0078] Specifically, step S40 in the method includes:

[0079] According to the pipeline pump icing monitoring data within the historical time, collect the sample infrared uniformity parameter set and the sample visible light uniformity parameter set, and collect the proportion of the number of times of icing of the pipeline pump under different sample infrared uniformity parameters and sample visible light uniformity parameters, and label to obtain the sample icing probability set;

[0080] Based on a multi-layer feedforward neural network, construct an icing predictor, wherein the icing predictor includes an infrared icing prediction branch and a visible light icing prediction branch;

[0081] Respectively use the sample infrared uniformity parameter set and the sample visible light uniformity parameter set as input data, and use the sample icing probability set as output data, and respectively perform network parameter training and optimization on the infrared icing prediction branch and the visible light icing prediction branch until convergence;

[0082] Input the infrared uniformity parameter and the visible light uniformity parameter into the infrared icing prediction branch and the visible light icing prediction branch respectively, and predict and output to obtain the infrared icing probability and the visible light icing probability.

[0083] In the embodiment of the present application, first, according to the pipeline pump icing monitoring data within the historical time, collect the sample infrared uniformity parameter set and the sample visible light uniformity parameter set, and then count the proportion of the number of times of icing of the pipeline pump under different sample infrared uniformity parameters and sample visible light uniformity parameters, and label to obtain the sample icing probability set, thereby establishing a mapping relationship between different infrared uniformity parameters and sample visible light uniformity parameters and the icing probability. Exemplarily, in the historical pipeline pump icing monitoring data, collect the infrared uniformity parameter set and the visible light uniformity parameter set corresponding to 1000 different timestamps, and then respectively count the number of times of icing under different infrared uniformity parameters and visible light uniformity parameters. For example, the number of times of icing when the infrared uniformity parameter is 16.1 is 50 times, and the number of times of icing when the visible light uniformity parameter is 13.5 is 500 times, then the corresponding icing probabilities are 0.05 and 0.5 respectively, and then label to obtain the icing probability set of different infrared uniformity parameters and visible light uniformity parameters.

[0084] Secondly, based on a multi-layer feedforward neural network, construct an icing predictor, wherein the icing predictor includes an infrared icing prediction branch and a visible light icing prediction branch. Exemplarily, a two-branch feedforward neural network is used to perform infrared icing prediction and visible light icing prediction respectively. The infrared icing prediction branch adopts a three-layer fully connected network (128-64-32 nodes), and realizes efficient feature extraction through the PReLU activation function and the batch normalization layer; the visible light icing prediction branch is a lightweight structure (96-48-24 nodes).

[0085] Again, using the sample infrared uniformity parameter set and the sample visible light uniformity parameter set as input data respectively, and the sample icing probability set as output data, the network parameters of the infrared icing prediction branch and the visible light icing prediction branch are trained and optimized until convergence. Among them, the model training can be achieved through the following technical path: using the Adaptive Moment Estimation optimizer (Adam), the loss function is a combination of mean squared error and KL divergence, which not only ensures the probability prediction accuracy but also enhances the classification certainty. During the training process, an early stopping mechanism (patience = 20) is introduced to prevent overfitting, and a cosine annealing strategy for the learning rate (initial lr = 0.001) is adopted to improve the convergence efficiency.

[0086] Finally, input the infrared uniformity parameter and the visible light uniformity parameter into the infrared icing prediction branch and the visible light icing prediction branch respectively, and predict the output to obtain the infrared icing probability and the visible light icing probability. Exemplarily, input the infrared uniformity parameter (16.1) and the visible light uniformity parameter (13.5) into the infrared icing prediction branch and the visible light icing prediction branch of the icing predictor respectively, and output the infrared icing probability (0.5) and the visible light icing probability (0.6).

[0087] Further, the "processing and annotating using the visible light interference coefficient to obtain the processed result of the icing detection image" includes:

[0088] Calculating the average visible light interference coefficient according to the icing detection data within the historical time;

[0089] Calculating the ratio of the average visible light interference coefficient to the visible light interference coefficient, correcting and calculating the preset visible light weight to obtain the visible light weight, and calculating and obtaining the infrared weight;

[0090] Using the visible light weight and the infrared weight, calculating the weighted sum of the visible light icing probability and the infrared icing probability to obtain the fused icing probability; using the visible light interference coefficient to annotate the fused icing probability to obtain the processed result of the icing detection image.

[0091] In the embodiment of the present application, first, the average visible light interference coefficient is calculated according to the icing detection data within the historical time. Among them, the average visible light interference coefficient is the mean of the visible light interference coefficients in multiple previous detections.

[0092] Secondly, by calculating the ratio of the average visible light interference coefficient to the visible light interference coefficient, the preset visible light weight is corrected and calculated to obtain the visible light weight, and the infrared weight is calculated (where the visible light weight + the infrared weight = 1). Specifically, the preset visible light weight is preset by those skilled in the art according to the actual situation (such as 0.5), and the correction calculation is performed by multiplying the ratio of the average visible light interference coefficient to the visible light interference coefficient by the preset visible light weight. Therefore, the greater the current visible light interference coefficient, the smaller the corrected visible light weight, and the relatively larger the infrared weight, thereby reducing the influence of the reflection situation on the visible light icing prediction and further improving the accuracy of the icing detection. Exemplarily, the preset visible light weight is 0.5, the average visible light interference coefficient is 0.3, and the current visible light interference coefficient is 0.5. Through the correction calculation: , the obtained visible light weight is 0.3, and the infrared weight is 0.7. Further, in the case of reflection interference, the current visible light interference coefficient is 0.9. Through the correction calculation: , the obtained visible light weight is 0.17, and the infrared weight is 0.83. This is because due to the reflection interference, the accuracy of predicting the icing probability of the visible light image is low, so the weight is small.

[0093] Finally, the visible light weight and the infrared weight are used to perform weighted calculation on the visible light icing probability and the infrared icing probability to obtain the fused icing probability; then the visible light interference coefficient is used to label the fused icing probability to obtain the icing detection image processing result. Exemplarily, after the correction calculation, the obtained visible light weight is 0.3, the infrared weight is 0.7, the visible light icing probability is 0.8, and the infrared icing probability is 0.6. Therefore, the fused icing probability is obtained through the weighted calculation: . Finally, the fused icing probability is labeled by the visible light interference coefficient to obtain the icing detection image processing result.

[0094] In summary, based on the icing predictor, this application inputs the infrared uniformity parameter and the visible light uniformity parameter, and outputs the infrared icing probability and the visible light icing probability. Then, according to the ratio of the average visible light interference coefficient to the current visible light interference coefficient, the preset visible light weight is dynamically corrected and calculated to obtain the visible light weight, and thus the fused icing probability is calculated. Finally, the visible light interference coefficient is used to label the fused icing probability to obtain the icing detection image processing result. In this way, the adverse influence of the reflection situation on the icing detection result is reduced, and the accuracy and robustness of the icing detection result are improved.

[0095] In summary, the embodiments of this application at least have the following technical effects:

[0096] Compared with the prior art, the present application collects visible light images and infrared images on the surface of the pipeline pump, fuses the optical characteristics and thermodynamic characteristics on the surface of the pipeline pump, and cooperates with the timing analysis to provide effective data support for ice detection.

[0097] Secondly, based on the decrease in the standard deviation of the gray values of the infrared gray image and the visible light gray image of the pipeline pump after icing, according to the gray infrared image, calculate the standard deviation of the gray values of all pixel points in the image to obtain the infrared uniformity parameter, and then according to the gray visible light image, calculate the standard deviation of the gray values of all pixel points in the image to obtain the visible light uniformity parameter. In this way, the ice formation situation of the pipeline pump is intuitively reflected by the quantified value, and effective support is provided for subsequent ice detection.

[0098] Thirdly, a multi-layer feedforward neural network is adopted to construct a visible light uniformity predictor. By inputting the real-time time, the predicted visible light uniformity parameter is output. Then, according to the predicted visible light uniformity parameter and the actually detected visible light uniformity parameter, the visible light interference coefficient is calculated to reflect the probability of the current visible light uniformity parameter being affected by reflection. In this way, the misjudgment caused by the influence of reflection is reduced.

[0099] Finally, based on the ice formation predictor, by inputting the infrared uniformity parameter and the visible light uniformity parameter, the infrared ice formation probability and the visible light ice formation probability are output. Then, according to the ratio of the average visible light interference coefficient to the current visible light interference coefficient, the preset visible light weight is dynamically corrected and calculated to obtain the visible light weight, and the fused ice formation probability is calculated therefrom. Finally, the visible light interference coefficient is used to label the fused ice formation probability to obtain the ice detection image processing result. In this way, the visible light weight and the infrared weight are dynamically adjusted to reduce the adverse influence of the reflection situation on the ice detection result, and the accuracy and robustness of the ice detection result are improved.

[0100] Through the above technical solutions, the present application collects visible light images and infrared images on the surface of the pipeline pump with timing characteristics, based on the significant difference in the standard deviation of the image gray values before and after icing, and fully considers the adverse influence of the reflection situation on the ice detection result. The visible light weight and the infrared weight are dynamically adjusted through the visible light interference coefficient to obtain the fused ice formation probability. In this way, the accuracy and robustness of ice detection are improved.

[0101] Example 2, as Figure 2 shown, based on the same inventive concept as the method for processing images on the surface of a pipeline pump for ice detection provided in Example 1, the embodiment of the present invention further provides a system for processing images on the surface of a pipeline pump for ice detection, including:

[0102] The data acquisition module 11 is used to collect visible light images and infrared images of the surface of the pipeline pump through visible light devices and infrared devices, and perform preprocessing to obtain grayscale visible light images and grayscale infrared images, and sense the real-time acquisition time;

[0103] The uniformity analysis module 12 is used to perform infrared uniformity analysis based on the grayscale infrared image to obtain infrared uniformity parameters, and perform visible light uniformity analysis based on the grayscale visible light image to obtain visible light uniformity parameters;

[0104] The interference analysis module 13 is used to perform prediction of the visible light uniformity of the pipeline pump according to the real-time time to obtain predicted visible light uniformity parameters, and verify them with the visible light uniformity parameters to obtain a visible light interference coefficient;

[0105] The fusion output module 14 is used to perform icing probability prediction according to the infrared uniformity parameters and visible light uniformity parameters respectively to obtain an infrared icing probability and a visible light icing probability, and perform processing and annotation using the visible light interference coefficient to obtain an icing detection image processing result.

[0106] Among them, the data acquisition module 11 is specifically used for:

[0107] Collect visible light images and infrared images of the surface of the pipeline pump through visible light devices and infrared devices;

[0108] Crop and grayscale process the visible light image and the infrared image, retain the images in the pipeline pump area, and obtain grayscale visible light images and grayscale infrared images;

[0109] Sense the real-time acquisition time.

[0110] Among them, the uniformity analysis module 12 is specifically used for:

[0111] Perform infrared uniformity analysis calculation according to all the grayscale values in the grayscale infrared image to obtain infrared uniformity parameters;

[0112] Perform visible light uniformity analysis calculation according to all the grayscale values in the grayscale visible light image to obtain visible light uniformity parameters.

[0113] Furthermore, the "perform infrared uniformity analysis calculation according to all the grayscale values in the grayscale infrared image to obtain infrared uniformity parameters" includes:

[0114] Extract all the grayscale values of all the pixel points in the grayscale infrared image to obtain an infrared grayscale value matrix;

[0115] Calculate the standard deviation of all the infrared grayscale values in the infrared grayscale value matrix to obtain infrared uniformity parameters.

[0116] Among them, the interference analysis module 13 is specifically used for:

[0117] Collect a set of real-time times of samples according to the monitoring data of the pipeline pump in the unfrozen state within the historical time, and collect the visible light uniformity parameters of the visible light image of the pipeline pump under the influence of reflection at different real-time times of samples, and label to obtain a set of sample visible light uniformity parameters;

[0118] Use a multi-layer feedforward neural network to construct a visible light uniformity predictor;

[0119] Use the set of real-time times of samples and the set of sample visible light uniformity parameters to train and optimize the network parameters of the visible light uniformity predictor until convergence;

[0120] Input the real-time time into the visible light uniformity predictor, and predict and output the predicted visible light uniformity parameters;

[0121] Perform verification calculations based on the predicted visible light uniformity parameters and the visible light uniformity parameters to obtain the visible light interference coefficient.

[0122] Further, the "perform verification calculations based on the predicted visible light uniformity parameters and the visible light uniformity parameters to obtain the visible light interference coefficient" includes:

[0123] Calculate the absolute difference between the predicted visible light uniformity parameters and the visible light uniformity parameters;

[0124] Calculate the ratio of the absolute difference to the predicted visible light uniformity parameters, and use 1 minus this ratio to obtain the visible light interference coefficient.

[0125] Among them, the fusion output module 14 is specifically used for:

[0126] Collect a set of sample infrared uniformity parameters and a set of sample visible light uniformity parameters according to the pipeline pump icing monitoring data within the historical time, and collect the proportion of the number of times the pipeline pump freezes under different sample infrared uniformity parameters and sample visible light uniformity parameters, and label to obtain a set of sample icing probabilities;

[0127] Based on a multi-layer feedforward neural network, construct an icing predictor, where the icing predictor includes an infrared icing prediction branch and a visible light icing prediction branch;

[0128] Respectively use the set of sample infrared uniformity parameters and the set of sample visible light uniformity parameters as input data, and use the set of sample icing probabilities as output data to train and optimize the network parameters of the infrared icing prediction branch and the visible light icing prediction branch until convergence;

[0129] Input the infrared uniformity parameter and the visible light uniformity parameter into the infrared icing prediction branch and the visible light icing prediction branch respectively, and predict and output to obtain the infrared icing probability and the visible light icing probability.

[0130] Further, the step of "processing and labeling using the visible light interference coefficient to obtain the image processing result of icing detection" includes:

[0131] Calculate the average visible light interference coefficient according to the icing detection data within the historical time;

[0132] Calculate the ratio of the average visible light interference coefficient to the visible light interference coefficient, perform a correction calculation on the preset visible light weight to obtain the visible light weight, and calculate and obtain the infrared weight;

[0133] Use the visible light weight and the infrared weight to perform a weighted calculation on the visible light icing probability and the infrared icing probability to obtain the fused icing probability; use the visible light interference coefficient to label the fused icing probability to obtain the image processing result of icing detection.

[0134] In summary, the embodiments of the present application at least have the following technical effects:

[0135] Compared with the prior art, the present application collects visible light images and infrared images on the surface of the pipeline pump through the data acquisition module; the uniformity analysis module quantifies the standard deviation of the image gray values before and after icing to reflect the icing condition of the pipeline pump; the interference analysis module reflects the probability of the current visible light uniformity parameter being affected by reflection; the fusion output module dynamically adjusts the visible light weight and the infrared weight, reducing the adverse impact of the reflection situation on the icing detection result. In this way, the accuracy and robustness of icing detection are improved.

[0136] It should be noted that in the above embodiments, the descriptions of each embodiment have their own focuses. For the parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0138] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general purpose computers, special purpose computers, embedded computers or other programmable data processing devices to produce a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks

[0139] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including instruction means that implement the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks

[0141] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once they learn the basic inventive concept.

[0142] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An image processing method for the surface of a pipeline pump used for icing detection, characterized in that, The method includes: Collecting visible light images and infrared images of the surface of the pipeline pump through visible light devices and infrared devices, and performing preprocessing to obtain grayscale visible light images and grayscale infrared images, and sensing the real-time acquisition time; According to the grayscale infrared image, performing infrared uniformity analysis to obtain infrared uniformity parameters, and according to the grayscale visible light image, performing visible light uniformity analysis to obtain visible light uniformity parameters; According to the real-time time, performing prediction of the visible light uniformity of the pipeline pump to obtain predicted visible light uniformity parameters, and verifying with the visible light uniformity parameters to obtain a visible light interference coefficient, including: According to the monitoring data of the pipeline pump in the unfrozen state during the historical time, collecting a set of sample real-time times, and collecting the visible light uniformity parameters of the visible light images of the pipeline pump under the influence of reflection at different sample real-time times, and labeling to obtain a set of sample visible light uniformity parameters; Using a multi-layer feedforward neural network to construct a visible light uniformity predictor; Using the set of sample real-time times and the set of sample visible light uniformity parameters to train and optimize the network parameters of the visible light uniformity predictor until convergence; Inputting the real-time time into the visible light uniformity predictor, and predicting and outputting predicted visible light uniformity parameters; According to the predicted visible light uniformity parameters and the visible light uniformity parameters, performing verification calculation to obtain a visible light interference coefficient, including: According to the monitoring data of the pipeline pump in the unfrozen state during the historical time, collecting a set of sample real-time times, and collecting the visible light uniformity parameters of the visible light images of the pipeline pump under the influence of reflection at different sample real-time times, and labeling to obtain a set of sample visible light uniformity parameters; Using a multi-layer feedforward neural network to construct a visible light uniformity predictor; Using the set of sample real-time times and the set of sample visible light uniformity parameters to train and optimize the network parameters of the visible light uniformity predictor until convergence; Inputting the real-time time into the visible light uniformity predictor, and predicting and outputting predicted visible light uniformity parameters; According to the predicted visible light uniformity parameters and the visible light uniformity parameters, performing verification calculation to obtain a visible light interference coefficient; According to the infrared uniformity parameters and the visible light uniformity parameters, respectively performing icing probability prediction to obtain an infrared icing probability and a visible light icing probability, and using the visible light interference coefficient for processing and labeling to obtain an icing detection image processing result.

2. The method for processing an image of the surface of a pipeline pump for ice detection according to claim 1, wherein, Collecting visible light images and infrared images of the surface of the pipeline pump through visible light devices and infrared devices, and performing preprocessing to obtain grayscale visible light images and grayscale infrared images, and sensing the real-time acquisition time, including: Collecting visible light images and infrared images of the surface of the pipeline pump through visible light devices and infrared devices; Cropping and grayscale processing the visible light image and the infrared image, retaining the images of the pipeline pump area, to obtain grayscale visible light images and grayscale infrared images; Sensing the real-time acquisition time.

3. The method for processing the surface image of a pipeline pump for icing detection according to claim 1, characterized in that, According to the grayscale infrared image, performing infrared uniformity analysis to obtain infrared uniformity parameters, and according to the grayscale visible light image, performing visible light uniformity analysis to obtain visible light uniformity parameters, including: Based on all the gray values in the gray infrared image, infrared uniformity analysis and calculation are carried out to obtain infrared uniformity parameters; Based on all the gray values in the gray visible light image, visible light uniformity analysis and calculation are carried out to obtain visible light uniformity parameters.

4. The method for processing the image of the surface of a pipeline pump for ice detection according to claim 3, wherein, Based on all the gray values in the gray infrared image, infrared uniformity analysis and calculation are carried out to obtain infrared uniformity parameters, including: Extract all the gray values of all pixel points in the gray infrared image to obtain an infrared gray value matrix; Calculate the standard deviation of all the infrared gray values in the infrared gray value matrix to obtain infrared uniformity parameters.

5. The method for processing the image of the surface of the pipeline pump for icing detection according to claim 1, characterized in that, Based on the infrared uniformity parameters and visible light uniformity parameters, ice formation probability prediction is respectively carried out to obtain an infrared ice formation probability and a visible light ice formation probability, including: According to the pipeline pump ice formation monitoring data within historical time, collect a sample infrared uniformity parameter set and a sample visible light uniformity parameter set, and collect the proportion of the number of times of ice formation of the pipeline pump under different sample infrared uniformity parameters and sample visible light uniformity parameters, and label to obtain a sample ice formation probability set; Based on a multi-layer feedforward neural network, construct an ice formation predictor, wherein the ice formation predictor includes an infrared ice formation prediction branch and a visible light ice formation prediction branch; Respectively use the sample infrared uniformity parameter set and the sample visible light uniformity parameter set as input data, and use the sample ice formation probability set as output data to respectively train and optimize the network parameters of the infrared ice formation prediction branch and the visible light ice formation prediction branch until convergence; Input the infrared uniformity parameters and visible light uniformity parameters into the infrared ice formation prediction branch and the visible light ice formation prediction branch respectively, and predict and output to obtain an infrared ice formation probability and a visible light ice formation probability.

6. The method for processing the image of the pipeline pump surface for icing detection according to claim 1, wherein, Use the visible light interference coefficient for processing and annotation to obtain an ice formation detection image processing result, including: According to the ice formation detection data within historical time, calculate the average visible light interference coefficient; Calculate the ratio of the average visible light interference coefficient to the visible light interference coefficient, correct and calculate the preset visible light weight to obtain the visible light weight, and calculate and obtain the infrared weight; Use the visible light weight and the infrared weight to perform weighted calculation on the visible light ice formation probability and the infrared ice formation probability to obtain a fused ice formation probability; use the visible light interference coefficient to annotate the fused ice formation probability to obtain an ice formation detection image processing result.

7. A pipeline pump surface image processing system for icing detection, characterized in that, For implementing a pipeline pump surface image processing method for ice formation detection according to any one of claims 1-6, the system includes: A data acquisition module, configured to collect a visible light image and an infrared image of the pipeline pump surface through a visible light device and an infrared device, and perform preprocessing to obtain a gray visible light image and a gray infrared image, and sense and collect the real-time time; A uniformity analysis module, configured to perform infrared uniformity analysis based on the gray infrared image to obtain infrared uniformity parameters, and perform visible light uniformity analysis based on the gray visible light image to obtain visible light uniformity parameters; An interference analysis module, which is used to predict the visible light uniformity of the pipeline pump according to the real-time time, obtain the predicted visible light uniformity parameter, and verify it with the visible light uniformity parameter to obtain the visible light interference coefficient, including: According to the monitoring data of the pipeline pump in the unfrozen state during the historical time, collect the set of sample real-time times, and collect the visible light uniformity parameters of the visible light image of the pipeline pump under the influence of reflection at different sample real-time times, and label to obtain the set of sample visible light uniformity parameters; Use a multi-layer feedforward neural network to construct a visible light uniformity predictor; Use the set of sample real-time times and the set of sample visible light uniformity parameters to train and optimize the network parameters of the visible light uniformity predictor until convergence; Input the real-time time into the visible light uniformity predictor, and predict and output the predicted visible light uniformity parameter; According to the predicted visible light uniformity parameter and the visible light uniformity parameter, perform verification calculations to obtain the visible light interference coefficient, including: According to the monitoring data of the pipeline pump in the unfrozen state during the historical time, collect the set of sample real-time times, and collect the visible light uniformity parameters of the visible light image of the pipeline pump under the influence of reflection at different sample real-time times, and label to obtain the set of sample visible light uniformity parameters; Use a multi-layer feedforward neural network to construct a visible light uniformity predictor; Use the set of sample real-time times and the set of sample visible light uniformity parameters to train and optimize the network parameters of the visible light uniformity predictor until convergence; Input the real-time time into the visible light uniformity predictor, and predict and output the predicted visible light uniformity parameter; According to the predicted visible light uniformity parameter and the visible light uniformity parameter, perform verification calculations to obtain the visible light interference coefficient; A fusion output module, which is used to predict the icing probability according to the infrared uniformity parameter and the visible light uniformity parameter respectively, obtain the infrared icing probability and the visible light icing probability, and use the visible light interference coefficient for processing and labeling to obtain the processing result of the icing detection image.

Images