Pipeline pump surface image processing method and system for icing detection

Abstract

The invention provides a pipeline pump surface image processing method and system for icing detection. The method comprises the following steps: acquiring a visible light image and an infrared image of the surface of the pipeline pump through visible light equipment and infrared equipment, preprocessing the visible light image and the infrared image to obtain a gray visible light image and a gray infrared image, and acquiring real-time time; performing infrared uniformity analysis according to the gray infrared image to obtain an infrared uniformity parameter, and performing visible light uniformity analysis according to the gray visible light image to obtain a visible light uniformity parameter; according to the real-time time, the visible light uniformity of the pipeline pump is predicted, and a visible light interference coefficient is obtained; and according to the infrared uniformity parameter and the visible light uniformity parameter, icing probability prediction is carried out to obtain an infrared icing probability and a visible light icing probability, and a visible light interference coefficient is adopted to carry out processing and marking to obtain a processing result. The technical problems that in the prior art, the freezing detection accuracy of the pipeline pump is low, and the environmental adaptability is poor are solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a pipeline pump surface image processing method and system for ice detection. Background Art

[0002] Freezing of pipeline pumps can cause equipment failures and safety hazards. In the existing technology, the detection of icing of pipeline pumps is mainly carried out through manual inspections or single sensor detection (such as only using temperature sensors or visible light imaging detection). However, manual inspections are inefficient and single sensor detection is easily affected by environmental interference, resulting in low accuracy and poor environmental adaptability of the icing detection method. Summary of the invention

[0003] The present invention aims to solve the technical problems of low accuracy and poor environmental adaptability of pipeline pump ice detection in the prior art and provides a pipeline pump surface image processing method and system for ice detection.

[0004] The technical solution of the present invention to solve the above technical problems is as follows: In a first aspect, the present invention provides a pipeline pump surface image processing method for ice detection, comprising: Visible light images and infrared images of the pipeline pump surface are collected through visible light equipment and infrared equipment, and preprocessed to obtain grayscale visible light images and grayscale infrared images, as well as sense and collect real-time time; Performing an infrared uniformity analysis based on the grayscale infrared image to obtain an infrared uniformity parameter, and performing a visible light uniformity analysis based on the grayscale visible light image to obtain a visible light uniformity parameter; According to the real-time time, predict the visible light uniformity of the pipeline pump to obtain a predicted visible light uniformity parameter, and verify it with the visible light uniformity parameter to obtain a visible light interference coefficient; According to the infrared uniformity parameter and the visible light uniformity parameter, icing probability prediction is performed respectively to obtain infrared icing probability and visible light icing probability, and the visible light interference coefficient is used for processing and annotation to obtain icing detection image processing results.

[0005] In a second aspect, the present invention provides a pipeline pump surface image processing system for ice detection, comprising: The data acquisition module is used to collect visible light images and infrared images of the pipeline pump surface through visible light equipment and infrared equipment, and perform preprocessing to obtain grayscale visible light images and grayscale infrared images, as well as sense and collect real-time time; a uniformity analysis module, configured to perform infrared uniformity analysis according to the grayscale infrared image to obtain infrared uniformity parameters, and to perform visible light uniformity analysis according to the grayscale visible light image to obtain visible light uniformity parameters; An interference analysis module, used to predict the visible light uniformity of the pipeline pump according to the real-time time, obtain a predicted visible light uniformity parameter, and verify it with the visible light uniformity parameter to obtain a visible light interference coefficient; The fusion output module is used to predict the icing probability according to the infrared uniformity parameter and the visible light uniformity parameter, obtain the infrared icing probability and the visible light icing probability, and use the visible light interference coefficient for processing and annotation to obtain the icing detection image processing result.

[0006] The beneficial effects of the present invention are: This application first provides reliable multimodal data support for ice detection by collecting visible light images and infrared images of the pipeline pump surface with time series characteristics; then calculates the standard deviation of the image grayscale value, and intuitively reflects the ice condition of the pipeline pump through quantified numerical values; and fully considers the adverse effect of reflection on the ice detection results, and obtains the visible light interference coefficient through calculation to reflect the probability that the current visible light uniformity parameter is affected by reflection; finally, the visible light weight and infrared weight are dynamically adjusted through the visible light interference coefficient to obtain the fused ice probability, and output accurate ice detection results that are resistant to environmental interference.

[0007] Through the above technical solution, the present application collects visible light images and infrared images of the pipeline pump surface with time series characteristics, based on the significant difference in the standard deviation of the grayscale value of the image before and after icing, and fully considers the adverse effect of reflection on the icing detection result, and dynamically adjusts the visible light weight and infrared weight through the visible light interference coefficient to obtain the fused icing probability. In this way, the accuracy and robustness of icing detection are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic flow chart of a pipeline pump surface image processing method for ice detection provided by the present invention; Figure 2 A schematic structural diagram of a pipeline pump surface image processing system for ice detection provided by the present invention.

[0009] In the accompanying drawings, the components represented by the reference numerals are as follows: Data acquisition module 11, uniformity analysis module 12, interference analysis module 13, fusion output module 14. DETAILED DESCRIPTION

[0010] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0011] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0012] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0013] Embodiment 1, as Figure 1 As shown, an embodiment of the present invention provides a tool life assessment method combined with a tool log, comprising: S10: collect visible light images and infrared images of the surface of the pipeline pump through visible light equipment and infrared equipment, and perform preprocessing to obtain grayscale visible light images and grayscale infrared images, and sense the real-time collection time; In the prior art, ice detection of pipeline pumps is mainly carried out through manual inspection or single sensor detection, but manual inspection is inefficient and single sensor detection is extremely susceptible to environmental interference. For example, the visible light imaging method is extremely susceptible to interference from changes in light. As a result, traditional ice detection methods have delayed response, poor environmental adaptability, and a high false alarm rate.

[0014] To address the above problems, the present application collects visible light images (reflecting the surface optical properties) and infrared images (reflecting the temperature distribution) and records the timestamp of the collection moment, and then performs preprocessing to obtain grayscale visible light images and grayscale infrared images with time series characteristics, which serve as a reliable basis for ice detection.

[0015] Specifically, step S10 in the method includes: Collect visible light images and infrared images of the pipeline pump surface through visible light equipment and infrared equipment; The visible light image and the infrared image are cropped and gray-scaled, the image of the pipeline pump area is retained, and a gray-scale visible light image and a gray-scale infrared image are obtained; Perception collection real time.

[0016] In the embodiment of the present application, firstly, the visible light image and infrared image of the pipeline pump surface are collected by visible light equipment and infrared equipment, and the timestamp of the collection time is recorded. Among them, the visible light image shows significant differences in optical properties before and after the pipeline pump is frozen. Specifically, when it is not frozen, the surface of the pump body mainly presents the gray and dark tone of the metal material. Due to the influence of the surface roughness, the reflection distribution is uneven and may be accompanied by rust or oil stains and other mottled patches; after freezing, the formation of the ice layer will greatly enhance the mirror reflection effect, resulting in the covered area showing a bright white feature, and the surface texture tends to be smooth. Experimental data show that the grayscale value of the frozen area is usually 30%-50% higher than that of the unfrozen area. The infrared image shows significant differences in thermal distribution characteristics before and after the pipeline pump is frozen. Specifically, when it is not frozen, the temperature distribution of the pump body is uneven due to the influence of running heat and environmental thermal radiation; after freezing, the phase change latent heat effect of the ice layer will make the surface temperature distribution tend to be uniform. Experimental data show that the standard deviation of the infrared image in the frozen area will drop by more than 40%, and form a temperature step greater than 5°C with the non-frozen area. Therefore, after the pipeline pump is frozen, the visible light image appears bright features and the infrared image tends to be uniform, forming complementary criteria and providing a reliable basis for ice detection.

[0017] For example, the visible light device uses a common optical camera to collect visible light images in the wavelength range of 380-780nm to reflect the optical reflection characteristics of the pipeline pump surface. The infrared device uses a thermal imaging camera to collect infrared radiation with a wavelength of 3-5μm or 8-14μm to reflect the temperature distribution characteristics of the object surface.

[0018] Secondly, the collected visible light images and infrared images are cropped and grayscale processed to obtain grayscale visible light images and grayscale infrared images. The cropping process is to cut off the redundant parts and only retain the image of the pipeline pump area, so as to highlight the key points and reduce the amount of information for image processing. The grayscale processing is to convert the visible light image and the infrared image into grayscale visible light images and grayscale infrared images. Specifically, converting the color visible light image into a black and white grayscale image can eliminate color interference, highlight the high reflectivity of the ice layer (specifically manifested as a bright white area), and reduce the data dimension to improve processing efficiency. The infrared image can be processed by grayscale processing methods such as adaptive histogram equalization to enhance the contrast of the ice area and improve the accuracy of feature recognition. In this way, the data format of the visible light image and the infrared image is unified, which is convenient for subsequent feature extraction and detection analysis.

[0019] Furthermore, the real-time time of the acquisition moment is recorded. The real-time time is the acquisition timestamp accurate to the minute when the visible light device and the infrared device acquire the visible light image and the infrared image of the pipeline pump surface, and the grayscale visible light image and the grayscale infrared image with time series characteristics are obtained by acquiring the real-time time. Exemplarily, the real-time acquisition time of a certain visible light image is: 2024-05-05 14:00.

[0020] In summary, compared with the existing technology, the present application collects visible light images and infrared images of the pipeline pump surface, integrates the optical characteristics and thermodynamic characteristics of the pipeline pump surface, and combines with time series analysis to provide reliable multimodal data support for subsequent pipeline pump icing detection.

[0021] S20: performing infrared uniformity analysis according to the grayscale infrared image to obtain infrared uniformity parameters, and performing visible light uniformity analysis according to the grayscale visible light image to obtain visible light uniformity parameters; Icing will significantly change the thermodynamic and optical characteristics of the outer surface of the pipeline pump: after freezing, the overall temperature distribution on the outside of the pipeline pump tends to be uniform, which reduces the grayscale value discreteness of the infrared grayscale image (that is, the standard deviation decreases); and most of the outer area of ​​the pipeline pump is wrapped in ice. The high reflective properties of the ice layer will make its surface appear uniformly bright white, which reduces the grayscale value discreteness of the visible light grayscale image (that is, the standard deviation decreases).

[0022] Therefore, based on the obvious difference in the standard deviation of the grayscale values ​​of the collected grayscale infrared images and grayscale visible light images before and after ice formation, the present application calculates all the grayscale values ​​of all pixels in the grayscale infrared images and grayscale visible light images respectively, and then calculates their standard deviations to obtain infrared uniformity parameters and visible light uniformity parameters.

[0023] Specifically, step S20 in the method includes: Performing infrared uniformity analysis and calculation according to all grayscale values ​​in the grayscale infrared image to obtain infrared uniformity parameters; According to all grayscale values ​​in the grayscale visible light image, visible light uniformity analysis and calculation are performed to obtain visible light uniformity parameters.

[0024] In the embodiment of the present application, firstly, infrared uniformity analysis and calculation are performed based on all grayscale values ​​in the grayscale infrared image to obtain infrared uniformity parameters. The infrared uniformity parameters are the standard deviation of all grayscale values ​​of all pixels in the grayscale infrared image. After freezing, the overall temperature outside the pipeline pump is relatively consistent, so the standard deviation is small.

[0025] Secondly, according to all the gray values ​​in the grayscale visible light image, a visible light uniformity analysis calculation is performed to obtain a visible light uniformity parameter. The visible light uniformity parameter is the standard deviation of all the gray values ​​of all the pixels in the grayscale visible light image. After freezing, most of the outside of the pipeline pump is the transparent white of ice, so the standard deviation is small.

[0026] Furthermore, the “performing infrared uniformity analysis calculation according to all grayscale values ​​in the grayscale infrared image to obtain infrared uniformity parameters” includes: Extracting all grayscale values ​​of all pixels in the grayscale infrared image to obtain an infrared grayscale value matrix; The standard deviation of all infrared gray values ​​in the infrared gray value matrix is ​​calculated to obtain infrared uniformity parameters.

[0027] In the embodiment of the present application, first, all gray values ​​of all pixels in the grayscale infrared image are extracted to obtain an infrared grayscale value matrix. Exemplarily, based on the preprocessed grayscale infrared image, automated extraction is achieved through a scientific computing library (such as Python's OpenCV or NumPy): an image processing interface is called to convert the grayscale infrared image into a two-dimensional array matrix, in which each matrix element image[y,x] accurately corresponds to the pixel grayscale value at the image coordinate (x,y), thereby obtaining the infrared grayscale value matrix of the grayscale infrared image.

[0028] Secondly, the infrared uniformity parameter is obtained by calculating the standard deviation of all infrared gray values ​​in the infrared gray value matrix. Specifically, the standard deviation is calculated by the following formula: ; Where N is the total number of pixels, x i is the gray value of the i-th element in the matrix, and u is the arithmetic mean of the matrix (i.e., the average gray value of the grayscale infrared image).

[0029] Exemplarily, the grayscale value of a grayscale infrared image with a resolution of 640×480 is extracted through Python's OpenCV, and 311040 pixels are traversed to obtain a 640×480 infrared grayscale value matrix, and then its standard deviation is calculated as the infrared uniformity parameter.

[0030] In summary, compared with the prior art, this application is based on the reduction of the standard deviation of the grayscale values ​​of the infrared grayscale image and the visible light grayscale image of the pipeline pump after icing. First, according to the grayscale infrared image, the standard deviation of the grayscale values ​​of all pixels in the image is calculated to obtain the infrared uniformity parameter, and then according to the grayscale visible light image, the standard deviation of the grayscale values ​​of all pixels in the image is calculated to obtain the visible light uniformity parameter. The quantified value intuitively reflects the icing condition of the pipeline pump and provides effective support for subsequent icing detection.

[0031] S30: predicting the visible light uniformity of the pipeline pump according to the real-time time, obtaining a predicted visible light uniformity parameter, and verifying the predicted visible light uniformity parameter with the visible light uniformity parameter to obtain a visible light interference coefficient; Although the above-mentioned visible light uniformity parameters can reflect the icing situation on the pipeline pump surface from the perspective of optical characteristics, they are easily affected by changes in the angle of incidence of the sun. In particular, when the pipeline pump surface is fully reflected under strong reflection conditions, the obtained grayscale visible light image will show a highly consistent grayscale value distribution, which is similar to the image characteristics under the icing state, thus leading to misjudgment. For example, at noon in summer, when the sun shines directly on the pipeline pump, its metal surface is fully reflective, and the grayscale value of the obtained grayscale visible light image is very uniform, but in fact it is not frozen.

[0032] Therefore, in response to the above problems, the present application considers the influence of reflection on the visible light uniformity parameters, constructs a visible light uniformity predictor, inputs real-time time, outputs predicted visible light uniformity parameters, and then analyzes the similarity between the predicted visible light uniformity parameters and the visible light uniformity parameters to obtain the visible light interference coefficient. The larger the visible light interference coefficient, the greater the probability that the current visible light uniformity parameters are affected by reflection, thereby reducing misjudgment caused by the influence of reflection.

[0033] Specifically, step S30 in the method includes: According to the monitoring data of the pipeline pump in the unfrozen state in the historical time, the sample real-time time set is collected, and the visible light uniformity parameters of the visible light image of the pipeline pump under the influence of reflection at different sample real-time times are collected, and the sample visible light uniformity parameter set is obtained by annotation; A multi-layer feed-forward neural network is used to construct a visible light uniformity predictor; Using the sample real-time time set and the sample visible light uniformity parameter set, perform network parameter training optimization on the visible light uniformity predictor until convergence; Inputting the real time into the visible light uniformity predictor, and predicting and outputting a predicted visible light uniformity parameter; A verification calculation is performed according to the predicted visible light uniformity parameter and the visible light uniformity parameter to obtain a visible light interference coefficient.

[0034] In the embodiment of the present application, first, a sample real-time time set of different time periods (accurate to minutes) when the pipeline pump is not frozen is collected, and the visible light uniformity parameters of the visible light images under the influence of reflection at different real-time times are collected, and the sample visible light uniformity parameter set is obtained by annotation. For example, a pipeline pump is directly exposed to the sun at noon in summer (11:00 to 14:00), and reflection occurs on its surface. Therefore, the sample real-time time set is continuously collected at intervals of 15 minutes, and then the visible light uniformity parameters of the visible light images of the pipeline pump under the sample real-time are collected, and the set of real-time and visible light uniformity parameters corresponding to each other is obtained by annotation.

[0035] Secondly, a multi-layer feedforward neural network is used to construct a visible light uniformity predictor. Exemplarily, the visible light uniformity predictor is constructed by a three-layer fully connected neural network (MLP), the input layer receives a normalized time feature vector (including year, month, day, hour, and minute, such as 2024-05-05 14:00), the hidden layer adopts a progressive design of 64-32 nodes, and each layer is configured with a ReLU activation function and Dropout (p=0.2) regularization. The output layer predicts the output of the predicted visible light uniformity parameters.

[0036] Again, the sample real-time time set and the sample visible light uniformity parameter set are used to perform network parameter training and optimization on the visible light uniformity predictor until convergence. Exemplarily, the sample real-time time set and the sample visible light uniformity parameter set are divided into training set, validation set and test set in a ratio of 7:1.5:1.5. After continuous training and optimization and qualified testing, a converged (convergence condition can be set to accuracy>95%) visible light uniformity predictor is obtained. Furthermore, the model training process can be implemented through the following technical paths: using mean square error loss function and L2 regularization, preventing overfitting through early stopping method, and dynamically adjusting the learning rate. The model convergence criterion is test set MSE<0.5, corresponding to a prediction error less than 0.7 gray levels, to ensure that it can accurately predict the output predicted visible light uniformity parameters.

[0037] Finally, the real time is input into the visible light uniformity predictor, and the predicted visible light uniformity parameter is output. Exemplarily, the real time 2024-05-05 14:00 is input into the visible light uniformity predictor, and the predicted visible light uniformity parameter is output as 13.8.

[0038] Furthermore, a verification calculation is performed based on the predicted visible light uniformity parameter and the visible light uniformity parameter to obtain a visible light interference coefficient. The visible light interference coefficient reflects the degree to which the visible light uniformity parameter is affected by reflections. The larger the visible light interference coefficient, the greater the probability that the current visible light uniformity parameter is affected by reflections.

[0039] Specifically, the “performing verification calculation according to the predicted visible light uniformity parameter and the visible light uniformity parameter to obtain a visible light interference coefficient” includes: Calculating an absolute difference between the predicted visible light uniformity parameter and the visible light uniformity parameter; The ratio of the absolute difference to the predicted visible light uniformity parameter is calculated, and the ratio is subtracted from 1 to obtain a visible light interference coefficient.

[0040] In the embodiment of the present application, the absolute difference between the predicted visible light uniformity parameter and the visible light uniformity parameter is first calculated. The absolute difference reflects the similarity between the predicted visible light uniformity parameter and the visible light uniformity parameter. The smaller the absolute difference, the higher the probability of the existence of the reflection at the current moment. The larger the absolute difference, the lower the probability of the existence of the reflection at the current moment. For example, during the noon period on a sunny day, the predicted visible light uniformity parameter is 13.8, the actually detected visible light uniformity parameter is 13.5, and the absolute difference is 0.3; during the same period on a cloudy day, the predicted visible light uniformity parameter is still 13.8, the actually detected visible light uniformity parameter is 16.1, and the absolute difference is 2.3.

[0041] Secondly, the ratio of the absolute difference to the predicted visible light uniformity parameter is calculated, and the ratio is subtracted from 1 to obtain the visible light interference coefficient. The visible light interference coefficient reflects the probability that the current visible light uniformity parameter is affected by reflection. The larger the visible light interference coefficient, the greater the probability that the current visible light uniformity parameter is affected by reflection, and the smaller the visible light interference coefficient, the smaller the probability that the current visible light uniformity parameter is affected by reflection. For example, during the noon period on a sunny day, the predicted visible light uniformity parameter is 13.5, the actually detected visible light uniformity parameter is 13.8, the absolute difference is 0.3, and the visible light interference coefficient is 0.98; during the same period on a cloudy day, the predicted visible light uniformity parameter is still 13.5, and the actually detected visible light uniformity parameter is 16.1. At this time, the absolute difference is 2.3, and the visible light interference coefficient is 0.83.

[0042] In summary, compared with the prior art, this application uses a multi-layer feedforward neural network to construct a visible light uniformity predictor, which outputs predicted visible light uniformity parameters by inputting real time, and then calculates the visible light interference coefficient based on the predicted visible light uniformity parameters and the actually detected visible light uniformity parameters, and uses this coefficient to reflect the probability that the current visible light uniformity parameters are affected by reflection. In this way, the misjudgment of results caused by reflection is reduced.

[0043] S40: performing icing probability prediction according to the infrared uniformity parameter and the visible light uniformity parameter, respectively, obtaining infrared icing probability and visible light icing probability, and performing processing and labeling using the visible light interference coefficient to obtain an icing detection image processing result.

[0044] When the icing condition of the pipeline pump is fused and predicted based on the aforementioned infrared uniformity parameters and visible light uniformity parameters, since the visible light uniformity parameters are easily affected by reflections, the visible light weight and infrared weight cannot remain unchanged during the prediction process. Specifically, when the visible light interference coefficient increases (indicating that the probability of being affected by reflections is high), the visible light weight should be reduced and the infrared weight should be increased to reduce the adverse impact of reflections on the icing prediction results.

[0045] In response to the above problems, this application first builds an icing predictor based on a multi-layer feedforward neural network, inputs infrared uniformity parameters and visible light uniformity parameters respectively, and outputs infrared icing probability and visible light icing probability. Then, according to the ratio of the average visible light interference coefficient and the current visible light interference coefficient, the preset visible light weight is corrected and calculated to obtain the visible light weight, thereby calculating the fused icing probability. Finally, the visible light interference coefficient is used to mark the fused icing probability to obtain the icing detection image processing result.

[0046] Specifically, step S40 in the method includes: According to the pipeline pump icing monitoring data in the historical period, the sample infrared uniformity parameter set and the sample visible light uniformity parameter set are collected, and the proportion of the number of pipeline pump icing under different sample infrared uniformity parameters and sample visible light uniformity parameters is collected, and the sample icing probability set is obtained by marking; Based on a multi-layer feedforward neural network, an icing predictor is constructed, wherein the icing predictor includes an infrared icing prediction branch and a visible light icing prediction branch; The sample infrared uniformity parameter set and the sample visible light uniformity parameter set are respectively used as input data, and the sample icing probability set is used as output data, and network parameter training and optimization are performed on the infrared icing prediction branch and the visible light icing prediction branch respectively until convergence; The infrared uniformity parameter and the visible light uniformity parameter are respectively input into the infrared icing prediction branch and the visible light icing prediction branch, and the infrared icing probability and the visible light icing probability are obtained by prediction output.

[0047] In the embodiment of the present application, firstly, based on the pipeline pump icing monitoring data in the historical time, the sample infrared uniformity parameter set and the sample visible light uniformity parameter set are collected, and then the proportion of the number of times the pipeline pump is frozen under different sample infrared uniformity parameters and sample visible light uniformity parameters is counted, and the sample icing probability set is obtained by marking, thereby establishing a mapping relationship between different infrared uniformity parameters and sample visible light uniformity parameters and icing probabilities. Exemplarily, in the historical pipeline pump icing monitoring data, the infrared uniformity parameter set and the visible light uniformity parameter set corresponding to 1000 different timestamps are collected, and then the number of times icing occurs under different infrared uniformity parameters and visible light uniformity parameters are counted respectively. For example, when the infrared uniformity parameter is 16.1, the number of times icing occurs is 50 times, and when the visible light uniformity parameter is 13.5, the number of times icing occurs is 500 times, and the corresponding icing probabilities are 0.05 and 0.5 respectively, and then the icing probability sets of different infrared uniformity parameters and visible light uniformity parameters are obtained by marking.

[0048] Secondly, an icing predictor is constructed based on a multi-layer feedforward neural network, wherein the icing predictor includes an infrared icing prediction branch and a visible light icing prediction branch. Exemplarily, a dual-branch feedforward neural network is used to perform infrared icing prediction and visible light icing prediction respectively. The infrared icing prediction branch adopts a three-layer fully connected network (128-64-32 nodes), and achieves efficient feature extraction through the PReLU activation function and batch normalization layer; the visible light icing prediction branch is a lightweight structure (96-48-24 nodes).

[0049] Again, the sample infrared uniformity parameter set and the sample visible light uniformity parameter set are used as input data, and the sample icing probability set is used as output data, and the network parameter training and optimization of the infrared icing prediction branch and the visible light icing prediction branch are performed respectively until convergence. Among them, model training can be achieved through the following technical paths: Adaptive moment estimation optimizer (Adam) is used, and the loss function is a combination of mean square error and KL divergence, which not only ensures the accuracy of probability prediction but also enhances classification certainty. An early stopping mechanism (patience=20) is introduced during training to prevent overfitting, and a learning rate cosine annealing strategy (initial lr=0.001) is used to improve convergence efficiency.

[0050] Finally, the infrared uniformity parameter and the visible light uniformity parameter are respectively input into the infrared icing prediction branch and the visible light icing prediction branch, and the prediction outputs are used to obtain the infrared icing probability and the visible light icing probability. For example, the infrared uniformity parameter (16.1) and the visible light uniformity parameter (13.5) are respectively input into the infrared icing prediction branch and the visible light icing prediction branch of the icing predictor, and the infrared icing probability (0.5) and the visible light icing probability (0.6) are output.

[0051] Furthermore, the “using the visible light interference coefficient for processing and marking to obtain an ice detection image processing result” includes: Calculate the average visible light interference coefficient based on the ice detection data in the historical period; Calculating the ratio of the average visible light interference coefficient to the visible light interference coefficient, performing correction calculation on the preset visible light weight to obtain the visible light weight, and calculating the infrared weight; The visible light weight and the infrared weight are used to perform weighted calculation on the visible light icing probability and the infrared icing probability to obtain a fused icing probability; the visible light interference coefficient is used to mark the fused icing probability to obtain an icing detection image processing result.

[0052] In the embodiment of the present application, firstly, the average visible light interference coefficient is calculated based on the ice detection data in the historical time, wherein the average visible light interference coefficient is the average value of the visible light interference coefficients in the previous multiple detections.

[0053] Secondly, by calculating the ratio of the average visible light interference coefficient and the visible light interference coefficient, the preset visible light weight is corrected and calculated to obtain the visible light weight, and the infrared weight is calculated to obtain (wherein, visible light weight + infrared weight = 1). Specifically, the preset visible light weight is pre-set by technicians in this field according to actual conditions (such as 0.5), and the correction calculation is performed by multiplying the ratio of the average visible light interference coefficient and the visible light interference coefficient by the preset visible light weight. Therefore, the larger the current visible light interference coefficient, the smaller the corrected visible light weight, and the larger the infrared weight, thereby reducing the impact of reflection conditions on visible light icing prediction, thereby improving the accuracy of icing detection. Exemplarily, the preset visible light weight is 0.5, the average visible light interference coefficient is 0.3, and the current visible light interference coefficient is 0.5. Through the correction calculation: , the visible light weight is 0.3, and the infrared weight is 0.7. Furthermore, in the case of reflection interference, the current visible light interference coefficient is 0.9, and the correction calculation is: , the visible light weight is 0.17, and the infrared weight is 0.83. This is because the accuracy of visible light image in predicting the probability of icing is low due to reflection interference, so the weight is small.

[0054] Finally, the visible light weight and infrared weight are used to perform weighted calculation on the visible light icing probability and the infrared icing probability to obtain the fused icing probability; then the visible light interference coefficient is used to mark the fused icing probability to obtain the icing detection image processing result. For example, after correction calculation, the visible light weight is 0.3, the infrared weight is 0.7, the visible light icing probability is 0.8, and the infrared icing probability is 0.6. Therefore, the fused icing probability is obtained by weighted calculation: Finally, the fused icing probability is annotated by the visible light interference coefficient to obtain the icing detection image processing result.

[0055] In summary, the present application is based on an icing predictor, which outputs infrared icing probability and visible light icing probability by inputting infrared uniformity parameters and visible light uniformity parameters. Then, according to the ratio of the average visible light interference coefficient and the current visible light interference coefficient, the preset visible light weight is dynamically corrected and calculated to obtain the visible light weight, thereby calculating the fused icing probability. Finally, the visible light interference coefficient is used to mark the fused icing probability to obtain the icing detection image processing result. In this way, the adverse effect of reflection on the icing detection result is reduced, and the accuracy and robustness of the icing detection result are improved.

[0056] In summary, the embodiments of the present application have at least the following technical effects: Compared with the existing technology, the present application collects visible light images and infrared images of the pipeline pump surface, integrates the optical characteristics and thermodynamic characteristics of the pipeline pump surface, and cooperates with time series analysis to provide effective data support for ice detection.

[0057] Secondly, according to the decrease in the standard deviation of the grayscale values ​​of the infrared grayscale image and the visible light grayscale image of the pipeline pump after freezing, the standard deviation of the grayscale values ​​of all pixels in the image is calculated based on the grayscale infrared image to obtain the infrared uniformity parameter, and then the standard deviation of the grayscale values ​​of all pixels in the image is calculated based on the grayscale visible light image to obtain the visible light uniformity parameter. In this way, the icing condition of the pipeline pump can be intuitively reflected through quantitative values, and effective support can be provided for subsequent icing detection.

[0058] Thirdly, a multi-layer feedforward neural network is used to construct a visible light uniformity predictor, which outputs the predicted visible light uniformity parameter by inputting the real time, and then calculates the visible light interference coefficient based on the predicted visible light uniformity parameter and the actually detected visible light uniformity parameter, which reflects the probability that the current visible light uniformity parameter is affected by reflection. In this way, misjudgment caused by reflection is reduced.

[0059] Finally, based on the icing predictor, by inputting the infrared uniformity parameter and the visible light uniformity parameter, the infrared icing probability and the visible light icing probability are output. Then, according to the ratio of the average visible light interference coefficient and the current visible light interference coefficient, the preset visible light weight is dynamically corrected and calculated to obtain the visible light weight, thereby calculating the fused icing probability. Finally, the visible light interference coefficient is used to mark the fused icing probability to obtain the icing detection image processing result. In this way, the visible light weight and the infrared weight are dynamically adjusted to reduce the adverse effect of the reflection on the icing detection result, and improve the accuracy and robustness of the icing detection result.

[0060] Through the above technical solution, the present application collects visible light images and infrared images of the pipeline pump surface with time series characteristics, based on the significant difference in the standard deviation of the grayscale value of the image before and after icing, and fully considers the adverse effect of reflection on the icing detection result, and dynamically adjusts the visible light weight and infrared weight through the visible light interference coefficient to obtain the fused icing probability. In this way, the accuracy and robustness of icing detection are improved.

[0061] Embodiment 2, as Figure 2 As shown, based on the same inventive concept as the pipeline pump surface image processing method for ice detection provided in the first embodiment, the embodiment of the present invention further provides a pipeline pump surface image processing system for ice detection, including: The data acquisition module 11 is used to collect visible light images and infrared images of the pipeline pump surface through visible light equipment and infrared equipment, and perform preprocessing to obtain grayscale visible light images and grayscale infrared images, and sense the real-time acquisition time; A uniformity analysis module 12, configured to perform infrared uniformity analysis based on the grayscale infrared image to obtain infrared uniformity parameters, and perform visible light uniformity analysis based on the grayscale visible light image to obtain visible light uniformity parameters; The interference analysis module 13 is used to predict the visible light uniformity of the pipeline pump according to the real-time time, obtain the predicted visible light uniformity parameter, verify it with the visible light uniformity parameter, and obtain the visible light interference coefficient; The fusion output module 14 is used to predict the icing probability according to the infrared uniformity parameter and the visible light uniformity parameter, obtain the infrared icing probability and the visible light icing probability, and use the visible light interference coefficient for processing and annotation to obtain the icing detection image processing result.

[0062] Wherein, the data acquisition module 11 is specifically used for: Collect visible light images and infrared images of the pipeline pump surface through visible light equipment and infrared equipment; The visible light image and the infrared image are cropped and gray-scaled, the image of the pipeline pump area is retained, and a gray-scale visible light image and a gray-scale infrared image are obtained; Perception collection real time.

[0063] Wherein, the uniformity analysis module 12 is specifically used for: Performing infrared uniformity analysis and calculation according to all grayscale values ​​in the grayscale infrared image to obtain infrared uniformity parameters; According to all grayscale values ​​in the grayscale visible light image, visible light uniformity analysis and calculation are performed to obtain visible light uniformity parameters.

[0064] Furthermore, the “performing infrared uniformity analysis calculation according to all grayscale values ​​in the grayscale infrared image to obtain infrared uniformity parameters” includes: Extracting all grayscale values ​​of all pixels in the grayscale infrared image to obtain an infrared grayscale value matrix; The standard deviation of all infrared gray values ​​in the infrared gray value matrix is ​​calculated to obtain infrared uniformity parameters.

[0065] Wherein, the interference analysis module 13 is specifically used for: According to the monitoring data of the pipeline pump in the unfrozen state in the historical time, the sample real-time time set is collected, and the visible light uniformity parameters of the visible light image of the pipeline pump under the influence of reflection at different sample real-time times are collected, and the sample visible light uniformity parameter set is obtained by annotation; A multi-layer feed-forward neural network is used to construct a visible light uniformity predictor; Using the sample real-time time set and the sample visible light uniformity parameter set, perform network parameter training optimization on the visible light uniformity predictor until convergence; Inputting the real time into the visible light uniformity predictor, and predicting and outputting a predicted visible light uniformity parameter; A verification calculation is performed according to the predicted visible light uniformity parameter and the visible light uniformity parameter to obtain a visible light interference coefficient.

[0066] Furthermore, the “performing verification calculation according to the predicted visible light uniformity parameter and the visible light uniformity parameter to obtain a visible light interference coefficient” includes: Calculating an absolute difference between the predicted visible light uniformity parameter and the visible light uniformity parameter; The ratio of the absolute difference to the predicted visible light uniformity parameter is calculated, and the ratio is subtracted from 1 to obtain a visible light interference coefficient.

[0067] The fusion output module 14 is specifically used for: According to the pipeline pump icing monitoring data in the historical period, the sample infrared uniformity parameter set and the sample visible light uniformity parameter set are collected, and the proportion of the number of pipeline pump icing under different sample infrared uniformity parameters and sample visible light uniformity parameters is collected, and the sample icing probability set is obtained by marking; Based on a multi-layer feedforward neural network, an icing predictor is constructed, wherein the icing predictor includes an infrared icing prediction branch and a visible light icing prediction branch; The sample infrared uniformity parameter set and the sample visible light uniformity parameter set are respectively used as input data, and the sample icing probability set is used as output data, and network parameter training and optimization are performed on the infrared icing prediction branch and the visible light icing prediction branch respectively until convergence; The infrared uniformity parameter and the visible light uniformity parameter are respectively input into the infrared icing prediction branch and the visible light icing prediction branch, and the infrared icing probability and the visible light icing probability are obtained by prediction output.

[0068] Furthermore, the “using the visible light interference coefficient for processing and marking to obtain an ice detection image processing result” includes: Calculate the average visible light interference coefficient based on the ice detection data in the historical period; Calculating the ratio of the average visible light interference coefficient to the visible light interference coefficient, performing correction calculation on the preset visible light weight to obtain the visible light weight, and calculating the infrared weight; The visible light weight and the infrared weight are used to perform weighted calculation on the visible light icing probability and the infrared icing probability to obtain a fused icing probability; the visible light interference coefficient is used to mark the fused icing probability to obtain an icing detection image processing result.

[0069] In summary, the embodiments of the present application have at least the following technical effects: Compared with the prior art, the present application collects visible light images and infrared images of the pipeline pump surface through the data acquisition module; quantifies the standard deviation of the grayscale value of the image before and after icing through the uniformity analysis module to reflect the icing condition of the pipeline pump; reflects the probability of the current visible light uniformity parameter being affected by reflection through the interference analysis module; and dynamically adjusts the visible light weight and infrared weight through the fusion output module, thereby reducing the adverse effect of reflection on the icing detection result. In this way, the accuracy and robustness of icing detection are improved.

[0070] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0071] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] Although preferred embodiments of the present invention have been described, additional changes and modifications may occur to these embodiments once those skilled in the art understand the basic inventive concepts.

[0076] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and variations.

Claims

1. A pipeline pump surface image processing method for ice detection, characterized in that: The method comprises: Visible light images and infrared images of the pipeline pump surface are collected through visible light equipment and infrared equipment, and preprocessed to obtain grayscale visible light images and grayscale infrared images, as well as sense and collect real-time time; Performing an infrared uniformity analysis based on the grayscale infrared image to obtain an infrared uniformity parameter, and performing a visible light uniformity analysis based on the grayscale visible light image to obtain a visible light uniformity parameter; According to the real-time time, predict the visible light uniformity of the pipeline pump to obtain a predicted visible light uniformity parameter, and verify it with the visible light uniformity parameter to obtain a visible light interference coefficient; According to the infrared uniformity parameter and the visible light uniformity parameter, icing probability prediction is performed respectively to obtain infrared icing probability and visible light icing probability, and the visible light interference coefficient is used for processing and annotation to obtain icing detection image processing results.

2. The pipeline pump surface image processing method for ice detection according to claim 1 is characterized in that: Visible light images and infrared images of the pipeline pump surface are collected through visible light equipment and infrared equipment, and preprocessed to obtain grayscale visible light images and grayscale infrared images, as well as sense the real-time collection time, including: Collect visible light images and infrared images of the pipeline pump surface through visible light equipment and infrared equipment; The visible light image and the infrared image are cropped and gray-scaled, the image of the pipeline pump area is retained, and a gray-scale visible light image and a gray-scale infrared image are obtained; Perception collection real time.

3. The pipeline pump surface image processing method for ice detection according to claim 1 is characterized in that: According to the grayscale infrared image, infrared uniformity analysis is performed to obtain infrared uniformity parameters, and according to the grayscale visible light image, visible light uniformity analysis is performed to obtain visible light uniformity parameters, including: Performing infrared uniformity analysis and calculation according to all grayscale values ​​in the grayscale infrared image to obtain infrared uniformity parameters; According to all grayscale values ​​in the grayscale visible light image, visible light uniformity analysis and calculation are performed to obtain visible light uniformity parameters.

4. The pipeline pump surface image processing method for ice detection according to claim 3 is characterized in that: According to all grayscale values ​​in the grayscale infrared image, infrared uniformity analysis and calculation are performed to obtain infrared uniformity parameters, including: Extracting all grayscale values ​​of all pixels in the grayscale infrared image to obtain an infrared grayscale value matrix; The standard deviation of all infrared gray values ​​in the infrared gray value matrix is ​​calculated to obtain infrared uniformity parameters.

5. The pipeline pump surface image processing method for ice detection according to claim 1, characterized in that: According to the real-time time, the visible light uniformity of the pipeline pump is predicted to obtain the predicted visible light uniformity parameter, which is verified with the visible light uniformity parameter to obtain the visible light interference coefficient, including: According to the monitoring data of the pipeline pump in the unfrozen state in the historical time, the sample real-time time set is collected, and the visible light uniformity parameters of the visible light image of the pipeline pump under the influence of reflection at different sample real-time times are collected, and the sample visible light uniformity parameter set is obtained by annotation; A multi-layer feed-forward neural network is used to construct a visible light uniformity predictor; Using the sample real-time time set and the sample visible light uniformity parameter set, perform network parameter training optimization on the visible light uniformity predictor until convergence; Inputting the real time into the visible light uniformity predictor, and predicting and outputting a predicted visible light uniformity parameter; A verification calculation is performed according to the predicted visible light uniformity parameter and the visible light uniformity parameter to obtain a visible light interference coefficient.

6. The pipeline pump surface image processing method for ice detection according to claim 5 is characterized in that: According to the predicted visible light uniformity parameter and the visible light uniformity parameter, a verification calculation is performed to obtain a visible light interference coefficient, including: Calculating an absolute difference between the predicted visible light uniformity parameter and the visible light uniformity parameter; The ratio of the absolute difference to the predicted visible light uniformity parameter is calculated, and the ratio is subtracted from 1 to obtain a visible light interference coefficient.

7. The pipeline pump surface image processing method for ice detection according to claim 1, characterized in that: According to the infrared uniformity parameter and the visible light uniformity parameter, icing probability prediction is performed respectively to obtain infrared icing probability and visible light icing probability, including: According to the pipeline pump icing monitoring data in the historical period, the sample infrared uniformity parameter set and the sample visible light uniformity parameter set are collected, and the proportion of the number of pipeline pump icing under different sample infrared uniformity parameters and sample visible light uniformity parameters is collected, and the sample icing probability set is obtained by marking; Based on a multi-layer feedforward neural network, an icing predictor is constructed, wherein the icing predictor includes an infrared icing prediction branch and a visible light icing prediction branch; The sample infrared uniformity parameter set and the sample visible light uniformity parameter set are respectively used as input data, and the sample icing probability set is used as output data, and network parameter training and optimization are performed on the infrared icing prediction branch and the visible light icing prediction branch respectively until convergence; The infrared uniformity parameter and the visible light uniformity parameter are respectively input into the infrared icing prediction branch and the visible light icing prediction branch, and the infrared icing probability and the visible light icing probability are obtained by prediction output.

8. The pipeline pump surface image processing method for ice detection according to claim 1, characterized in that: The visible light interference coefficient is used for processing and annotation to obtain ice detection image processing results, including: Calculate the average visible light interference coefficient based on the ice detection data in the historical period; Calculating the ratio of the average visible light interference coefficient to the visible light interference coefficient, performing correction calculation on the preset visible light weight to obtain the visible light weight, and calculating the infrared weight; The visible light weight and the infrared weight are used to perform weighted calculation on the visible light icing probability and the infrared icing probability to obtain a fused icing probability; the visible light interference coefficient is used to mark the fused icing probability to obtain an icing detection image processing result.

9. A pipeline pump surface image processing system for ice detection, characterized in that: A pipeline pump surface image processing method for ice detection according to any one of claims 1 to 8 is used to execute the system comprising: The data acquisition module is used to collect visible light images and infrared images of the pipeline pump surface through visible light equipment and infrared equipment, and perform preprocessing to obtain grayscale visible light images and grayscale infrared images, as well as sense and collect real-time time; a uniformity analysis module, configured to perform infrared uniformity analysis according to the grayscale infrared image to obtain infrared uniformity parameters, and to perform visible light uniformity analysis according to the grayscale visible light image to obtain visible light uniformity parameters; An interference analysis module, used to predict the visible light uniformity of the pipeline pump according to the real-time time, obtain a predicted visible light uniformity parameter, and verify it with the visible light uniformity parameter to obtain a visible light interference coefficient; The fusion output module is used to predict the icing probability according to the infrared uniformity parameter and the visible light uniformity parameter, obtain the infrared icing probability and the visible light icing probability, and use the visible light interference coefficient for processing and annotation to obtain the icing detection image processing result.

Images