Icing state detection method and system based on image processing

Abstract

The invention discloses an icing state detection method and system based on image processing, and relates to the field of image recognition, and the method comprises the steps: collecting visual detection data, and carrying out the preprocessing; extracting visual detection image features of the visual detection data by adopting a multi-scale convolutional network, and dynamically fusing the visual detection image features through a space attention mechanism to obtain fused features; obtaining the ice layer edge sub-pixel coordinates of the fusion features and calculating the ice layer edge spacing of the fusion features to obtain an actual physical thickness value; constructing an icing risk prediction model, calculating an ice layer growth rate through an optical flow method, and generating a risk index of an icing state; setting a risk threshold value, comparing the risk index with the risk threshold value, and judging the risk level of the icing state; and setting a three-level early warning signal, marking the early warning area of the icing state, and carrying out early warning notification. According to the invention, the fusion feature is generated through dynamic fusion of a space attention mechanism, and the comprehensive perception capability of icing state detection is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to an ice-covered state detection method and system based on image processing. Background Art

[0002] With the abnormal changes in global climate, transmission line icing monitoring technology has gradually become an important means to ensure the safe operation of the power grid. In recent years, image processing has been widely used in the field of icing monitoring. For example, ice images are pre-processed through OpenCV image processing, the contours of the line before and after icing are extracted, and the ice thickness is calculated. In addition, automatic detection of ice thickness is achieved by combining active and passive detection. However, existing methods still have some problems in practical applications.

[0003] Existing icing status detection methods mainly rely on a single image processing method, which is difficult to fully reflect the icing status of transmission lines in complex environments. For example, in severe weather conditions such as rain, snow, fog and haze, image quality is easily affected, resulting in low accuracy in ice edge detection. Secondly, there are limitations in feature extraction and fusion, making it difficult to fully utilize multi-source image feature information, thus affecting the comprehensive assessment of icing status. In addition, the ability to predict and warn risks is weak, and it is impossible to accurately assess the impact of icing on the safe operation of transmission lines, making it difficult to implement timely and effective early warning measures. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an icing status detection method based on image processing to solve the problem that it is difficult to fully reflect the icing status of the transmission line in a complex environment by mainly relying on a single image processing method.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an ice cover state detection method based on image processing, which includes collecting visual detection data and performing preprocessing; using a multi-scale convolutional network to extract visual detection image features of the visual detection data, and dynamically fusing the visual detection image features through a spatial attention mechanism to obtain a fusion feature; obtaining the sub-pixel coordinates of the ice layer edge of the fusion feature and calculating the ice layer edge spacing of the fusion feature to obtain the actual physical thickness value; constructing an ice cover risk prediction model, calculating the ice layer growth rate through an optical flow method, dynamically fusing the ice layer growth rate and the actual physical thickness value to generate a risk index of the ice cover state; setting a risk threshold, comparing the risk index and the risk threshold, and determining the risk level of the ice cover state; setting a three-level warning signal according to the risk level of the ice cover state, marking the warning area of the ice cover state and issuing a warning notification.

[0008] As a preferred solution of the ice cover state detection method based on image processing of the present invention, wherein: the visual detection data includes visible light image data and infrared radiation data;

[0009] The pre-processing includes defogging enhancement, non-uniformity correction and temperature calibration.

[0010] As a preferred solution of the ice cover state detection method based on image processing described in the present invention, wherein: the multi-scale convolutional network is used to extract visual detection image features of visual detection data, and the visual detection image features are dynamically fused through the spatial attention mechanism to obtain fused features. The specific steps are as follows:

[0011] A small-size convolution kernel is used to extract ice texture features from visible light image data;

[0012] A large-size convolution kernel is used to extract the temperature gradient characteristics of infrared radiation data;

[0013] Use the channel attention mechanism to perform channel expansion on the temperature gradient feature to obtain the temperature gradient expansion feature;

[0014] Use the dilated convolution kernel to perform channel expansion on the ice texture features to obtain the ice texture expansion features;

[0015] The ice texture extension features and temperature gradient extension features are aligned using bilinear interpolation.

[0016] Multiply the ice layer texture extension feature and the temperature gradient extension feature element by element to obtain an interactive feature map;

[0017] The interaction feature map is mean-pooled using the Sigmoid function to generate a spatial attention weight map;

[0018] The weights of the ice texture extension features and the temperature gradient extension features are adjusted by the confidence of the spatial attention weight map, and the aligned ice texture extension features and the temperature gradient extension features are weighted and summed according to the adjusted weights to obtain the fusion features.

[0019] As a preferred solution of the ice cover state detection method based on image processing described in the present invention, wherein: the sub-pixel coordinates of the ice layer edge of the fused feature are obtained and the distance between the ice layer edges of the fused feature is calculated to obtain the actual physical thickness value, the specific steps are as follows:

[0020] Use the Scharr gradient operator to obtain the horizontal and vertical gradients of the fusion features;

[0021] Calculate the gradient direction angle through the inverse tangent function;

[0022] Use the Pythagorean theorem to synthesize the horizontal gradient and the vertical gradient to obtain the gradient amplitude;

[0023] The gradient direction angle and gradient magnitude are stored as a gradient direction matrix and a gradient magnitude matrix;

[0024] Using the sub-pixel edge positioning algorithm, the ice edge candidate points in the gradient amplitude are screened, and spline interpolation is performed on the ice edge candidate points along the gradient direction angle to obtain the sub-pixel coordinates of the ice edge;

[0025] Based on the gradient direction angle, the ice edge candidate points are matched with the sub-pixel coordinates of the ice edge;

[0026] Calculating the ice edge distance between the paired ice edge candidate points;

[0027] The ice edge distance is converted into actual physical thickness value through geometric projection method.

[0028] As a preferred solution of the ice cover state detection method based on image processing described in the present invention, wherein: the ice cover risk prediction model is constructed, the ice layer growth rate is calculated by the optical flow method, the ice layer growth rate and the actual physical thickness value are dynamically integrated to generate the ice cover state risk index, and the specific steps are as follows:

[0029] The convolutional neural network (CNN) is used as the basic framework of the ice risk prediction model to build an ice risk prediction model.

[0030] Define the input layer, optical flow processing layer, feature fusion layer, and output layer of the ice risk prediction model;

[0031] The input layer receives the actual physical thickness value;

[0032] The optical flow processing layer uses the optical flow method to perform vector operations on multiple actual physical thickness values to generate the ice growth rate;

[0033] The feature fusion layer uses a weighted superposition algorithm to perform point-by-point weighted superposition of the actual physical thickness value and ice growth rate to generate a risk index for the ice cover state.

[0034] The output layer outputs the risk index of the icing state.

[0035] As a preferred embodiment of the ice cover state detection method based on image processing of the present invention, wherein: setting a risk threshold, comparing the risk index and the risk threshold, and determining the risk level of the ice cover state, the specific steps are:

[0036] Set risk thresholds, defining them as low, medium, and high risk thresholds;

[0037] When the risk index is at the low risk threshold, the risk level of the icing state is determined to be low risk;

[0038] When the risk index is at the medium risk threshold, the risk level of the icing state is determined to be medium risk;

[0039] When the risk index is at the high risk threshold, the risk level of the icing state is determined to be high risk.

[0040] As a preferred solution of the ice cover state detection method based on image processing of the present invention, wherein: the three-level warning signal is set according to the risk level of the ice cover state, the warning area of the ice cover state is marked and the warning notification is issued, the specific steps are:

[0041] Set three levels of warning signals according to the risk level of icing conditions;

[0042] Use OpenCV functions to mark high-risk areas with red rectangles, medium-risk areas with orange outlines, and low-risk areas with yellow text in the actual physical thickness values;

[0043] When the risk level is high, a red warning signal is triggered and an alarm SMS is sent to the power dispatch center via the MQTT protocol;

[0044] When the risk level is medium, an orange warning signal is triggered and an alarm message is pushed to the monitoring platform of the power base station via the long-distance wide area network LoRaWAN;

[0045] When the risk level is low, a yellow warning signal is triggered and recorded as a detection log file.

[0046] In a second aspect, the present invention provides an ice cover state detection system based on image processing, comprising:

[0047] Acquisition module, used to collect visual inspection data and perform preprocessing;

[0048] A fusion module is used to extract visual detection image features from visual detection data using a multi-scale convolutional network, and dynamically fuse the visual detection image features through a spatial attention mechanism to obtain fused features;

[0049] A calculation module is used to obtain the sub-pixel coordinates of the ice layer edge of the fused features and calculate the distance between the ice layer edges of the fused features to obtain the actual physical thickness;

[0050] The prediction module is used to build an icing risk prediction model, calculate the ice growth rate through the optical flow method, and dynamically integrate the ice growth rate and actual physical thickness value to generate an icing risk index;

[0051] The judgment module is used to set a risk threshold, compare the risk index with the risk threshold, and obtain the risk level of the icing state;

[0052] The processing module is used to set three-level warning signals according to the risk level of the icing state, mark the warning area of the icing state and issue warning notifications.

[0053] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the ice cover status detection method based on image processing as described in the first aspect of the present invention is implemented.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the ice cover status detection method based on image processing as described in the first aspect of the present invention is implemented.

[0055] The beneficial effects of the present invention are as follows: the present invention generates fusion features through dynamic fusion of the spatial attention mechanism, effectively improving the comprehensive perception ability of ice state detection, and adjusting the contribution weight of the ice texture features through the dynamic changes of the ice texture features in the actual scene, so as to perform ice state detection under complex environmental interference. When the ice edge is blocked by rain and snow, causing the ice texture to be blurred, the spatial attention weight is inclined to the infrared temperature gradient feature, thereby enhancing the ability to capture the cold and hot interface; in areas with uniform temperature distribution but obvious cracks on the ice surface, the spatial attention weight focuses on the analysis of ice texture details and accurately locates defects. Weight normalization ensures the physical consistency of the fusion features, provides accurate input information for subsequent sub-pixel ice edge positioning, and significantly improves the generalization ability and anti-interference performance of ice state detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 Flowchart of the ice cover status detection method based on image processing.

[0058] Figure 2 Flowchart of feature fusion in

[15] .

[0059] Figure 3 Flowchart for generating actual physical thickness values.

[0060] Figure 4Flowchart of early warning notification. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0063] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0064] Reference Figures 1 to 4 This embodiment provides an ice-covered state detection method based on image processing, comprising the following steps:

[0065] S1. Collect visual inspection data and perform preprocessing.

[0066] Visual detection data includes visible light image data and infrared radiation data;

[0067] Preprocessing includes dehazing enhancement, non-uniformity correction and temperature calibration;

[0068] Visible light visual detectors and infrared thermal imaging visual detectors are deployed at key nodes of transmission line towers (conductor suspension points and insulator string connection points). The image resolution of the visible light visual detector is set to 3840×2160 pixels and the frame rate is set to 30 frames per second. The temperature measurement range of the infrared thermal imaging visual detector is -40°C to +20°C, and the temperature accuracy is ±2°C at a detection distance of 10 meters.

[0069] Visible light image data of ice-covered areas of transmission lines are collected through a visible light visual detector;

[0070] Visible light image data includes ice texture, light transmission characteristics, and ice edge morphology;

[0071] The uncorrected infrared radiation data of the ice-covered area is collected by infrared thermal imaging visual detector;

[0072] Perform dehazing and enhancement processing on visible light image data, and convert the collected visible light image data from the standard RGB color space to the HSV color space through digital image processing technology;

[0073] The conversion process is to normalize the RGB channel values (0-255) and convert them to the range of 0-1. Based on the angle between the maximum and minimum values of the three RGB channels, the hue, saturation and lightness are calculated.

[0074] The purpose of the conversion is to separate the brightness components of ice texture and ice edge morphology in visible light image data for independent processing while preserving hue, saturation and brightness;

[0075] Using a multi-scale Gaussian kernel group, the brightness component (V channel) of the ice texture and ice edge morphology in the visible light image data is filtered and convolved to obtain three illumination component maps;

[0076] The Gaussian kernel group consists of large-scale kernel, medium-scale kernel, and small-scale kernel;

[0077] Capturing global illumination distribution through large-scale kernels (15×15 pixels);

[0078] The main structure of the image is extracted through the mid-scale kernel (7×7 pixels);

[0079] Ice texture that preserves surface details using a small-scale kernel (3×3 pixels);

[0080] Using subtraction operations, the illumination components are calculated, and the large-scale, medium-scale, and small-scale reflection maps are superimposed in a weight ratio of 6:3:1 to obtain the comprehensive defogging result;

[0081] Perform filter convolution operations on the V channel through large-scale, medium-scale, and small-scale Gaussian kernels to generate illumination component maps, and then subtract the illumination components from the original V channel to obtain large-scale, medium-scale, and small-scale reflection maps;

[0082] The reflectance image is the image component reflecting the ice texture and ice edge morphology obtained by subtracting the illumination component generated by multi-scale Gaussian filtering from the V channel of the ice texture and ice edge morphology. Based on the comprehensive defogging results, the average brightness value of the defogging visible light image data is calculated.

[0083] Dynamically adjust the gamma value based on the average brightness value to restore ice surface details under the interference of rain and snow. For example, when the average brightness value is greater than 150, set the gamma value to 0.6 to compress the highlight area to prevent overexposure and loss of ice texture. When the average brightness is ≤150, set the gamma value to 1.2 to enhance the ice surface details in the dark areas and highlight the ice edges and cracks.

[0084] Through the two-point calibration method, the infrared radiation data is corrected for non-uniformity to eliminate the noise of the infrared thermal imaging visual detector. For example, the blackbody radiation calibration temperature, the high temperature calibration point is 20 ° C, and the low temperature calibration point in the semiconductor refrigeration constant temperature environment is -5 ° C;

[0085] Temperature calibration refers to converting the infrared radiation data after non-uniformity correction into actual temperature values through linear mapping. The formula is:

[0086] Actual temperature value = corrected infrared radiation data × 0.04 - 273.15.

[0087] S2. A multi-scale convolutional network is used to extract visual detection image features from visual detection data, and the visual detection image features are dynamically fused through a spatial attention mechanism to obtain fused features.

[0088] Visual detection image features include ice texture features and temperature gradient features;

[0089] A small-size convolution kernel is used to extract ice texture features from visible light image data;

[0090] Use a small-size square convolution kernel group (such as a 3×3 convolution kernel group) to scan the visible light image data, capture the texture features of the ice layer (such as ice crystals and fine cracks), and generate a visible light feature map containing 64 texture patterns while maintaining the resolution of the visible light feature map (3840×2160 pixels).

[0091] Each texture pattern corresponds to an ice texture feature channel. For example, the first 32 channels correspond to ice crystal arrangement patterns in different directions, and the last 32 channels identify the defect morphology of cracks and bubbles.

[0092] A large-size convolution kernel is used to extract the temperature gradient characteristics of infrared radiation data;

[0093] A large square convolution kernel (7×7) is used to scan infrared radiation data, identify temperature gradient abrupt changes (such as the hot and cold interface between ice and air), capture temperature gradient characteristics, and generate an infrared signature map containing 64 temperature distribution patterns at a resolution adapted to the sensor specifications (640×512 pixels).

[0094] Each channel of the temperature gradient feature represents the temperature boundary in different directions. For example, channel 8 corresponds to the north-south hot and cold dividing line, and channel 21 detects the annular temperature anomaly area;

[0095] Through the channel attention mechanism, the temperature gradient feature is channel-expanded (dimension: 3840×2160×128) to obtain the temperature gradient expansion feature and quantify the gradient direction;

[0096] Use the dilated convolution kernel to expand the channels of the ice texture features (expanded to 128 channels), and obtain the ice texture expanded feature dimension: 3840×2160×128);

[0097] The ice layer texture extension features and temperature gradient extension features are aligned using bilinear interpolation method;

[0098] Feature alignment refers to resolution matching and spatial registration;

[0099] There is a spatial scale difference between the resolution of the visible light feature map (3840×2160) and the infrared feature map (640×512);

[0100] A hierarchical upsampling strategy is used to perform a 3x interpolation on the infrared feature map, and then adaptively scale it to match the visible light resolution, preserving edge sharpness, and using bilinear interpolation to calculate the newly added pixel values.

[0101] The ice texture characteristics and temperature gradient characteristics are weighted averaged according to the distance ratio of the four adjacent pixels;

[0102] Thin plate spline interpolation is used to maintain the continuity of temperature gradient characteristics and avoid mosaic effect;

[0103] Spatial registration is to establish the coordinate correspondence between the same physical location in the visible light image data and the infrared radiation data at key points (such as metal parts of insulators), and perform local affine transformation correction on areas with an offset of more than 2 pixels to ensure consistency in spatial position;

[0104] Multiply the ice texture extension feature and the temperature gradient extension feature after feature alignment element by element to obtain an interactive feature map;

[0105] The interaction feature map is mean-pooled using the Sigmoid function to generate a spatial attention weight map (dimension: 3840×2160×1);

[0106] The spatial attention weight is normalized by the Sigmoid function to a range of 0 to 1.

[0107] The weights of the ice texture extension feature and the temperature gradient extension feature are adjusted by the confidence of the spatial attention weight map, and the aligned ice texture extension feature (dimension: 3840×2160×128) and the temperature gradient extension feature (dimension: 3840×2160×128) are weighted summed according to the adjusted weights to obtain the fusion feature (dimension: 3840×2160×128). The formula is:

[0108] Fusion feature = ice texture extension feature × ice texture extension feature weight + temperature gradient extension feature × temperature gradient extension feature weight;

[0109] The fused features include the spatial distribution information of ice layer texture extension features and temperature gradient extension features.

[0110] S3. Obtain the sub-pixel coordinates of the ice layer edge of the fused features and calculate the ice layer edge spacing of the fused features to obtain the actual physical thickness value.

[0111] Use the semantic segmentation network to perform pixel-level classification on the fused features and generate a binary semantic mask map to distinguish ice areas from non-ice areas.

[0112] Use convolutional neural network (CNN) as the basic framework of semantic segmentation network to build a semantic segmentation network;

[0113] The semantic segmentation network includes input layer, extraction layer, classification layer and output layer;

[0114] The input layer receives the fused features;

[0115] The extraction layer uses depth-wise separable convolution (3×3 convolution kernel) to extract the semantic features of the fused features, and downsamples the extracted semantic features through pooling to obtain global semantic information;

[0116] The classification layer uses point convolution (1×1 convolution kernel) to compress the semantic information into the number of categories, such as ice layer and non-ice layer. The softmax function is used to generate the category probability of each pixel, and the argmax operation is used to convert the category probability into a discrete label (0 or 1).

[0117] The output layer outputs a binary semantic mask map, where pixels with a value of 1 represent ice areas and pixels with a value of 0 represent non-ice areas.

[0118] Extract the binary semantic mask of the ice layer area from the semantic mask map, and smooth the mask boundary and reduce noise through morphological operations (3×3 kernel dilation and erosion);

[0119] Multiply the ice layer binary mask and the fusion feature element by element to generate the ice layer area feature map;

[0120] The Scharr horizontal gradient operator is used to convolve the ice layer area feature map to obtain the horizontal gradient of the fused feature, which is used to reflect the intensity change of the ice edge in the left and right directions.

[0121] The Scharr vertical gradient operator is used to convolve the feature map of the same ice layer area to obtain the vertical gradient of the fused feature, which is used to capture the intensity difference in the upper and lower directions of the ice edge;

[0122] Through the inverse tangent function, the horizontal gradient and the vertical gradient are inversely tangented to calculate the gradient direction angle. The formula is:

[0123]

[0124] Among them, θ represents the gradient direction angle, G xIndicates the horizontal gradient value, G y Represents the vertical gradient value, arctan represents the inverse tangent function;

[0125] Using the Pythagorean theorem to synthesize the horizontal gradient and the vertical gradient, we can get the gradient amplitude. The formula is:

[0126]

[0127] Where M represents the gradient amplitude;

[0128] The gradient direction angle and gradient magnitude are stored as gradient direction matrix and gradient magnitude matrix, which are only valid within the ice layer semantic mask;

[0129] The ratio of the horizontal gradient to the vertical gradient is calculated using the four-quadrant inverse tangent function to obtain the gradient angle of each pixel and store it as a gradient direction matrix. The gradient direction angle represents the extension direction of the ice edge (e.g., 0° is a horizontal edge, 90° is a vertical edge, and the angle range is 0°-180° with an accuracy of 1°).

[0130] The gradient amplitude indicates the strength of the ice edge (the range of gradient amplitude is 0 to 255);

[0131] Using the sub-pixel edge positioning algorithm, the ice edge candidate points in the gradient amplitude are screened, and spline interpolation is performed on the ice edge candidate points along the gradient direction angle (the normal direction of the ice edge) to obtain the sub-pixel coordinates of the ice edge;

[0132] From the gradient magnitude matrix, pixels with gradient magnitudes greater than a gradient threshold within the ice layer semantic mask are extracted and marked as candidate ice edge points. The gradient threshold is dynamically adjusted based on the on-site environmental conditions (such as light intensity and haze level).

[0133] Filter out the ice edge candidate points in the non-iced area blocked by metal parts of the tower and vegetation, and only retain the valid ice edge candidate points on the surface of the conductors and insulators;

[0134] Perform cubic spline interpolation on each candidate point on the edge of the ice layer along the gradient direction angle (normal direction of the ice layer edge);

[0135] The cubic spline interpolation method is to search for 5 adjacent pixels (the current pixel and 2 pixels on both sides) with a step size of 0.1 pixel, fit the cubic spline curve, and use the maximum value of the gradient amplitude as the sub-pixel coordinate of the ice layer;

[0136] The Canny edge detection algorithm is used to extract the ice area boundary in the binary semantic mask image;

[0137] Using the ice region boundary to correct the sub-pixel edge coordinates, check whether the sub-pixel edge coordinates of each ice edge candidate point are located in the ice region or close to the ice region boundary;

[0138] When the ice edge candidate point is close to the boundary between the ice layer and the non-ice layer, it is considered that the ice edge candidate point has edge drift; the corrected ice edge candidate point is locally smoothed (such as weighted averaging) to ensure the continuity of the ice edge curve;

[0139] According to the gradient direction angle in the gradient direction matrix, the extension direction of the sub-pixel coordinates of the ice edge (the tangent direction of the ice edge) is determined, and the candidate ice edge points are paired along the gradient direction angle (the normal direction of the ice edge).

[0140] Calculating the ice edge distance between the paired ice edge candidate points;

[0141] The Moore-Neighbor algorithm is used to trace the connected edge points along the edge tangent direction to close the edge contour of the ice layer;

[0142] Calculate the vertical spacing of the paired ice edge candidate points and verify whether the vertical spacing is within the theoretical range of the wire diameter (such as ±10% error). Otherwise, it is considered a mismatch.

[0143] According to the gradient direction consistency of the gradient direction matrix, the upper and lower ice edge points or the left and right ice edge points of the same ice layer are paired, and the ice edge spacing is calculated, such as the spacing between the left edge point and the right edge point Δp = 15.6 pixels;

[0144] The ice edge distance is converted into actual physical thickness through geometric projection method. The formula is:

[0145] Actual physical thickness = (ice edge spacing × pixel size × object distance) ÷ focal length;

[0146] The unit is millimeter.

[0147] S4. Construct an icing risk prediction model, calculate the ice growth rate through the optical flow method, dynamically integrate the ice growth rate and the actual physical thickness value, and generate an icing risk index.

[0148] The convolutional neural network (CNN) is used as the basic framework of the ice risk prediction model to build an ice risk prediction model.

[0149] The ice risk prediction model is defined as an input layer, an optical flow processing layer, a feature fusion layer, and an output layer.

[0150] The input layer receives the actual physical thickness value;

[0151] The input layer sorts the actual physical thickness values into time series to form ice thickness variation data in the time dimension, which is used to generate ice growth rate.

[0152] The optical flow processing layer uses the optical flow method to perform vector operations on the actual physical thickness values of two consecutive frames to generate the ice growth rate;

[0153] The ice layer displacement vector matrix includes horizontal displacement and vertical displacement, such as horizontal displacement +0.5 pixel / frame and vertical displacement -0.2 pixel / frame;

[0154] The formula for ice growth rate is,

[0155]

[0156] Where Δx represents the horizontal displacement, Δy represents the vertical displacement, and Δt represents the time interval used for the horizontal and vertical displacements;

[0157] The unit of ice growth rate is mm / min;

[0158] The feature fusion layer uses a weighted superposition algorithm to perform point-by-point weighted superposition of the actual physical thickness value and ice growth rate to generate an ice cover risk index. The formula is:

[0159] Ice cover risk index = actual physical thickness × 0.6 + ice growth rate × 0.4;

[0160] The output layer outputs the risk index of the icing state.

[0161] S5. Set a risk threshold, compare the risk index with the risk threshold, and determine the risk level of the icing condition.

[0162] Based on the State Grid icing regulations, risk thresholds are set and defined as low, medium, and high risk thresholds;

[0163] When the risk index is at the low risk threshold, the risk level of the icing state is determined to be low risk;

[0164] When the risk index is at the medium risk threshold, the risk level of the icing state is determined to be medium risk;

[0165] When the risk index is at the high risk threshold, the risk level of the icing state is determined to be high risk;

[0166] For example, according to the State Grid icing specifications, when the risk index is <5, the risk index is in the low range, and the risk threshold is defined as the low risk threshold; when 5≤risk index<10, the risk index is in the middle range, and the risk threshold is defined as the low risk threshold; when the risk index is ≥10, the risk index is in the high range, and the risk threshold is defined as the high risk threshold.

[0167] S6. Set a three-level warning signal according to the risk level of the icing state, mark the warning area of the icing state and issue a warning notification.

[0168] Set three levels of warning signals according to the risk level of icing conditions;

[0169] The three-level warning signals refer to red warning signal, orange warning signal and yellow warning signal;

[0170] Use OpenCV functions to mark high-risk areas with red rectangles, medium-risk areas with orange outlines, and low-risk areas with yellow text in the ice thickness distribution map;

[0171] When the risk level is high, a red warning signal is triggered and an alarm SMS is sent to the power dispatch center via the MQTT protocol;

[0172] When the risk level is medium, an orange warning signal is triggered and an alarm message is pushed to the monitoring platform of the power base station via the long-distance wide area network LoRaWAN;

[0173] When the risk level is low, a yellow warning signal is triggered and recorded as a detection log file.

[0174] The high-risk level of icing is set as a red warning signal, with the color code RGB (255, 0, 0). When a red warning signal appears, the sound and light alarm is triggered and an alarm text message is sent to the dispatch center SCADA via the MQTT protocol (for example, the text message content is "high-risk icing alarm, ice thickness in area A is 18 mm, and ice growth rate is 0.08 mm / min").

[0175] The medium risk level of icing is an orange warning signal, colored as RGB (255, 165, 0). When an orange warning signal appears, an alarm message is pushed to the monitoring platform interface of the base station via the long-distance wide area network LoRaWAN (e.g., message content: Medium risk icing alarm: ice thickness in area B is 12 mm, ice growth rate is 0.05 mm / min);

[0176] The low-risk level of the icing state is a yellow warning signal, and the color is marked as RGB (255, 255, 0). When the yellow warning signal appears, the actual physical thickness value is recorded as a detection log file (such as log entry, low-risk icing: ice thickness in area C is 8 mm, and ice growth rate is 0.02 mm / min).

[0177] This embodiment further provides an ice cover state detection system based on image processing, comprising:

[0178] Acquisition module, used to collect visual inspection data and perform preprocessing;

[0179] A fusion module is used to extract visual detection image features from visual detection data using a multi-scale convolutional network, and dynamically fuse the visual detection image features through a spatial attention mechanism to obtain fused features;

[0180] A calculation module is used to obtain the sub-pixel coordinates of the ice layer edge of the fused features and calculate the distance between the ice layer edges of the fused features to obtain the actual physical thickness value;

[0181] The prediction module is used to build an icing risk prediction model, calculate the ice growth rate through the optical flow method, and dynamically integrate the ice growth rate and actual physical thickness value to generate an icing risk index;

[0182] The judgment module is used to set the risk threshold, compare the risk index with the risk threshold, and obtain the risk level of the icing state;

[0183] The processing module is used to set three-level warning signals according to the risk level of the icing state, mark the warning area of the icing state and issue warning notifications.

[0184] This embodiment also provides a computer device, which is suitable for the ice cover status detection method based on image processing, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the ice cover status detection method based on image processing proposed in the above embodiment.

[0185] The computer device may be a terminal, comprising a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0186] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the ice cover status detection method based on image processing as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0187] In summary, the present invention dynamically fuses and generates fusion features through the spatial attention mechanism, effectively improving the comprehensive perception ability of ice state detection. Through the dynamic changes of ice layer texture features in actual scenes, the contribution weight of ice layer texture features is adjusted, so that ice state detection can be performed under complex environmental interference. When the ice layer edge is obscured by rain and snow, the spatial attention weight is tilted towards the infrared temperature gradient feature, enhancing the ability to capture the cold and hot interface; in areas with uniform temperature distribution but obvious cracks on the ice surface, the spatial attention weight focuses on the analysis of ice layer texture details and accurately locates defects. Weight normalization ensures the physical consistency of the fusion features, provides accurate input information for subsequent sub-pixel ice edge positioning, and significantly improves the generalization ability and anti-interference performance of ice state detection.

[0188] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting ice coverage based on image processing, characterized in that: include, Collect visual inspection data and perform preprocessing; A multi-scale convolutional network is used to extract visual detection image features from visual detection data, and the visual detection image features are dynamically fused through a spatial attention mechanism to obtain fused features. Obtain the sub-pixel coordinates of the ice layer edge of the fused features and calculate the ice layer edge spacing of the fused features to obtain the actual physical thickness value; An icing risk prediction model was constructed, which used the optical flow method to calculate the ice growth rate. The ice growth rate and actual physical thickness values were dynamically integrated to generate an icing risk index. Set a risk threshold, compare the risk index with the risk threshold, and determine the risk level of the icing condition; Three-level warning signals are set according to the risk level of the icing status, the warning areas of the icing status are marked and warning notifications are issued.

2. The ice-covering state detection method based on image processing according to claim 1, characterized in that: The visual detection data includes visible light image data and infrared radiation data; The pre-processing includes defogging enhancement, non-uniformity correction and temperature calibration.

3. The ice-covering state detection method based on image processing according to claim 2, characterized in that: The multi-scale convolutional network is used to extract visual detection image features of visual detection data, and the visual detection image features are dynamically fused through the spatial attention mechanism to obtain fused features. The specific steps are: A small-size convolution kernel is used to extract ice texture features from visible light image data; A large-size convolution kernel is used to extract the temperature gradient characteristics of infrared radiation data; Use the channel attention mechanism to perform channel expansion on the temperature gradient feature to obtain the temperature gradient expansion feature; Use the dilated convolution kernel to perform channel expansion on the ice texture features to obtain the ice texture expansion features; The ice texture extension features and temperature gradient extension features are aligned using bilinear interpolation. Multiply the ice layer texture extension feature and the temperature gradient extension feature element by element to obtain an interactive feature map; The interaction feature map is mean-pooled using the Sigmoid function to generate a spatial attention weight map; The weights of the ice texture extension features and the temperature gradient extension features are adjusted by the confidence of the spatial attention weight map, and the aligned ice texture extension features and the temperature gradient extension features are weighted and summed according to the adjusted weights to obtain the fusion features.

4. The ice-covering state detection method based on image processing according to claim 3, characterized in that: The sub-pixel coordinates of the ice layer edge of the fused features are obtained and the distance between the ice layer edges of the fused features is calculated to obtain the actual physical thickness value. The specific steps are: Use the Scharr gradient operator to obtain the horizontal and vertical gradients of the fusion features; Calculate the gradient direction angle through the inverse tangent function; Use the Pythagorean theorem to synthesize the horizontal gradient and the vertical gradient to obtain the gradient amplitude; The gradient direction angle and gradient magnitude are stored as a gradient direction matrix and a gradient magnitude matrix; Using the sub-pixel edge positioning algorithm, the ice edge candidate points in the gradient amplitude are screened, and spline interpolation is performed on the ice edge candidate points along the gradient direction angle to obtain the sub-pixel coordinates of the ice edge; Based on the gradient direction angle, the ice edge candidate points are matched with the sub-pixel coordinates of the ice edge; Calculating the ice edge distance between the paired ice edge candidate points; The ice edge distance is converted into actual physical thickness value through geometric projection method.

5. The ice-covered state detection method based on image processing according to claim 4, characterized in that: The ice risk prediction model is constructed, the ice growth rate is calculated by the optical flow method, the ice growth rate and the actual physical thickness value are dynamically integrated to generate the ice risk index. The specific steps are: The convolutional neural network (CNN) is used as the basic framework of the ice risk prediction model to build an ice risk prediction model. Define the input layer, optical flow processing layer, feature fusion layer, and output layer of the ice risk prediction model; The input layer receives the actual physical thickness value; The optical flow processing layer uses the optical flow method to perform vector operations on multiple actual physical thickness values to generate the ice growth rate; The feature fusion layer uses a weighted superposition algorithm to perform point-by-point weighted superposition of the actual physical thickness value and ice growth rate to generate a risk index for the ice cover state. The output layer outputs the risk index of the icing state.

6. The ice-covered state detection method based on image processing according to claim 5, characterized in that: The risk threshold is set, and the risk index and the risk threshold are compared to determine the risk level of the icing state. The specific steps are: Set risk thresholds, defining them as low, medium, and high risk thresholds; When the risk index is at the low risk threshold, the risk level of the icing state is determined to be low risk; When the risk index is at the medium risk threshold, the risk level of the icing state is determined to be medium risk; When the risk index is at the high risk threshold, the risk level of the icing state is determined to be high risk.

7. The ice-covered state detection method based on image processing according to claim 6, characterized in that: The three-level warning signal is set according to the risk level of the icing state, the warning area of the icing state is marked and a warning notification is issued. The specific steps are: Set three levels of warning signals according to the risk level of icing conditions; Use OpenCV functions to mark high-risk areas with red rectangles, medium-risk areas with orange outlines, and low-risk areas with yellow text in the actual physical thickness values; When the risk level is high, a red warning signal is triggered and an alarm SMS is sent to the power dispatch center via the MQTT protocol; When the risk level is medium, an orange warning signal is triggered and an alarm message is pushed to the monitoring platform of the power base station via the long-distance wide area network LoRaWAN; When the risk level is low, a yellow warning signal is triggered and recorded as a detection log file.

8. An ice cover detection system based on image processing, based on the ice cover detection method based on image processing according to any one of claims 1 to 7, characterized in that: include, Acquisition module, used to collect visual inspection data and perform preprocessing; A fusion module is used to extract visual detection image features from visual detection data using a multi-scale convolutional network, and dynamically fuse the visual detection image features through a spatial attention mechanism to obtain fused features; A calculation module is used to obtain the sub-pixel coordinates of the ice layer edge of the fused features and calculate the distance between the ice layer edges of the fused features to obtain the actual physical thickness value; The prediction module is used to build an icing risk prediction model, calculate the ice growth rate through the optical flow method, and dynamically integrate the ice growth rate and actual physical thickness value to generate an icing risk index; A determination module is used to set a risk threshold, compare the risk index with the risk threshold, and determine the risk level of the icing state; The processing module is used to set three-level warning signals according to the risk level of the icing state, mark the warning area of the icing state and issue warning notifications.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the ice coating state detection method based on image processing according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the ice coating state detection method based on image processing according to any one of claims 1 to 7 are implemented.

Images