Image recognition processing method and system for three-span two adjacent point locations of power transmission line

Abstract

The invention discloses an image recognition processing method for three-span two-adjacent point locations of a power transmission line, and relates to the field of power transmission line monitoring. The method comprises an image receiving stage, a data preprocessing stage, a first feature extraction stage, an image fusion stage, a second feature extraction stage, a risk factor analysis stage and a multi-stage evaluation stage. In the data preprocessing stage, images are denoised and aligned, and in the first feature extraction stage, regional importance feature vectors are obtained. The image fusion adopts a dynamic weight weighted average method, and the weight is calculated based on local entropy, local energy and regional importance. And a fusion feature set is obtained through second feature extraction, a risk factor coefficient is calculated in combination with historical data, and early warning is output through third-level review. The system comprises an image receiving module, a data preprocessing module, a feature extraction module, an image fusion module, a risk factor analysis module, a multi-level review module and an early warning action module. According to the method, through feature screening and fusion optimization, the accuracy and scene adaptability of risk identification are improved, and safe operation of the power transmission line is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power transmission line monitoring, in particular to a kind of image recognition processing method and system of three-span two adjacent point of power transmission line. BACKGROUND

[0002] As an important part of power system, the safe and stable operation of power transmission line is directly related to national energy security and social and economic development. The "three-span two adjacent" point is always the focus and difficulty area of power transmission line operation and maintenance due to the complex environment and high safety risk. These areas are easily affected by natural environmental changes (such as heavy rain, strong wind, etc.) and human activities (such as construction, excavation, etc.), which may cause tower foundation soil erosion, line equipment overheating, foreign matter invasion and other safety hazards. If not discovered and handled in time, it may cause line fault trip or even major safety accidents.

[0003] Traditional "three-span two adjacent" point monitoring mainly relies on manual inspection, which not only has high labor intensity and low efficiency, but also is limited by terrain, climate and other factors, making it difficult to achieve real-time monitoring all-weather and full coverage. With the development of image acquisition technology and artificial intelligence technology, image recognition-based monitoring methods have been gradually applied to power transmission line monitoring. However, the existing technology has some shortcomings. The received images are directly extracted and recognized, and there is no targeted screening, the redundant information interference is serious, and the features cannot be distinguished, which leads to the centralization and fuzzification of features. SUMMARY

[0004] The technical problem to be solved by the present application is to provide an image recognition processing method and system that is more accurate and more suitable for special point.

[0005] To solve the above technical problems, the technical solution adopted by the present application is:

[0006] An image recognition processing method for three-span two adjacent point of power transmission line, the key is that,

[0007] In the image receiving stage, the system receives the images transmitted by the image acquisition device set at the three-span two adjacent point of power transmission line;

[0008] In the data preprocessing stage, the image data is preprocessed;

[0009] In the first feature extraction stage, the features are preliminarily extracted;

[0010] In the image fusion stage, the visible light image and the infrared image are weighted and fused;

[0011] In the second feature extraction stage, the feature extraction model is used to extract the features from the fused image data;

[0012] In the risk factor analysis phase, the risk factor coefficients of each feature in the feature set are obtained based on historical data and the feature set.

[0013] In the multi-level review stage, each feature in the feature set is evaluated using multi-level standards, and early warnings are issued.

[0014] Preferably, the data preprocessing stage includes denoising and aligning the image;

[0015] The denoising process includes median filtering and second-order differential sharpening. First, a template is selected, and different sizes of templates are used for image data in different regions, followed by median filtering. Then, the filtered image is subjected to second-order differential sharpening using the Laplacian operator to obtain clear on-site graphic data.

[0016] Then, multiple on-site image data were registered to obtain the overall visible light and infrared images of the three spans and two adjacent points of the transmission line.

[0017] Preferably, in the first feature extraction stage, features are extracted and stitched together from the visible light image and the infrared image to obtain a region importance feature vector. This includes texture details in visible light and thermal radiation features in infrared images.

[0018] Preferably, the image fusion uses a weighted averaging method to dynamically weight the images. The dynamic weights are based on local entropy calculation, local energy calculation, and region importance calculation. The weight calculation formula is as follows:

[0019]

[0020] Where H(x,y) is the local entropy value of the visible light image, E(x,y) is the local energy of the infrared image, and λ′(x,y) is the region importance factor;

[0021] Image fusion is performed based on weights, using the following formula:

[0022] I[x,y]=w[x,y]·I V [x,y]+[1-w[x,y]]·I T [x,y]

[0023] Where I V For visible light images, I T Infrared image;

[0024] The local entropy calculation is performed by dividing the visible light image into regions, calculating the probability of gray values ​​appearing in each region, and then calculating the local entropy value.

[0025] The local energy calculation adopts the same division method as the visible light image to calculate the sum of squares of pixel gray values in each local region to represent the local energy;

[0026] The region importance calculation through the uncertainty evaluation modified random forest model includes the following steps:

[0027] Building a data set: collect historical image data and extract features to form a training feature vector combined with distance parameters, and label the region importance level;

[0028] Training a random forest model: taking the feature vector as input and the region importance level as output, and setting the value of the basic importance factor according to different region importance levels;

[0029] Calculate the prediction uncertainty: for each pixel position (x, y), quantify the uncertainty by calculating the variance U(x, y) of the prediction results of the internal decision tree of the random forest;

[0030] By calculating the variance of the prediction results of the internal decision tree to quantify the uncertainty, and taking it as a correction term of the region importance factor;

[0031] First, for each pixel position (x, y), obtain the prediction probability distribution T1[x, y], T2[x, y],..., T n [x, y] of all decision trees in the random forest;

[0032] Calculate the variance of the prediction probability

[0033] U(x, y) = Var[T1[x, y], T2[x, y],..., T n [x, y]]

[0034] as the uncertainty measure of this position; according to the uncertainty, modify the region importance factor:

[0035] λ'(x, y) = λ base (x, y)·(1+α·U(x, y))

[0036] where α is an adjustable parameter, λ base [x, y] is the basic importance factor based on the original features.

[0037] Preferably, the second feature extraction stage performs feature extraction on the fused image to obtain a fused feature set;

[0038] The risk factor analysis module calculates a risk factor coefficient of each feature according to historical data in combination with the fusion feature set; the risk factor analysis module comprises a feature-risk association database, which contains historical image features and corresponding risk event records; the risk factor analysis module calculates a correlation coefficient of each feature and a risk event based on the database; and the correlation coefficient is taken as the risk factor coefficient.

[0039] Preferably, the multi-stage review stage is divided into three stages:

[0040] 1) The first-stage review is performed on a single feature, a threshold is set, when a feature exceeds the threshold, a first-stage warning is issued, and the second-stage review is entered;

[0041] 2) The second-stage review is performed on each feature, a total risk score is obtained by multiplying and adding the feature score and the corresponding risk factor coefficient, a second-stage warning is issued, and the third-stage review is entered;

[0042] 3) The third-stage review is performed on a single feature, an extreme condition threshold is set, when a feature triggers the extreme condition threshold, a third-stage warning and the corresponding feature are issued.

[0043] An image recognition processing system for three-span two-adjacent point positions of a power transmission line is used to implement the above method, and the key lies in comprising the following parts:

[0044] An image receiving module, through which the system is connected to an image acquisition device arranged at the three-span two-adjacent point positions of the power transmission line;

[0045] A data preprocessing module connected to an output end of the image receiving module;

[0046] A feature extraction module, a first input end of which is connected to the data preprocessing module, a first output end of which is connected to an image fusion module, a second input end of which is connected to the image fusion module, and a second output end of which is connected to a risk factor analysis module;

[0047] An image fusion module: receiving a visible light image and an infrared image, and outputting a fusion image;

[0048] A risk factor analysis module: receiving a fusion image feature, connected to a multi-stage review module, and outputting a risk factor coefficient;

[0049] A multi-stage review module: receiving a fusion image feature, outputting different levels of warnings after being judged by multi-stage standards.

[0050] Preferably, the multi-stage review module is externally connected to a warning action module, and actions are performed according to the warning levels, including:

[0051] A first-stage warning action: increasing the frequency of on-site monitoring; a second-stage warning action: generating a detailed repair work order, and labeling a fault position and related features; and a third-stage warning action: applying for a partial power outage action, and reporting dangerous features.

[0052] The beneficial effects produced by the above technical solutions are:

[0053] The present application realizes accurate screening and optimization of features through progressive processing of twice feature extraction and dynamic weight fusion. The first feature extraction stage extracts core features related to risks, and the image fusion stage dynamically adjusts weights based on local entropy value, energy and regional importance, strengthens key features and weakens redundant information, so that the second feature extraction is more suitable for three-span two-adjacent scenes, and the accuracy of picture recognition and processing and the adaptability of special points are improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Figure 1 is a flow chart of an image recognition processing method for three-span two-adjacent points of a power transmission line proposed by the present application;

[0056] Figure 2 is a structural schematic diagram of an image recognition processing system for three-span two-adjacent points of a power transmission line. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0058] An image recognition processing method for three-span two-adjacent points of a power transmission line, such as Figure 1 . includes an image receiving stage, a data preprocessing stage, a first feature extraction stage, an image fusion stage, a second feature extraction stage, a risk factor analysis stage and a multi-level review stage.

[0059] The image receiving stage realizes information collection through device deployment and data transmission. Visible light cameras and infrared thermal imagers are installed at key positions of towers, crossing towers and adjacent areas of the "three-span two-adjacent" points of the power transmission line, and the image acquisition equipment periodically collects images and communicates with the system image receiving module through the wireless module.

[0060] The installation position, shooting angle, and focal length of the device are determined based on the actual range of the monitoring area, the topography, and the characteristics of the target object, to ensure that clear, comprehensive, and fully-covered visible light images are obtained. Near the tower of the power transmission line that crosses the railway, the camera should be installed at a position that can completely capture the ground conditions within a certain range on both sides of the railway. The shooting angle should be adjusted to cover the area extending from the tower foundation to the boundary of the railway. The focal length should be set to clearly distinguish the small textures and vegetation details on the ground, to meet the needs of subsequent accurate analysis of various soil erosion-related features in the images.

[0061] Similarly, for the infrared thermal imager, the appropriate installation position and angle should be selected based on its imaging principle and the matching requirements with the visible light images, so that it can cover the same area as the visible light camera and ensure that the obtained infrared images meet the corresponding standards in terms of thermal radiation resolution and temperature measurement accuracy, to accurately capture the temperature anomalies on the ground.

[0062] The image acquisition device automatically collects image data at preset time intervals and transmits the collected visible light images and infrared images to the image receiving module in the back end in real time, providing a continuous and stable data source for subsequent fusion and feature extraction operations.

[0063] In the data preprocessing stage, the pictures are denoised and aligned:

[0064] Median filtering: The image is divided into regions. Considering the different influences and characteristics of soil areas, vegetation areas, and water areas on the power transmission line, a 3x3 template is used in the soil area to avoid excessive smoothing that would obscure small terrain features and preserve edge features. A 5x5 template is used in the vegetation area to filter out noise caused by leaf reflections and shadows while preserving the integrity of the vegetation canopy. A 7x7 template is used in the water area to smooth out water surface ripple interference and highlight the shoreline boundary. The median filtering removes salt and pepper noise and improves denoising efficiency.

[0065] Second-order differential sharpening: The Laplacian operator is applied to the filtered image to enhance edge details such as tower edges and soil layers.

[0066] In median filtering, the gray values of each pixel in the template field are arranged in ascending order, and the middle gray value is used to replace the middle pixel value. Finally, all pixel points in the image are processed in this way to obtain the median filtered image g(x, y), which is

[0067] g(x, y) = Med{f(x-k, y-l), (k, l ∈ W)}

[0068] Where, W is the domain window size, f(x-k, y-l) is the pixel value in the domain template. In order to find the center value more simply, the total number of pixels contained in the W domain is generally an odd value.

[0069] Image definition improvement of second-order differential sharpening: In order to ensure the stability of the power transmission line, the inspection unmanned aerial vehicle cannot approach the power transmission line components, and at the same time, due to the limitation of the equipment, the image data obtained by the unmanned aerial vehicle has some fuzzy samples. In order to improve the detection accuracy, the fuzzy samples are sharpened. The second-order differential discrete formula adopts Laplace operator, and the Laplace transform of the binary function f(x, y) is defined as:

[0070]

[0071] The second-order partial derivative in x and y directions is:

[0072]

[0073] The above formula can be obtained:

[0074]

[0075] After obtaining the formula, the template based on Laplace operator can be obtained. The Laplace operator emphasizes the mutation of the gray scale in the image, and does not emphasize the slow transformation area of the image, so some gradually changing light gray edge lines will become the background color of the picture outline. Then the original picture and the Laplace image are superimposed, the original image is maintained, and the enhanced edge image can be seen, so as to realize the sharpening of the image.

[0076] Image alignment: the multiple images are registered, the perspective transformation matrix is calculated according to the feature points of the tower top and the wire suspension point, the orthographic correction is adopted for the inclined terrain such as slope, and the image deformation caused by the terrain undulation is eliminated, and finally the images are spliced into the visible light panorama and infrared panorama covering the whole three-span two-adjacent area.

[0077] The first feature extraction stage,

[0078] The core purpose of this stage is to comprehensively extract the key features closely related to soil erosion from the original visible light image and infrared image collected at the beginning, which can directly reflect the soil erosion condition and potential risk. These features will serve as the cornerstone of subsequent image fusion and overall soil erosion risk analysis, and are the extremely critical starting link in the whole monitoring process.

[0079] Visible light feature extraction:

[0080] Soil exposure (F1): It is one of the core indicators to measure the risk of soil erosion. By converting the RGB image to HSV space, the soil area is accurately segmented according to the specific feature threshold of soil in HSV (H: 15-45, S: 20-80, V: 30-70). Subsequently, a 3x3 erosion operator is used to remove interference factors such as weeds and gravel, and a 5x5 dilation operator is used to fill the small holes inside the soil area. Finally, according to the formula

[0081]

[0082] The specific value is calculated, the value range is between 0-100%, which clearly quantifies the degree of soil exposure in a certain area, providing an intuitive basis for judging the possibility of rain directly eroding the soil.

[0083] Soil texture feature (F2): This feature focuses on reflecting the roughness of the soil surface and the development of cracks, and its complexity is closely related to the activity level of soil erosion. By calculating the contrast (Contrast) and entropy (Entropy) of the soil area in 0°, 45°, 90°, and 135° directions, using the gray level co-occurrence matrix (GLCM), the contrast

[0084] ∑ i,j (i-j) 2 P(i,j)

[0085] The higher the value, the greater the fluctuation of the soil surface, and the higher the entropy

[0086]

[0087] The higher the value, the more complex the texture, and the average of the contrast and entropy in the four directions is obtained

[0088] F2 = [Contrast mean , Entropy mean ]

[0089] to comprehensively represent the texture feature of the soil and assist in judging the erosion condition of the soil.

[0090] Vegetation coverage (F3): Since vegetation plays an important role in soil fixation and rainwater interception in reducing the risk of soil erosion, it is necessary to accurately extract the vegetation coverage feature. First, use the normalized difference vegetation index (NDVI) approximation formula

[0091]

[0092] ∈=1e-5 to avoid denominator zero, determine the vegetation area, NDVI>0.3 area is determined as vegetation, and then further distinguish herb, NDVI 0.3-0.5, shrubs NDVI 0.5-0.7, trees NDVI>0.7, and according to their different soil holding capacity to give the corresponding weight coefficient, and finally through

[0093] F3=∑each type of vegetation area x weight / total area of the region

[0094] The calculation, the value range is 0-1.0, intuitive presentation of vegetation in the region of soil and water conservation contribution degree.

[0095] Gully development characteristics (F4): gully as a typical form of soil and water loss, its length, depth, density and other parameters are essential to assess the development trend of soil and water loss. First, the edge profile of the soil area is extracted by using Canny operator, and the linear features with continuous length>30cm are selected as the candidate area of gully, then combined with the image shooting angle (known) and the gray difference (shadow length) on both sides of the gully, the gully depth is determined by estimating formula

[0096] d=h x tan θ

[0097] h is the length of the shadow pixels, θ is the solar elevation angle, calculated by shooting time) to determine the gully depth, finally in the form of

[0098] F4=[length / region diagonal length, depth / average slope of the region, number of gullies / region area]

[0099] Quantified, the value range is 0-1.0, which provides powerful data support for in-depth analysis of the dynamic change of soil and water loss.

[0100] Surface crust condition (F5): soil surface crust formed by raindrop splashing will affect the rainwater infiltration rate and surface runoff, so this feature needs to be extracted. Through texture analysis, according to the characteristics of smooth, low texture (GLCM entropy value<2.0) of crust area in visible light image, which is in sharp contrast with the surrounding loose soil (entropy value>3.0) to identify the crust area, and then according to

[0101] F5=crust area / total soil area

[0102] Calculate the area ratio, the value range is 0-1.0, to measure the potential impact of crust on soil and water loss.

[0103] Feature extraction in infrared image:

[0104] Soil moisture distribution (F6): Since soil moisture is closely related to soil shear strength and water and soil loss risks such as landslides and collapses, it is necessary to accurately extract moisture information. Based on the temperature value T of the infrared image, with the aid of the calibration curve measured in the laboratory, the moisture M = 120-1.5T, T

[0105] in ℃, the temperature is converted into humidity value, and then the area ratio of the humidity > 60% in the region is calculated, denoted as F6, with a value range of 0-1.0, which provides key data for judging the possibility of water and soil loss caused by changes in soil moisture.

[0106] Surface runoff path (F7): As a "channel" of water and soil loss, the runoff path has obvious characteristic performance in the infrared image. By setting the area with a temperature lower than the regional average by 2℃ as the runoff candidate area, and then using the skeleton extraction algorithm to obtain the length and branch number of the runoff path, according to

[0107] F7 = total length of runoff / region perimeter

[0108] The calculation is taken, with a value range of 0-1.0 (the higher the value, the greater the risk of runoff erosion), to track and analyze the impact of surface runoff on water and soil loss.

[0109] Construction of regional importance feature vector: In order to eliminate the dimensional differences between different features, the above-mentioned 7 features (F1-F7) are first normalized to the [0, 1] interval. The 7 features are spliced with the original power transmission line features to form a regional importance feature vector, ensuring that these important features can be considered in the subsequent image fusion process and effectively retained in the fused image, providing a core basis for subsequent comprehensive analysis.

[0110] Dynamic weight calculation in image fusion stage:

[0111] Local entropy H[x, y]: The visible light image is divided into local regions of 16x16 pixels, and the probability p iwhere i represents different gray values, the value range is usually 0-255, specifically, each pixel point in the region is traversed, the gray value is recorded, then the number of occurrences of each gray value is counted, and then the number is divided by the total number of pixel points in the region, that is, the probability of the corresponding gray value can be obtained, and then the formula

[0112]

[0113] The calculation essentially represents the complexity of the surface texture of the region, the higher the entropy value, the more diverse the texture of the region, and the region with complex surface morphology such as gully and bare soil will have a higher entropy value. These regions with rich texture often have more obvious manifestations in the process of soil erosion, and soil erosion traces will be formed in gully areas by water flow, and bare soil areas are also prone to soil erosion, so calculating the local entropy value can effectively locate these key areas closely related to soil erosion.

[0114] Local energy E (x,y) : The same division is adopted for the infrared image, and for the pixel points in each region, the gray value of each pixel is calculated by using nested loop, and the square sum E (x,y) of the gray value is calculated to represent the thermal radiation intensity of the region. In specific calculation, an accumulation variable is set, and the gray value of each pixel is squared and accumulated in the variable, and finally the local energy value of the region is obtained. The formula is:

[0115] E (x,y) =∑I(i,j) 2

[0116] The high local energy region usually corresponds to the abnormal situation of the ground temperature, in the area with high soil moisture, due to the evaporation of water, heat will be absorbed, resulting in the region on the infrared image showing a relatively low temperature state, and its local energy value will be different from the surrounding normal region. This temperature anomaly region often has an inherent relationship with soil erosion, because the change of soil moisture is one of the important factors affecting soil erosion, so by calculating the local energy, these potential soil erosion related regions in the infrared image can be highlighted, providing important thermal feature information for subsequent fusion and feature extraction.

[0117] Regional importance factor (S):

[0118] This part comprehensively considers the geographical and environmental characteristics closely related to soil erosion risk, and determines the basic importance factor by random forest model,

[0119] Training the random forest model: the feature vectors of the historical images after the first feature extraction are spliced with the distance features of the two adjacent point positions of the three spans as input, and the region importance level annotated by artificial is taken as output. The region importance level is annotated by artificial according to the rich practical experience and the deep understanding of the actual situation of the surrounding environment of the transmission line and soil erosion. For those regions with high soil exposure, low vegetation coverage and obvious gully development, combined with field investigation, it is judged that they have high risk in soil erosion, and artificial annotation is high importance level. For the region with dense vegetation, relatively stable soil condition and no obvious runoff path, it is annotated as low importance level, and the importance of the region is appropriately increased when the distance between the two adjacent point positions of the three spans is closer. And the basic importance factor is output according to the region importance level.

[0120] Uncertainty correction of random forest algorithm model: in order to reduce the influence of this uncertainty and improve the accuracy of the region importance factor, the key step of uncertainty correction is needed. In this process, the variance U[x,y] of the prediction results of 100 decision trees in the random forest is calculated. As a statistical quantity to measure the degree of data dispersion, variance can well reflect the difference between the prediction results of different decision trees. If the variance is large, it means that the judgment of the region importance between the decision trees is greatly different, which means that the model has high uncertainty in determining the region importance; on the contrary, the smaller the variance is, the higher the consistency between the decision trees is, and the stronger the certainty of the model is.

[0121] The calculation formula is:

[0122]

[0123] , where T i [x,y] represents the probability prediction of the i th decision tree for the position (x, y) belonging to the high-risk region, and n is the total number of decision trees in the random forest. In the actual "three-span two-adjacent" environment, the terrain is undulating, the vegetation coverage is uneven, and the human disturbance factors are more common. The difference between different decision trees in judging the soil erosion risk of the same region may be large. The uncertainty reflected by this difference exactly embodies the difficulty of risk judgment and the potential high risk degree of the region, so it must be considered in the determination of the region importance factor.

[0124] Correct the region importance factor:

[0125] Based on the basic importance factor λ base [x,y] and the uncertainty measure U[x,y], the formula

[0126] λ′(x,y)=λbase (x, y) · (1 + a · U(x, y))

[0127] The region importance factor is modified, where a is an adjustable parameter, and the value of a directly determines the degree of influence of uncertainty factors on the region importance factor. Reasonable adjustment of a can enable the fusion process to adaptively adjust the weight distribution according to the actual complexity and risk characteristics of different regions, and highlight those regions that may hide higher soil erosion risks due to higher uncertainty, for example, in regions with complex terrain, frequent vegetation and temperature changes, the prediction difference between decision trees is often larger, and the uncertainty is higher. By appropriately increasing the value of a, the importance weight of the region in the fused image can be correspondingly increased, so that the related features are more prominent after fusion, facilitating subsequent accurate extraction of soil erosion related features and risk assessment.

[0128] According to the modified region importance factor λ'(x, y) calculated above, the local entropy value H(x, y) of the visible light image, and the local energy E(x, y) of the infrared image, the dynamic fusion weight w(x, y) is calculated according to the following formula:

[0129]

[0130] The design principle of this formula is to dynamically adjust the fusion weight of different regions by considering the region importance factor, the texture complexity of the visible light image (reflected by the local entropy value), and the thermal radiation intensity of the infrared image (reflected by the local energy). For regions that are important in terms of soil erosion risk (i.e. λ'(x, y) value is large), have rich surface texture (H(x, y) value is high), and have obvious thermal radiation characteristics (E(x, y) value is high), a relatively large fusion weight is given, so that the features of these regions can be more prominent in the fused image, thereby facilitating the subsequent extraction of key information related to soil erosion. For relatively unimportant or uncharacteristic regions, a smaller fusion weight is given, which ensures the integrity of the overall image information while avoiding irrelevant information interference, improving the relevance and effectiveness of image fusion.

[0131] Image fusion: according to the formula

[0132] I[x, y] = w[x, y] · I V [x, y] + [1 - w[x, y]] · I T [x, y]

[0133] Generate a fused image, where I V is a visible light image, I TThe infrared image; through this weighted fusion operation, a fusion image I[x, y] is generated, which ingeniously fuses the rich texture detail information in the visible light image and the thermal radiation characteristic information in the infrared image, and highlights the various regional features closely related to the water and soil loss risk by means of dynamic adjustment of the regional importance factor, providing a more valuable and more beneficial data basis for the subsequent feature extraction link.

[0134] Second feature extraction and risk factor analysis fusion feature set: deep features are extracted from the fusion image. After the first stage of feature extraction and image fusion, this stage focuses on extracting features from the fused image that can more deeply and comprehensively reflect the actual situation and dynamic change trend of water and soil loss, aiming to provide more comprehensive information that reflects the essential characteristics of water and soil loss for water and soil loss monitoring, thereby achieving accurate monitoring and control of water and soil loss.

[0135] Risk factor calculation:

[0136] Building a feature-risk database: collecting risk events and corresponding feature data of three-span two-adjacent points.

[0137] Correlation coefficient: commonly used means such as Pearson correlation coefficient are used for calculation.

[0138] Taking soil exposure degree as an example, all data pairs with soil exposure degree values and corresponding risk events, such as rainwater erosion leading to soil loss near the tower and affecting the stability of the tower, are extracted from the database. Through statistical analysis of these large amounts of data pairs, the correlation coefficient between soil exposure degree and the corresponding risk events is calculated. If the correlation coefficient is high, it means that there is a strong linear correlation between soil exposure degree and the occurrence of risk events, i.e. the change of soil exposure degree is likely to have a significant impact on the safe operation of the transmission line; on the contrary, if the correlation coefficient is low, it means that the correlation between the feature and the risk event is relatively weak.

[0139] Calculate the Pearson correlation coefficient of each feature and risk event as the risk factor coefficient, ranging from 0 to 1.

[0140] Multi-level review stage

[0141] Primary review: Set a threshold for each feature, and if any feature exceeds the threshold, issue a primary warning and enter the secondary review. For the soil exposure feature, based on a large amount of historical data and research on the impact of soil erosion on tower stability, a reasonable percentage is set as the threshold. When the soil exposure exceeds 60%, it means that the soil erosion in this area may have been relatively serious, posing a potential threat to the stability of the tower foundation.

[0142] Secondary review: Based on the primary review, the secondary review further considers the features and their corresponding risk factor coefficients to assess the overall risk level.

[0143] The actual monitoring score of each feature is multiplied by the corresponding risk factor coefficient and then added to obtain a total risk score. The feature score can be quantitatively assigned based on the actual performance of the feature, and scored according to factors such as the degree of deviation from the normal range. For example, if the actual monitoring of vegetation coverage is close to the set ideal value, the score can be relatively high; if the vegetation coverage is much lower than the normal level, the score is lower. Then, multiply the feature score by its corresponding risk factor coefficient, which is the correlation coefficient with the risk event calculated earlier. After calculating all features, add them up to get the total risk score, which can comprehensively and comprehensively reflect the overall risk situation of the "three-span two-adjacent" point of the transmission line. When the comprehensive score exceeds the pre-set threshold, a secondary warning is issued and the tertiary review is entered.

[0144] Tertiary review: Set an extreme condition threshold, the tertiary review as the last line of defense in the multi-level review is the most stringent review of the risk situation. When this threshold is triggered, it is likely to cause a major safety accident and cause devastating damage to the normal operation of the transmission line, triggering a tertiary warning.

[0145] Through such a progressive, rigorous and meticulous risk factor analysis module and multi-level review stage, the risk situation of the "three-span two-adjacent" point of the transmission line can be comprehensively and accurately evaluated and warned, potential risks can be discovered in time and effective measures can be taken to prevent them, and the long-term safe operation of the transmission line can be guaranteed.

[0146] An image recognition processing system for a three-span two-adjacent point of a transmission line, comprising Figure 2 The following parts are included:

[0147] An image receiving module, the system connects the image acquisition device set at the three-span two-adjacent point of the transmission line through this module port;

[0148] A data preprocessing module connected to the output end of the image receiving module;

[0149] The feature extraction module has a first input end connected to the data preprocessing module and a first output end connected to the image fusion module; a second input end connected to the image fusion module and a second output end connected to the risk factor analysis module;

[0150] The image fusion module receives visible light images and infrared images and outputs a fused image.

[0151] The risk factor analysis module receives fused image features, is connected to the multi-level review module, and outputs risk factor coefficients.

[0152] The multi-level review module receives fused image features, judges through multi-level standards, and outputs different levels of early warnings.

[0153] To implement the above method, the system is externally connected to a warning action module which acts in response to the early warning signals of the multi-level review module.

[0154] When the multi-level review module issues a first-level early warning, the warning action module will correspondingly start a first-level early warning action, i.e., increase the on-site monitoring frequency. The first-level early warning indicates that an abnormal situation has occurred in a certain feature, but the risk level is relatively low at this stage. By increasing the monitoring frequency, the latest state information of the feature and the related area can be obtained more timely and in more detail. At the same time, other monitoring methods for the related area can be increased to closely observe the trend of the abnormal feature and timely discover whether it has further signs of deterioration, providing more sufficient data support for subsequent risk judgment and response measures.

[0155] Second-level early warning action

[0156] Once the second-level early warning signal is received, the warning action module will perform a second-level early warning action to generate a detailed repair work order and mark the fault location and related features. At this time, the risk has reached a relatively high level, and professional personnel need to be arranged for repair and investigation. The generated repair work order will record the location information of the abnormality in detail, such as which base tower, which part of the tower, etc., and clearly mark the related abnormal features, such as low vegetation coverage, certain degree of deviation of conductor sag, etc. Such a repair work order can accurately guide the repair personnel to the corresponding location, quickly lock the problem, and carry out targeted repair work, thereby improving the repair efficiency, timely eliminating the risk hidden danger, and ensuring the normal operation of the power transmission line.

[0157] Third-level early warning action

[0158] When the multi-stage review module issues a third-level early warning, the early warning action module will take the most urgent third-level early warning action, i.e., apply for partial power failure action, and report the dangerous features. Because the third-level early warning means that the risk has reached an extremely critical level, it is extremely likely to cause devastating damage to the power transmission line. In order to avoid major safety accidents and protect personnel safety, it is necessary to cut off the power supply of the relevant area in time, that is, to apply for partial power failure. At the same time, the dangerous features triggering the third-level early warning are reported in detail to the relevant command center, operation and maintenance team, etc., so that all parties can quickly understand the severity of the situation, coordinate resources, organize professional forces to arrive at the scene in the first time, take the most effective emergency measures, and minimize the loss, and ensure that the power transmission line can be safely restored to operation.

[0159] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to these embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for image recognition and processing of two adjacent points across three spans of a power transmission line, characterized in that, During the image reception phase, the system receives images transmitted by image acquisition devices located at two adjacent points across three spans of the transmission line; In the data preprocessing stage, the image data is preprocessed; The first feature extraction stage involves preliminary feature extraction. Image fusion stage: weighted fusion of visible light images and infrared images; In the second feature extraction stage, features are extracted from the fused image data using a feature extraction model. In the risk factor analysis phase, the risk factor coefficients of each feature in the feature set are obtained based on historical data and the feature set. In the multi-level review stage, each feature in the feature set is evaluated using multi-level standards, and early warnings are issued.

2. The image recognition processing method for three spans and two adjacent points of a transmission line according to claim 1, characterized in that, The data preprocessing stage includes denoising and aligning the image; The denoising process includes median filtering and second-order differential sharpening. First, a template is selected, and different sizes of templates are used for image data in different regions, followed by median filtering. Then, the filtered image is subjected to second-order differential sharpening using the Laplacian operator to obtain clear on-site graphic data. Then, multiple on-site image data were registered to obtain the overall visible light and infrared images of the three spans and two adjacent points of the transmission line.

3. The image recognition processing method for three spans and two adjacent points of a transmission line according to claim 1, characterized in that, In the first feature extraction stage, features are extracted and stitched from the visible light image and the infrared image. Next, we obtain the regional importance feature vector. This includes texture details in visible light and thermal radiation features in infrared images.

4. The image recognition processing method for three spans and two adjacent points of a transmission line according to claim 1, characterized in that, The image fusion method uses a weighted average to dynamically weight the images. The dynamic weights are based on local entropy calculation, local energy calculation, and region importance calculation. The weight calculation formula is as follows: Where H(x,y) is the local entropy value of the visible light image, E(x,y) is the local energy of the infrared image, and λ′(x,y) is the region importance factor; Image fusion is performed based on weights, using the following formula: I[x,y]=w[x,y]·I V [x,y]+[1-w[x,y]]·I T [x,y] Among them I V For visible light images, I T Infrared image; The local entropy calculation is performed by dividing the visible light image into regions, calculating the probability of gray values ​​appearing in each region, and then calculating the local entropy value. The local energy calculation uses the same partitioning method as the visible light image, and calculates the sum of squares of pixel gray values ​​in each local region to characterize the local energy. The calculation of regional importance uses a random forest model modified by uncertainty assessment, and includes the following steps: Dataset construction: Collect historical image data and extract features, combine them with distance parameters to form training feature vectors, and label the importance level of regions; Training the random forest model: using the aforementioned feature vector as input and the region importance level as output, and assigning values ​​to the basic importance factor according to different region importance levels; Calculate prediction uncertainty: For each pixel location (x,y), the uncertainty is quantified by calculating the variance U(x,y) of the prediction results of the decision tree inside the random forest; By calculating the variance quantification uncertainty of the prediction results of the internal decision tree in the output stage of the random forest, and using it as a correction term for the regional importance factor; First, for each pixel position (x, y), obtain the prediction probability distributions T1[x, y], T2[x, y], ..., T of all decision trees in the random forest. n [x,y]; Calculate the variance of the predicted probability U(x,y)=Var[T1[x,y],T2[x,y],...,T n [x,y]], As a measure of uncertainty at this location; the regional importance factor is adjusted based on the uncertainty: λ′(x,y)=λ base (x,y)·(1+α·U(x,y)), Where α is an adjustable parameter, λ base [x,y] represents the fundamental importance factor based on the original features.

5. The image recognition processing method for three spans and two adjacent points of a transmission line according to claim 1, characterized in that, The second feature extraction stage extracts features from the fused image to obtain a fused feature set; The risk factor analysis module calculates the risk factor coefficient of each feature based on historical data and the fusion feature set. This includes building a feature-risk association database, containing historical image features and corresponding risk event records; Based on the database, the correlation coefficient between each feature and the risk event is calculated; the correlation coefficient is used as the risk factor coefficient.

6. The image recognition processing method for three spans and two adjacent points of a transmission line according to claim 1, characterized in that, The multi-level review stage is divided into three levels of review: 1) Level 1 review: Review individual features and set thresholds. When a feature exceeds the threshold, an alert is issued and the review proceeds to Level 2 review. 2) Secondary review: Multiply each feature score by the corresponding risk factor coefficient and add them together to obtain the total risk score. Issue a secondary alarm and proceed to tertiary review. 3) Three-level review: set extreme condition thresholds for individual features. When a feature triggers an extreme condition threshold, a three-level warning and the corresponding feature are issued.

7. An image recognition and processing system for three spans and two adjacent points of a power transmission line, used to implement the method described in claims 1-6, characterized in that, Includes the following parts: The image receiving module is used by the system to connect to image acquisition devices located at two adjacent points across three spans of the power transmission line. The data preprocessing module is connected to the output of the image receiving module; The feature extraction module has a first input connected to the data preprocessing module and a first output connected to the image fusion module. The second input terminal is connected to the image fusion module, and the second output terminal is connected to the risk factor analysis module. Image fusion module: Receives visible light images and infrared images, and outputs a fused image; The risk factor analysis module receives fused image features, connects to a multi-level review module, and outputs risk factor coefficients. The multi-level review module receives fused image features, evaluates them according to multi-level standards, and outputs warnings of different levels.

8. The image recognition and processing system for three spans and two adjacent points of a transmission line according to claim 7, characterized in that, The multi-level review module is externally connected to an early warning action module, which performs actions based on the early warning level, including: Level 1 warning action: Increase the frequency of on-site monitoring; Level 2 warning action: Generate detailed maintenance work orders and mark the location of the fault and related characteristics; Level 3 warning action: Request a partial power outage and report the dangerous characteristics.

Images