Intelligent inspection and maintenance system for emergency fault of power line

Abstract

The invention relates to the technical field of data processing, in particular to an intelligent inspection and maintenance system for power line emergency faults, which comprises a multi-mode fusion module, and is used for generating an icing degree value of a power line by improving a DS evidence theory model according to data acquired by a laser scanning camera, a visible light camera and an infrared camera, the DS evidence theory model is improved to reflect the influence degree of real-time weather data on each acquisition value and the correlation degree among the acquisition data; the task adjusting module is used for adjusting the deicing power and the moving speed of the deicing robot for executing the deicing task according to the icing degree value and the predicted air temperature data; and the task execution judgment module is used for judging whether to control the deicing robot to start the deicing task or not according to the comparison result of the icing degree values and the icing safety values, and judging whether to control the deicing robot to end the deicing task or not according to the change trend of the plurality of icing degree values. According to the invention, unified scheduling management of the power line icing emergency fault by the command and scheduling platform through satellite communication is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent inspection and maintenance system for emergency faults of power lines. Background Art

[0002] In the event of an emergency failure of a power line, ice covering the surface of the power line increases the weight of the line, causing the line to shake, increasing the impact load on the line and reducing the insulation performance of the line, greatly increasing the safety risk of the power line.

[0003] In extreme environments where ice is formed on power lines, manual de-icing by climbing towers is less safe, so de-icing is performed using hot melt technology of de-icing robots hung on power lines. During this process, on-site personnel need to observe the degree of ice coverage on distant power lines through the camera of the drone and then set the working parameters of the de-icing robot. Specifically, on-site personnel need to operate the drone to prevent it from falling, observe the collected images of the drone, and adjust the working parameters of the de-icing robot. The operation is difficult in extreme environments with strong winds and heavy snow.

[0004] Given that my country's BeiDou-3 satellite system construction has also made great progress, at the end of 2018, the basic BeiDou-3 system was completed and provided global services, promoting the application of BeiDou in the power emergency system and achieving communication without blind spots. At present, the BeiDou system has fully covered all regions of my country, and BeiDou satellite signals can be received in any remote mountainous area or remote desert area. The industry satellite self-organizing network system solves the problem of emergency communication in harsh environments or emergencies through multi-hop relay. The platform has the characteristics of easy installation, flexible use, and simple operation. It can be widely used in special fields and special areas of the power industry, and provide communication convenience for production operations in special scenarios.

[0005] Therefore, how to use satellite communications so that on-site personnel only need to control the drone, while the command and dispatch platform can use the drone and the infrared camera that detects power lines to perform multi-modal data fusion for icing emergency fault judgment and maintenance control of de-icing robots is a technical problem that needs to be solved. Summary of the invention

[0006] To this end, the present invention provides an intelligent inspection and maintenance system for power line emergency faults. By performing multi-modal data fusion on the data collected by the drone's laser scanning camera, visible light camera and infrared camera for detecting power lines, emergency icing fault judgment and maintenance control of de-icing robots are carried out, the work difficulty of on-site personnel is reduced, and unified dispatching and management of power line emergency icing faults by the command and dispatch platform through satellite communications is realized.

[0007] To achieve the above-mentioned purpose, the present invention proposes an intelligent inspection and maintenance system for emergency faults of power lines. The command and dispatch platform communicates with the laser scanning camera of the inspection-end drone, the visible light camera and the infrared camera of the inspection-end power line, as well as the de-icing robot arranged on the power line at the maintenance end through a satellite communication network, including a multimodal fusion module, a task adjustment module and a task execution judgment module mounted on the command and dispatch platform, wherein:

[0008] The multimodal fusion module is used to generate an ice coverage value of the power line through an improved DS evidence theory model according to the collected data of the laser scanning camera, the visible light camera and the infrared camera, wherein the improved DS evidence theory model reflects the influence of real-time weather data on each collected value and the correlation between each collected data;

[0009] The task adjustment module is used to adjust the deicing power and moving speed of the deicing robot when performing the deicing task according to the ice coverage degree value and the predicted temperature data;

[0010] The task execution judgment module is used to judge whether to control the deicing robot to start the deicing task according to the comparison result of the ice coverage degree value and the ice coverage safety value, and to judge whether to control the deicing robot to end the deicing task according to the change trend of multiple ice coverage degree values.

[0011] Further, the multimodal fusion module includes a first evidence set generation unit, a coefficient generation unit, a second evidence set generation unit, a third evidence set generation unit and a calculation unit;

[0012] The first evidence set generating unit is used to generate a first evidence set reflecting the degree of ice coverage according to the numerical gradient corresponding to the collected data;

[0013] The coefficient generating unit is used to generate a coefficient matrix according to the real-time weather data;

[0014] The second evidence set generating unit is used to calculate a second evidence set according to the first evidence set and the coefficient matrix;

[0015] The third evidence set generating unit is used to generate a third evidence set reflecting the correlation between each collected data according to the second evidence set;

[0016] The calculation unit is used to calculate the ice coverage degree value by using the third evidence set through the improved DS evidence theory model.

[0017] Further, the real-time weather data includes real-time illumination data and the real-time temperature data, and the coefficient generation unit includes a laser scanning reliability generation subunit, a visible light image reliability generation subunit, an infrared data reliability generation subunit and a coefficient calculation subunit;

[0018] The laser scanning reliability generating subunit is used to generate laser scanning reliability according to the illumination dark gradient corresponding to the real-time illumination data;

[0019] The visible light image reliability generating subunit is used to generate visible light image reliability by corresponding the real-time illumination data to the illumination brightness gradient;

[0020] The infrared data reliability is used to generate the infrared data reliability by corresponding the real-time air temperature data to the extreme cold temperature gradient;

[0021] The coefficient calculation subunit is used to generate the coefficient matrix by normalizing the laser scanning reliability, the visible light image reliability and the infrared data reliability.

[0022] Further, the third evidence set generation unit includes a weight calculation subunit and a comprehensive calculation subunit;

[0023] The weight calculation subunit is used to calculate the weight corresponding to each collected data through the correlation coefficient formula;

[0024] The comprehensive calculation subunit performs normalization operation on the multiple weights and then performs weighted calculation on the second evidence set to generate the third evidence set.

[0025] In the above scheme, multimodal data fusion is realized by considering the data differences between various cameras and the impact of extreme weather on various data collected by cameras, thereby achieving accurate judgment on whether to carry out de-icing tasks.

[0026] Furthermore, the data collected by the laser scanning camera is point cloud image data;

[0027] The first evidence set generating unit extracts point cloud features of the power line area and the ice-covered area from the point cloud image data through a convolutional neural network model, and generates elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the point cloud ice-covered degree gradient interval;

[0028] Among them, the convolutional neural network model includes a point cloud multi-head attention pooling mechanism to process the spatial relationship of three-dimensional point cloud image data.

[0029] Furthermore, the multi-head attention pooling mechanism adopts the PTv2 network framework, whose weight encoding function is generated by grouping linear layers, normalization layers, activation layers and attention fully connected layers to adjust the spatial capture efficiency of three-dimensional point cloud image data.

[0030] Further, the data collected by the visible light camera is visual image data;

[0031] The first evidence set generating unit is used to identify the power line area and the ice-covered area of ​​the visual image data through a YOLO-based image recognition model, and generate elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the visible light ice-covered degree gradient interval.

[0032] Furthermore, the data collected by the infrared camera is thermal image data;

[0033] The first evidence set generating unit is used to generate elements of the first evidence set according to the temperatures of multiple acquisition points on the surface of the power line in the thermal image data corresponding to the infrared ice coverage gradient interval.

[0034] In the above scheme, it is achieved to obtain accurate power line ice coverage condition collection values ​​through different methods corresponding to multiple cameras.

[0035] Further, the task adjustment module includes a deicing power calculation unit and a moving speed calculation unit;

[0036] The deicing power calculation unit is used to calculate the deicing power according to the difference between the ice coverage degree value and the ice coverage standard value;

[0037] The moving speed calculation unit is connected to the deicing power calculation unit, and is used to generate a predicted ice formation rate according to the plurality of predicted air temperature data, and calculate the moving speed according to the predicted ice formation rate and the ice degree value.

[0038] Furthermore, the task execution judgment module is used to calculate an ice degree prediction value through a regression model using multiple ice degree values, and determine whether to control the deicing robot to end the deicing task based on a comparison result between the ice degree prediction value and the ice safety value.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. By integrating the data collected by the drone's laser scanning camera, visible light camera and infrared camera for detecting power lines into multi-modal data fusion for icing emergency fault judgment and de-icing robot maintenance control, the difficulty of work for on-site personnel is reduced, and the command and dispatch platform realizes unified dispatch management of power line icing emergency faults through satellite communications.

[0041] 2. Multimodal data fusion is achieved by taking into account the data differences between various cameras and the impact of extreme weather on various data collected by cameras, thus achieving accurate judgment on whether to carry out de-icing tasks.

[0042] 3. It is possible to obtain accurate power line ice coverage data in different ways using a variety of cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the general structure of an intelligent inspection and maintenance system for emergency faults of power lines according to an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the general flow of an intelligent inspection and maintenance system for emergency faults of power lines according to an embodiment of the present invention;

[0045] Figure 3 A detailed flow chart of an improved DS evidence theory model of an intelligent inspection and maintenance system for emergency faults of power lines according to an embodiment of the present invention;

[0046] Figure 4 The figure is a schematic diagram of the communication structure of the intelligent inspection and maintenance system for emergency faults of power lines according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0049] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0050] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0051] like Figures 1 to 4 As shown, the present invention provides an intelligent inspection and maintenance system for power line emergency faults, which reduces the difficulty of work for on-site personnel and realizes unified dispatching and management of power line emergency faults by the command and dispatch platform through satellite communications by fusing the collected data of the drone's laser scanning camera, visible light camera and infrared camera for detecting power lines for icing emergency fault judgment and maintenance control of de-icing robots.

[0052] like Figures 1 to 4 As shown, this embodiment proposes an intelligent inspection and maintenance system for emergency faults of power lines. The command and dispatch platform communicates with the laser scanning camera, visible light camera and infrared camera of the inspection end UAV and the de-icing robot set on the power line at the maintenance end through the satellite communication network, including a multi-modal fusion module, a task adjustment module and a task execution judgment module mounted on the command and dispatch platform, wherein:

[0053] The multimodal fusion module is used to generate an ice coverage value of the power line through an improved DS evidence theory model according to the collected data of the laser scanning camera, the visible light camera and the infrared camera, wherein the improved DS evidence theory model reflects the influence of real-time weather data on each collected value and the correlation between each collected data;

[0054] The task adjustment module is used to adjust the deicing power and moving speed of the deicing robot when performing the deicing task according to the ice coverage degree value and the predicted temperature data;

[0055] The task execution judgment module is used to judge whether to control the deicing robot to start the deicing task according to the comparison result of the ice coverage degree value and the ice coverage safety value, and to judge whether to control the deicing robot to end the deicing task according to the change trend of multiple ice coverage degree values.

[0056] The ice coverage safety value is that the ice coverage area in the collection area on the power line is less than 2.3 cubic meters.

[0057] It should be noted that if Figure 4As shown, the satellite communication network described in this embodiment is specifically a satellite communication network, a wireless public network, a power dedicated network, etc., which is realized through "satellite + 4G full-network satellite portable station", "satellite vehicle-mounted station + 4G / 5G base station", "Wo Command Portable Command Box", "satellite equipment, communication base station, voice radio, etc.", "broadband self-organizing network communication system", "narrowband self-organizing network communication system" and other methods to achieve the interconnection, unified management and comprehensive command and dispatch of inspection and maintenance equipment, and then build a highly reliable and feature-rich emergency communication dedicated network in the emergency fault handling of the power grid, realize the positioning, monitoring and visualization of the power line fault site, and meet the requirements of power emergency command work. Specifically, through the camera carried by the drone and the camera of the power line, inspection, monitoring, dispatch, command and other scenes of emergencies or targets in marginal areas are carried out, laying the foundation for more comprehensive and in-depth data analysis and application, and realizing a more efficient production method.

[0058] Furthermore, if Figure 3 As shown, the multimodal fusion module includes a first evidence set generation unit, a coefficient generation unit, a second evidence set generation unit, a third evidence set generation unit and a calculation unit; the first evidence set generation unit is used to generate a first evidence set reflecting the degree of ice coverage according to the numerical gradient corresponding to the collected data; the coefficient generation unit is used to generate a coefficient matrix according to the real-time weather data; the second evidence set generation unit is used to calculate the second evidence set according to the first evidence set and the coefficient matrix; the third evidence set generation unit is used to generate a third evidence set reflecting the correlation between the collected data according to the second evidence set; the calculation unit is used to calculate the ice coverage value by using the third evidence set through the improved DS evidence theory model.

[0059] Specifically, the first evidence set includes first evidence (ma(X), ma(Y), ma(Z), ma(M)), second evidence (mb(X), mb(Y), mb(Z), mb(M)) and third evidence (mc(X), mc(Y), mc(Z), mc(M)), wherein the first to third evidence are elements of the first evidence set, a, b, c represent laser scanning cameras, visible light cameras and infrared cameras, X represents very high ice coverage, Y represents high ice coverage, Z represents low ice coverage, and M represents very low ice coverage. For example, ma(X) represents a trust allocation function for a laser scanning camera with very high ice coverage.

[0060] Specifically, multiple collection points are set in a set area on a power line for camera collection and recognition, and multiple collection data of multiple collection points are matched to gradient intervals, and a trust allocation function is generated according to the number of collection points in each gradient interval. For example, among 50 collection points, after the collection data of the laser scanning camera is processed, the number of collection points in the intervals of very high ice coverage, high ice coverage, low ice coverage, and very low ice coverage is 10, 20, 15, and 5, respectively. According to the construction rules of DS evidence theory, the trust allocation functions in the first evidence generated are 0.2, 0.4, 0.3, and 0.1, respectively.

[0061] It can be understood that the construction rule of the trust allocation function of the DS evidence theory model is:

[0062]

[0063] That is, the trust allocation to the empty set is 0, and the sum of the trust allocations of all elements in the identification framework U (ie, the first evidence set to the third evidence set) is 1.

[0064] Therefore, the above process can avoid different setting interval accuracies caused by different unit magnitudes of different sensors.

[0065] Through the synthesis rule of the DS evidence theory model, the process of generating the ice coverage value according to the first to third evidences of the processed third evidence set is as follows: Identify a finite number of ma, mb, and mc functions in the framework U, then

[0066]

[0067]

[0068] In the formula, A represents one of X, Y, Z, and M, and K is the evidence conflict coefficient, which indicates the degree of conflict between the evidences. is a normalization factor, which can avoid assigning probability to the empty set when data fusion.

[0069] It should be noted that the multimodal fusion module described in this embodiment can process multiple laser scanning cameras, visible light cameras, and multiple infrared cameras of power lines of multiple drones. It only needs to expand mc in the above formulas to mn, thereby realizing the fusion judgment of multi-angle and multi-type camera acquisition data.

[0070] Furthermore, if Figure 3As shown, the real-time weather data includes real-time illumination data and the real-time temperature data, and the coefficient generation unit includes a laser scanning reliability generation subunit, a visible light image reliability generation subunit, an infrared data reliability generation subunit and a coefficient calculation subunit;

[0071] The laser scanning reliability generation subunit is used to generate laser scanning reliability by corresponding the real-time illumination data to the illumination darkness gradient; the visible light image reliability generation subunit is used to generate visible light image reliability by corresponding the real-time illumination data to the illumination brightness gradient; the infrared data reliability is used to generate infrared data reliability by corresponding the real-time temperature data to the temperature extreme cold gradient; the coefficient calculation subunit is used to generate the coefficient matrix by normalizing the laser scanning reliability, the visible light image reliability and the infrared data reliability.

[0072] It is understandable that clouds blocking sunlight will cause the light to dim, which in turn affects the performance of the laser scanning camera, making the laser intensity of the laser scanning camera shining on the target object lower, which in turn affects the reliability of its collected data. In weather with high light intensity, the data collected by the visible light camera may identify the accumulated water on the surface of the power line as ice. In extremely cold temperatures, due to the fixed setting of the infrared camera, its lens may be frosted or frozen due to condensation of water vapor in the air, blocking the emitted infrared radiation, which in turn affects the reliability of its collected data.

[0073] Specifically, the illumination darkness gradient and its corresponding laser scanning reliability are: 1 for 20,000 lux or more, 0.8 for less than 20,000 lux and 15,000 lux, 0.6 for less than 15,000 lux and 8,000 lux, 0.4 for less than 8,000 lux and 5,000 lux, and 0.2 for less than 5,000 lux. The illumination brightness gradient and its corresponding visible light image reliability are: 1 for less than 30,000 lux and 8,000 lux, 0.9 for less than 50,000 lux and 30,000 lux, 0.6 for less than 70,000 lux and 50,000 lux, 0.4 for less than 90,000 lux and 70,000 lux, and 0.2 for greater than 90,000 lux. The reliability of the extreme cold temperature gradient and its corresponding infrared data is: 0.9 for less than or equal to minus 18 degrees Celsius and greater than minus 19 degrees Celsius, 0.7 for less than or equal to minus 19 degrees Celsius and greater than minus 20 degrees Celsius, and 0.5 for less than or equal to minus 20 degrees Celsius.

[0074] Furthermore, if Figure 3As shown, the third evidence set generation unit includes a weight calculation subunit and a comprehensive calculation subunit; the weight calculation subunit is used to calculate the weight corresponding to each collected data through a correlation coefficient formula; the comprehensive calculation subunit normalizes the multiple weights and then performs weighted calculation with the second evidence set to generate the third evidence set.

[0075] In the above scheme, multimodal data fusion is realized by considering the data differences between various cameras and the impact of extreme weather on various data collected by cameras, thereby achieving accurate judgment on whether to carry out de-icing tasks.

[0076] Specifically, the weight calculation process is as follows:

[0077]

[0078]

[0079] In the formula, ρ(m i ,m j ) is the correlation coefficient between two different trust allocation functions, calculated by the correlation coefficient formula, r(m i ) is the trust distribution function m i The support of C(m i ) represents the trust distribution function m i The weights of , where i is a, b, c to n.

[0080] It should be noted that the correlation coefficient is a quantity that represents the degree of correlation between variables in statistics, and is the expected value of the covariate divided by the square root. Therefore, the support and credibility can represent the degree of correlation between the trust allocation functions of each sensor. By normalizing the support, the sum of each support is equal to 1, which conforms to the evidence construction rule of the DS evidence theory model. Through the statistical correlation coefficient and the normalization of the correlation coefficient, the third evidence set can represent the degree of correlation between the trust allocation functions of each sensor, avoiding sensor failures and data conflicts between sensors that lead to inaccurate ice coverage values ​​derived from the DS evidence theory model.

[0081] Furthermore, if Figure 2 As shown, the collected data of the laser scanning camera is point cloud image data; the first evidence set generation unit uses a convolutional neural network model to extract point cloud features of the power line area and the ice-covered area from the point cloud image data, and generates elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the point cloud ice-covered degree gradient interval; wherein the convolutional neural network model includes a point cloud multi-head attention pooling mechanism to process the spatial relationship of three-dimensional point cloud image data.

[0082] It is understandable that the point cloud multi-head attention pooling mechanism (Grouped Vector Attention, GVA) improves parameter efficiency by grouping attention weights in response to the situation where the spatial relationship of 3D point clouds is more complex than the spatial relationship of 2D pixels, thereby overcoming the problem of a sharp increase in the number of parameters as the model deepens and the number of channels increases, and enhancing the model's ability to capture the spatial position information of point clouds.

[0083] Furthermore, the multi-head attention pooling mechanism adopts the PTv2 network framework, whose weight encoding function is generated by grouping linear layers, normalization layers, activation layers and attention fully connected layers to adjust the spatial capture efficiency of three-dimensional point cloud image data.

[0084] Specifically, the expression of the multi-head attention pooling mechanism in the attention fully connected layer is:

[0085]

[0086] In the formula, f i represents the output of the multi-head attention pooling mechanism, M(pi) Represents the scene label corresponding to the point coordinate pi of the point cloud, c and g respectively represent the number of two dimensions of the weight encoding function, v represents the value vector, Wi represents the weight encoding matrix composed of multiple weight encoding functions, and Softmax represents the Softmax activation function.

[0087] Among them, the expression of the weight encoding function that constitutes the weight encoding matrix is:

[0088]

[0089] Among them, w(r) represents the weight encoding function, Linear represents the grouped linear layer, Act represents the activation layer, Norm represents the normalization layer, and ξ(r) represents the grouped learnable parameter matrix.

[0090] Specifically, the convolutional neural network model adopts the Res2-Unet network architecture, uses multi-level 3×3 convolution kernels and LeakyReLU activation, and sets the multi-head attention pooling mechanism between the residual network layer and the weighted layer to focus information recognition on key features and ignore other interference information. The process of the multi-head attention pooling mechanism is specifically as follows:

[0091] M s (F) = σ[MLP(f 7x7 (F avg +F max ))]

[0092] In the formula, F is the input feature map, M s(F) is the output feature map, σ represents the SIGMOD function, MLP represents the perceptron layer, and f 7x7 represents the 7x7 convolution operation of the convolution layer, F avg ,F max They represent average pooling and maximum pooling of the input feature map respectively.

[0093] A convolutional neural network model using the Res2-Unet network architecture is used as a feedforward network, and the multi-head attention pooling mechanism is connected before the feedforward network, so that the multi-head attention pooling mechanism is integrated into the existing Res2-Unet network architecture as a plug-and-play module, so that the multi-size features generated by the multi-head attention pooling mechanism can be input into the convolutional neural network model, so that the ice-covered area and power line area output by the model can better meet the intrinsic characteristics of the three-dimensional point cloud data.

[0094] Specifically, the point cloud ice coverage gradient intervals corresponding to very high ice coverage, high ice coverage, low ice coverage, and very low ice coverage are: ratio greater than or equal to 0.9, ratio less than 0.9 and greater than or equal to 0.6, ratio less than 0.6 and greater than or equal to 0.4, and ratio less than 0.4.

[0095] Furthermore, if Figure 2 As shown, the data collected by the visible light camera is visual image data; the first evidence set generation unit uses the visual image data to identify the power line area and the ice-covered area through a YOLO-based image recognition model, and generates the elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the visible light ice-covered degree gradient interval.

[0096] The YOLO is specifically YOLOv10. During the training process, learning rate decay and data enhancement are used to enhance the generalization ability and robustness of the model. The model is automatically trained by automatically increasing or decreasing hyperparameters with set values ​​until the fitting state is reached for the test phase. The hyperparameters include learning rate and batch size. In order to further improve the generalization ability of the model, we also use random rotation, scaling, cropping and color transformation to expand the sample set, while reducing the risk of overfitting. The main libraries used include PyTorch, NumPy, OpenCV, Pyside6, etc.

[0097] Specifically, the visible light ice coverage gradient ranges corresponding to very high ice coverage, high ice coverage, low ice coverage and very low ice coverage are: ratio greater than or equal to 0.8, ratio less than 0.8 and greater than or equal to 0.5, ratio less than 0.5 and greater than or equal to 0.3, and ratio less than 0.3.

[0098] Furthermore, if Figure 2As shown, the collected data of the infrared camera is thermal image data; the first evidence set generation unit generates elements of the first evidence set according to the temperatures of multiple collection points on the surface of the power line in the thermal image data corresponding to the infrared icing degree gradient interval.

[0099] Specifically, the preprocessed thermal image data is used to generate a binary thermal image through the OTSU (Otsu algorithm). Due to the large temperature difference between the power line area and the non-power line area, the power line area and the collection points set in the area can be obtained only by extracting and fitting the power line boundary.

[0100] Specifically, Figure 2 As shown, the infrared ice coverage gradient ranges corresponding to very high ice coverage, high ice coverage, low ice coverage and very low ice coverage are: regional average temperature is less than minus 15 degrees Celsius, regional average temperature is greater than or equal to minus 15 degrees Celsius and less than minus 5 degrees Celsius, regional average temperature is greater than or equal to minus 5 degrees Celsius and less than minus 4 degrees Celsius, and regional average temperature is greater than or equal to minus 4 degrees Celsius.

[0101] In the above scheme, it is achieved to obtain accurate power line ice coverage condition collection values ​​through different methods corresponding to multiple cameras.

[0102] Furthermore, if Figure 2 As shown, the task adjustment module includes a deicing power calculation unit and a moving speed calculation unit; the deicing power calculation unit is used to calculate the deicing power according to the difference between the icing degree value and the icing standard value; the moving speed calculation unit is used to generate a predicted icing generation rate according to a plurality of the predicted air temperature data, and calculate the moving speed according to the predicted icing generation rate and the icing degree value.

[0103] Specifically, the standard ice coverage value is the amount of ice removed by the de-icing robot when operating at standard power. The amount of ice removed is the moving speed multiplied by the maximum de-icing thickness, for example, 1.5 cubic meters per minute. Minutes are omitted when calculating the difference and are regarded as 1.5 cubic meters. The difference is divided by the standard ice coverage value and multiplied by the standard power to obtain the de-icing power.

[0104] Specifically, the predicted ice formation rate corresponding to the temperature, current wind speed and current humidity is obtained by weighting and averaging multiple predicted air temperature data in the current time period. For example, the weight of the predicted air temperature data in the last minute is set to 0.6, the weight of the predicted air temperature data in the last 2 minutes is set to 0.3, and the weight of the predicted air temperature data in the last 3 minutes is set to 0.1. The weighted average is obtained. For example, if the average value is minus 11.5 degrees Celsius, the corresponding ice formation rate is determined according to minus 11 degrees Celsius, the current wind speed and the current humidity. This process is a conventional process in meteorology and will not be repeated here. The moving speed calculation unit is used to add the predicted ice formation rate and the ice degree value and then multiply it by the conversion coefficient to generate the moving speed. For example, the added value is mostly 1.3 to 2.5 (in cubic meters). Multiplying it by 2.1, the moving speed is 2.7 to 5.3 in meters per minute.

[0105] Furthermore, if Figure 2 As shown, the task execution judgment module is used to calculate an ice degree prediction value through a regression model using multiple ice degree values, and to determine whether to control the deicing robot to end the deicing task based on the comparison result between the ice degree prediction value and the ice safety value.

[0106] The regression model can be expressed as:

[0107] P = softmax(W1x1+W2x2+B)

[0108] Where P is the predicted value of ice coverage output by the logistic regression model, W1, W2, and B are three training parameters, and x1 and x2 represent two predicted values ​​of ice coverage. Softmax is an activation function that can form a probability distribution of the input variables, which is the predicted value of ice coverage.

[0109] If the predicted ice coverage value is greater than the ice coverage safety value, the deicing robot is controlled to end the deicing task, otherwise no command signal for ending the deicing robot is sent to the deicing robot, and the deicing task is not controlled to end. The ice coverage safety value is preferably 18 mm, which can be adjusted according to the actual situation of the power line.

[0110] It can be understood that this embodiment reduces the difficulty of work for on-site personnel by fusing the collected data of the drone's laser scanning camera, visible light camera, and infrared camera for detecting power lines for emergency fault judgment of ice coverage and maintenance control of de-icing robots, and realizes unified dispatch management of emergency faults of power line ice coverage through the satellite communication command and dispatch platform. It realizes multimodal data fusion that takes into account the data differences between various cameras and the degree of influence of extreme weather on various data collected by the cameras, and realizes accurate judgment on whether to carry out de-icing tasks. It realizes obtaining accurate power line ice coverage collection values ​​in different ways corresponding to multiple cameras.

[0111] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent inspection and maintenance system for emergency power line faults, characterized in that: The command and dispatch platform communicates with the laser scanning camera, visible light camera and infrared camera of the inspection end UAV and the de-icing robot installed on the power line at the maintenance end through the satellite communication network, including a multi-modal fusion module, a task adjustment module and a task execution judgment module installed on the command and dispatch platform, wherein: The multimodal fusion module is used to generate an ice coverage value of the power line through an improved DS evidence theory model according to the collected data of the laser scanning camera, the visible light camera and the infrared camera, wherein the improved DS evidence theory model reflects the influence of real-time weather data on each collected value and the correlation between each collected data; The task adjustment module is used to adjust the deicing power and moving speed of the deicing robot when performing the deicing task according to the ice coverage degree value and the predicted temperature data; The task execution judgment module is used to judge whether to control the deicing robot to start the deicing task according to the comparison result of the ice coverage degree value and the ice coverage safety value, and to judge whether to control the deicing robot to end the deicing task according to the change trend of multiple ice coverage degree values.

2. The intelligent inspection and maintenance system for emergency faults of power lines according to claim 1 is characterized in that: The multimodal fusion module includes a first evidence set generation unit, a coefficient generation unit, a second evidence set generation unit, a third evidence set generation unit and a calculation unit; The first evidence set generating unit is used to generate a first evidence set reflecting the degree of ice coverage according to the numerical gradient corresponding to the collected data; The coefficient generating unit is used to generate a coefficient matrix according to the real-time weather data; The second evidence set generating unit is used to calculate a second evidence set according to the first evidence set and the coefficient matrix; The third evidence set generating unit is used to generate a third evidence set reflecting the correlation between each collected data according to the second evidence set; The calculation unit is used to calculate the ice coverage degree value by using the third evidence set through the improved DS evidence theory model.

3. The intelligent inspection and maintenance system for emergency power line failure according to claim 2 is characterized in that: The real-time weather data includes real-time illumination data and real-time temperature data, and the coefficient generation unit includes a laser scanning reliability generation subunit, a visible light image reliability generation subunit, an infrared data reliability generation subunit and a coefficient calculation subunit; The laser scanning reliability generating subunit is used to generate laser scanning reliability according to the illumination dark gradient corresponding to the real-time illumination data; The visible light image reliability generating subunit is used to generate visible light image reliability by corresponding the real-time illumination data to the illumination brightness gradient; The infrared data reliability is used to generate the infrared data reliability by corresponding the real-time air temperature data to the extreme cold temperature gradient; The coefficient calculation subunit is used to generate the coefficient matrix by normalizing the laser scanning reliability, the visible light image reliability and the infrared data reliability.

4. The intelligent inspection and maintenance system for emergency power line failure according to claim 3 is characterized in that: The third evidence set generation unit includes a weight calculation subunit and a comprehensive calculation subunit; The weight calculation subunit is used to calculate the weight corresponding to each collected data through the correlation coefficient formula; The comprehensive calculation subunit performs normalization operation on the multiple weights and then performs weighted calculation on the second evidence set to generate the third evidence set.

5. The intelligent inspection and maintenance system for emergency faults of power lines according to claim 2 is characterized in that: The data collected by the laser scanning camera is point cloud image data; The first evidence set generating unit extracts point cloud features of the power line area and the ice-covered area from the point cloud image data through a convolutional neural network model, and generates elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the point cloud ice-covered degree gradient interval; Among them, the convolutional neural network model includes a point cloud multi-head attention pooling mechanism to process the spatial relationship of three-dimensional point cloud image data.

6. The intelligent inspection and maintenance system for emergency power line faults according to claim 5, characterized in that: The multi-head attention pooling mechanism adopts the PTv2 network framework, and its weight encoding function is generated by grouping linear layers, normalization layers, activation layers and attention fully connected layers to adjust the spatial capture efficiency of three-dimensional point cloud image data.

7. The intelligent inspection and maintenance system for emergency faults of power lines according to claim 2 is characterized in that: The data collected by the visible light camera is visual image data; The first evidence set generating unit is used to identify the power line area and the ice-covered area of ​​the visual image data through a YOLO-based image recognition model, and generate elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the visible light ice-covered degree gradient interval.

8. The intelligent inspection and maintenance system for emergency power line failure according to claim 2 is characterized in that: The data collected by the infrared camera is thermal image data; The first evidence set generating unit is used to generate elements of the first evidence set according to the temperatures of multiple acquisition points on the surface of the power line in the thermal image data corresponding to the infrared ice coverage gradient interval.

9. The intelligent inspection and maintenance system for emergency faults of power lines according to any one of claims 1 to 8, characterized in that: The task adjustment module includes a deicing power calculation unit and a moving speed calculation unit; The deicing power calculation unit is used to calculate the deicing power according to the difference between the ice coverage degree value and the ice coverage standard value; The moving speed calculation unit is connected to the deicing power calculation unit, and is used to generate a predicted ice formation rate according to the plurality of predicted air temperature data, and calculate the moving speed according to the predicted ice formation rate and the ice degree value.

10. The intelligent inspection and maintenance system for emergency faults of power lines according to any one of claims 1 to 8, characterized in that: The task execution judgment module is used to calculate an ice degree prediction value through a regression model using multiple ice degree values, and to judge whether to control the deicing robot to end the deicing task according to the comparison result between the ice degree prediction value and the ice safety value.

Images