An intelligent inspection and maintenance system for power line emergency faults

Abstract

The present invention relates to the field of data processing technology, specifically to an intelligent inspection and maintenance system for power line emergency faults. The system comprises a multimodal fusion module, which generates power line ice coverage values based on data collected by a laser scanning camera, a visible light camera, and an infrared camera using an improved DS evidence theory model. The improved DS evidence theory model reflects the influence of real-time weather data on each collected value and the correlation between each collected data; a task adjustment module, which adjusts the deicing power and movement speed of a deicing robot performing a deicing task based on the ice coverage value and predicted temperature data; and a task execution judgment module, which determines whether to control the deicing robot to start the deicing task based on a comparison result of the ice coverage value and an ice safety value, and determines whether to control the deicing robot to end the deicing task based on the changing trends of multiple ice coverage values. The present invention implements unified dispatching and management of power line icing emergency faults by a command and dispatch platform via satellite communications.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an intelligent inspection and maintenance system for emergency faults of power lines. Background Art

[0002] In the event of a power line emergency failure, ice covering the surface of the power line increases the weight of the line, causing the line to shake, increase the impact load of the line, and reduce the insulation performance of the line, greatly increasing the safety risk of the power line.

[0003] In extreme environments where power lines are covered in ice, manual de-icing by climbing a tower is less safe. Therefore, de-icing is primarily performed using hot melt technology from de-icing robots mounted on the power lines. During this process, on-site personnel must use drone cameras to observe the extent of ice on the power lines from a distance and then set the de-icing robot's operating parameters. Specifically, on-site personnel must operate the drone to prevent it from falling while observing the drone's captured images and adjusting the de-icing robot's operating parameters. This is particularly challenging in extreme conditions such as strong winds and heavy snow.

[0004] my country has also made significant progress in developing its BeiDou-3 satellite system. By the end of 2018, the basic BeiDou-3 system was completed and providing global services. This has advanced the application of BeiDou in power emergency response systems, achieving comprehensive communication coverage. The BeiDou system now covers all regions of my country, enabling signals to be received in remote mountainous areas or desert regions. The industry-specific satellite self-organizing network system, through multi-hop relay, effectively addresses emergency communication challenges in harsh environments or emergencies. This platform, characterized by its ease of deployment, flexibility, and simplicity of operation, can be widely applied to specialized areas and regions within the power industry, facilitating communication for production operations in these challenging scenarios.

[0005] Therefore, how to use satellite communications so that on-site personnel only need to control the drone, while the command and dispatch platform can use the drone and infrared cameras that detect power lines to perform multimodal data fusion to perform icing emergency fault diagnosis and maintenance control of de-icing robots is a technical problem that needs to be solved. Summary of the Invention

[0006] To this end, the present invention provides an intelligent inspection and maintenance system for power line emergency faults. By performing multimodal data fusion on the data collected by the drone's laser scanning camera, visible light camera and infrared camera for detecting power lines, the system can judge icing emergency faults and control the maintenance of de-icing robots, thereby reducing the difficulty of work for on-site personnel and realizing unified dispatching and management of power line icing emergency faults by the command and dispatch platform through satellite communications.

[0007] To achieve the above objectives, the present invention proposes an intelligent inspection and maintenance system for power line emergency faults. The command and dispatch platform communicates with the laser scanning camera, visible light camera and infrared camera of the inspection-end UAV and the de-icing robot installed on the power line at the maintenance end through a satellite communication network. The system includes a multimodal fusion module, a task adjustment module and a task execution judgment module installed on the command and dispatch platform, wherein:

[0008] The multimodal fusion module is configured to generate an ice coverage value for the power line based on the collected data from the laser scanning camera, the visible light camera, and the infrared camera using an improved DS evidence theory model, wherein the improved DS evidence theory model reflects the influence of real-time weather data on each collected value and the correlation between each collected data;

[0009] The task adjustment module is used to adjust the deicing power and movement speed of the deicing robot when performing the deicing task according to the ice coverage value and the predicted temperature data;

[0010] The task execution judgment module is used to judge whether to control the deicing robot to start the deicing task based on the comparison result of the ice coverage degree value and the ice coverage safety value, and to judge whether to control the deicing robot to end the deicing task based on the change trends of multiple ice coverage degree values.

[0011] Furthermore, the multimodal fusion module includes a first evidence set generation unit, a coefficient generation unit, a second evidence set generation unit, a third evidence set generation unit and a calculation unit;

[0012] The first evidence set generating unit is configured to generate a first evidence set reflecting the degree of ice coverage according to the numerical gradient corresponding to the collected data;

[0013] The coefficient generating unit is used to generate a coefficient matrix according to the real-time weather data;

[0014] The second evidence set generating unit is configured to calculate a second evidence set based on the first evidence set and the coefficient matrix;

[0015] The third evidence set generating unit is configured to generate a third evidence set reflecting the degree of association between each collected data based on the second evidence set;

[0016] The calculation unit is used to calculate the ice coverage value by using the improved DS evidence theory model to apply the third evidence set.

[0017] Furthermore, the real-time weather data includes real-time illumination data and real-time temperature data, and the coefficient generation unit includes a laser scanning reliability generation subunit, a visible light image reliability generation subunit, an infrared data reliability generation subunit, and a coefficient calculation subunit;

[0018] The laser scanning reliability generating subunit is used to generate laser scanning reliability by corresponding the real-time illumination data to the illumination-darkness gradient;

[0019] The visible light image reliability generating subunit is configured to generate visible light image reliability by corresponding the real-time illumination data to the illumination brightness gradient;

[0020] The infrared data reliability is used to generate the infrared data reliability by corresponding the real-time temperature data to the extreme cold temperature gradient;

[0021] The coefficient calculation subunit is used to generate the coefficient matrix by normalizing the laser scanning reliability, the visible light image reliability and the infrared data reliability.

[0022] Furthermore, the third evidence set generation unit includes a weight calculation subunit and a comprehensive calculation subunit;

[0023] The weight calculation subunit is used to calculate the weight corresponding to each collected data through the correlation coefficient formula;

[0024] The comprehensive calculation subunit performs normalization operation on the multiple weights and then performs weighted calculation on the second evidence set to generate the third evidence set.

[0025] In the above scheme, multimodal data fusion is realized by considering the data differences between various cameras and the impact of extreme weather on the various data collected by the cameras, thereby achieving accurate judgment on whether to carry out the de-icing task.

[0026] Furthermore, the data collected by the laser scanning camera is point cloud image data;

[0027] The first evidence set generating unit extracts point cloud features of the power line area and the ice-covered area from the point cloud image data through a convolutional neural network model, and generates elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the point cloud ice coverage gradient interval;

[0028] Among them, the convolutional neural network model includes a point cloud multi-head attention pooling mechanism to process the spatial relationship of three-dimensional point cloud image data.

[0029] Furthermore, the multi-head attention pooling mechanism adopts the PTv2 network framework, whose weight encoding function is generated by grouping linear layers, normalization layers, activation layers and attention fully connected layers to adjust the spatial capture efficiency of three-dimensional point cloud image data.

[0030] Furthermore, the data collected by the visible light camera is visual image data;

[0031] The first evidence set generation unit is used to identify the power line area and the ice-covered area in the visual image data through a YOLO-based image recognition model, and generate elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the visible light ice coverage gradient interval.

[0032] Furthermore, the data collected by the infrared camera is thermal image data;

[0033] The first evidence set generating unit is used to generate elements of the first evidence set according to the temperatures of multiple acquisition points on the surface of the power line in the thermal image data corresponding to the infrared icing degree gradient interval.

[0034] In the above solution, accurate power line icing condition data can be acquired through different methods corresponding to multiple cameras.

[0035] Furthermore, the task adjustment module includes a deicing power calculation unit and a moving speed calculation unit;

[0036] The deicing power calculation unit is used to calculate the deicing power according to the difference between the ice coverage value and the ice coverage standard value;

[0037] The moving speed calculation unit is connected to the deicing power calculation unit, and is used to generate a predicted ice formation rate according to the plurality of predicted air temperature data, and calculate the moving speed according to the predicted ice formation rate and the ice degree value.

[0038] Furthermore, the task execution judgment module is used to calculate an ice coverage prediction value through a regression model using multiple ice coverage values, and to determine whether to control the deicing robot to end the deicing task based on the comparison result of the ice coverage prediction value and the ice coverage safety value.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] 1. By integrating the data collected by the drone's laser scanning camera, visible light camera, and infrared camera for detecting power lines into multimodal data fusion, emergency icing fault diagnosis and de-icing robot maintenance control are carried out, the work difficulty of on-site personnel is reduced, and the command and dispatch platform realizes unified dispatch and management of power line icing emergency faults through satellite communication.

[0041] 2. Multimodal data fusion is achieved by considering the data differences between various cameras and the impact of extreme weather on the various data collected by the cameras, making it possible to accurately judge whether to carry out the de-icing task.

[0042] 3. It is possible to obtain accurate power line ice coverage data in different ways using multiple cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Schematic diagram of the general structure of an intelligent inspection and maintenance system for power line emergency faults according to an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the general flow of an intelligent inspection and maintenance system for power line emergency faults according to an embodiment of the present invention;

[0045] Figure 3 Detailed flow chart of the improved DS evidence theory model for the intelligent inspection and maintenance system for power line emergency faults according to an embodiment of the present invention;

[0046] Figure 4 The figure is a schematic diagram of the communication structure of the intelligent inspection and maintenance system for emergency power line faults according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0048] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0049] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0050] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0051] like Figures 1 to 4 As shown, the present invention provides an intelligent inspection and maintenance system for power line emergency faults. By performing multimodal data fusion on the data collected by the drone's laser scanning camera, visible light camera and infrared camera for detecting power lines, the system can determine icing emergency faults and control the maintenance of de-icing robots. This reduces the difficulty of work for on-site personnel and realizes the unified dispatching and management of power line icing emergency faults by the command and dispatch platform through satellite communications.

[0052] like Figures 1 to 4 As shown, this embodiment proposes an intelligent inspection and maintenance system for power line emergency faults. The command and dispatch platform communicates with the laser scanning camera and visible light camera of the inspection-end drone and the infrared camera of the inspection-end power line, as well as the de-icing robot installed on the maintenance end through a satellite communication network. The system includes a multimodal fusion module, a task adjustment module, and a task execution judgment module installed on the command and dispatch platform, wherein:

[0053] The multimodal fusion module is configured to generate an ice coverage value for the power line based on the collected data from the laser scanning camera, the visible light camera, and the infrared camera using an improved DS evidence theory model, wherein the improved DS evidence theory model reflects the influence of real-time weather data on each collected value and the correlation between each collected data;

[0054] The task adjustment module is used to adjust the deicing power and movement speed of the deicing robot when performing the deicing task according to the ice coverage value and the predicted temperature data;

[0055] The task execution judgment module is used to judge whether to control the deicing robot to start the deicing task based on the comparison result of the ice coverage degree value and the ice coverage safety value, and to judge whether to control the deicing robot to end the deicing task based on the change trends of multiple ice coverage degree values.

[0056] The ice coverage safety value is that the ice coverage area in the collection area on the power line is less than 2.3 cubic meters.

[0057] It should be noted that if Figure 4As shown, the satellite communication network described in this embodiment is specifically a satellite communication network, a wireless public network, a power-dedicated network, etc., which is implemented through "satellite + 4G full-network satellite portable station", "satellite vehicle-mounted station + 4G / 5G base station", "Wo Command Portable Command Box", "satellite equipment, communication base station, voice radio, etc.", "broadband ad hoc communication system", "narrowband ad hoc communication system", etc., to achieve interconnection, unified management and comprehensive command and dispatch of inspection-end and maintenance-end equipment, and thus build a highly reliable and feature-rich emergency communication dedicated network for power grid emergency fault handling, realize the positioning, monitoring and visualization of power line fault sites, and meet the requirements of power emergency command work. Specifically, through the use of cameras carried by drones and cameras on power lines, inspection, monitoring, dispatching, command and other scenarios of emergencies or targets in marginal areas are carried out, laying the foundation for more comprehensive and in-depth data analysis and application, and realizing a more efficient production method.

[0058] Furthermore, if Figure 3 As shown, the multimodal fusion module includes a first evidence set generation unit, a coefficient generation unit, a second evidence set generation unit, a third evidence set generation unit and a calculation unit; the first evidence set generation unit is used to generate a first evidence set for reflecting the degree of icing according to the numerical gradient corresponding to the collected data; the coefficient generation unit is used to generate a coefficient matrix according to the real-time weather data; the second evidence set generation unit is used to calculate the second evidence set according to the first evidence set and the coefficient matrix; the third evidence set generation unit is used to generate a third evidence set reflecting the correlation between the collected data according to the second evidence set; the calculation unit is used to calculate the icing degree value by using the third evidence set through the improved DS evidence theory model.

[0059] Specifically, the first evidence set includes first evidence (ma(X), ma(Y), ma(Z), ma(M)), second evidence (mb(X), mb(Y), mb(Z), mb(M)) and third evidence (mc(X), mc(Y), mc(Z), mc(M)), wherein the first to third evidence are elements of the first evidence set, a, b, and c represent a laser scanning camera, a visible light camera, and an infrared camera, X represents a very high degree of ice coverage, Y represents a relatively high degree of ice coverage, Z represents a relatively low degree of ice coverage, and M represents a very low degree of ice coverage. For example, ma(X) represents a trust distribution function indicating that the laser scanning camera has a very high degree of ice coverage.

[0060] Specifically, multiple collection points are set up in a designated area on a power line for camera acquisition and recognition. The collected data from these collection points is then mapped to gradient intervals, and a trust allocation function is generated based on the number of collection points in each gradient interval. For example, among 50 collection points, after processing the data collected by the laser scanning camera, the number of collection points in the very high ice coverage interval, high ice coverage interval, low ice coverage interval, and very low ice coverage interval is 10, 20, 15, and 5, respectively. Based on the construction rules of DS evidence theory, the trust allocation functions in the generated first evidence are 0.2, 0.4, 0.3, and 0.1, respectively.

[0061] It can be understood that the construction rule of the trust allocation function of the DS evidence theory model is:

[0062]

[0063] That is, the trust allocation to the empty set is 0, and the sum of the trust allocations of all elements in the identification framework U (ie, the first to third evidence sets) is 1.

[0064] Therefore, the above process can avoid different unit magnitudes of different sensors causing different setting interval accuracies.

[0065] Through the synthesis rule of the DS evidence theory model, the process of generating the ice coverage value based on the first to third evidence of the processed third evidence set is as follows: Identify a finite number of ma, mb, and mc functions in the framework U, then

[0066]

[0067]

[0068] In the formula, A represents one of X, Y, Z, and M, and K is the evidence conflict coefficient, which represents the degree of conflict between the evidences. is a normalization factor that can avoid assigning probability to the empty set when fusion of data.

[0069] It should be noted that the multimodal fusion module described in this embodiment can process multiple laser scanning cameras, visible light cameras, and multiple infrared cameras of power lines of multiple drones. It is only necessary to expand mc in the above formulas to mn, thereby realizing the fusion judgment of multi-angle and multi-type camera acquisition data.

[0070] Furthermore, if Figure 3As shown, the real-time weather data includes real-time illumination data and real-time temperature data, and the coefficient generation unit includes a laser scanning reliability generation subunit, a visible light image reliability generation subunit, an infrared data reliability generation subunit, and a coefficient calculation subunit;

[0071] The laser scanning reliability generation subunit is used to generate laser scanning reliability by corresponding the real-time illumination data to the illumination darkness gradient; the visible light image reliability generation subunit is used to generate visible light image reliability by corresponding the real-time illumination data to the illumination brightness gradient; the infrared data reliability is used to generate infrared data reliability by corresponding the real-time temperature data to the temperature extreme cold gradient; the coefficient calculation subunit is used to generate the coefficient matrix by normalizing the laser scanning reliability, the visible light image reliability and the infrared data reliability.

[0072] Understandably, clouds blocking sunlight can dim the light, affecting the performance of laser scanning cameras and reducing the intensity of the laser beam hitting the target, thus impacting the reliability of the collected data. Furthermore, in high-intensity weather, data collected by visible light cameras may misidentify accumulated water on the surface of power lines as ice. In extremely cold temperatures, due to the fixed position of infrared cameras, their lenses may condense and freeze due to frost or ice, blocking the emitted infrared radiation and thus affecting the reliability of the collected data.

[0073] Specifically, the illumination darkness gradient and its corresponding laser scanning reliability are: 1 for 20,000 lux or greater, 0.8 for less than 20,000 lux and 15,000 lux or greater, 0.6 for less than 15,000 lux and 8,000 lux or greater, 0.4 for less than 8,000 lux and 5,000 lux, and 0.2 for less than 5,000 lux. The illumination brightness gradient and its corresponding visible light image reliability are: 1 for less than 30,000 lux and 8,000 lux or greater, 0.9 for less than 50,000 lux and 30,000 lux or greater, 0.6 for less than 70,000 lux and 50,000 lux or greater, 0.4 for less than 90,000 lux and 70,000 lux or greater, and 0.2 for greater than 90,000 lux. The reliability of the extreme cold temperature gradient and its corresponding infrared data is: 0.9 for less than or equal to minus 18 degrees Celsius and greater than minus 19 degrees Celsius, 0.7 for less than or equal to minus 19 degrees Celsius and greater than minus 20 degrees Celsius, and 0.5 for less than or equal to minus 20 degrees Celsius.

[0074] Furthermore, if Figure 3As shown, the third evidence set generation unit includes a weight calculation subunit and a comprehensive calculation subunit; the weight calculation subunit is used to calculate the weight corresponding to each collected data through the correlation coefficient formula; the comprehensive calculation subunit normalizes the multiple weights and then performs weighted calculation with the second evidence set to generate the third evidence set.

[0075] In the above scheme, multimodal data fusion is realized by considering the data differences between various cameras and the impact of extreme weather on the various data collected by the cameras, thereby achieving accurate judgment on whether to carry out the de-icing task.

[0076] Specifically, the weight calculation process is as follows:

[0077]

[0078]

[0079] Where, ρ(m i ,m j ) is the correlation coefficient between two different trust allocation functions, calculated by the correlation coefficient formula, r(m i ) is the trust distribution function m i The support of C(m i ) represents the trust distribution function m i where i is a, b, c to n.

[0080] It should be noted that the correlation coefficient is a statistical measure that represents the degree of correlation between variables. It is the expected value of the covariate divided by the square root. Therefore, the support and credibility can represent the degree of correlation between the trust allocation functions of each sensor. By normalizing the support, the sum of the individual supports is equal to 1, which conforms to the evidence construction rules of the DS evidence theory model. Through the statistical correlation coefficient and normalization of the correlation coefficient, the third evidence set can represent the degree of correlation between the trust allocation functions of each sensor, avoiding inaccurate ice coverage values derived from the DS evidence theory model due to sensor failures and data conflicts between sensors.

[0081] Furthermore, if Figure 2 As shown, the collected data of the laser scanning camera is point cloud image data; the first evidence set generation unit uses a convolutional neural network model to extract point cloud features of the power line area and the ice-covered area from the point cloud image data, and generates elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the point cloud ice coverage gradient interval; wherein the convolutional neural network model includes a point cloud multi-head attention pooling mechanism to process the spatial relationship of three-dimensional point cloud image data.

[0082] It is understandable that the point cloud multi-head attention pooling mechanism (Grouped Vector Attention, GVA) improves parameter efficiency by grouping attention weights in response to the situation where the spatial relationship of 3D point clouds is more complex than the spatial relationship of 2D pixels, thereby overcoming the problem of a sharp increase in the number of parameters as the model deepens and the number of channels increases, and enhancing the model's ability to capture the spatial position information of point clouds.

[0083] Furthermore, the multi-head attention pooling mechanism adopts the PTv2 network framework, whose weight encoding function is generated by grouping linear layers, normalization layers, activation layers and attention fully connected layers to adjust the spatial capture efficiency of three-dimensional point cloud image data.

[0084] Specifically, the expression of the multi-head attention pooling mechanism in the attention fully connected layer is:

[0085]

[0086] Where, f i represents the output of the multi-head attention pooling mechanism, M(pi) Represents the scene label corresponding to the point coordinate pi of the point cloud, c and g respectively represent the number of two dimensions of the weight encoding function, v is represented as a value vector, Wi represents the weight encoding matrix composed of multiple weight encoding functions, and Softmax represents the Softmax activation function.

[0087] Among them, the expression of the weight encoding function that constitutes the weight encoding matrix is:

[0088]

[0089] Among them, w(r) represents the weight encoding function, Linear represents the grouped linear layer, Act activation layer, Norm represents the normalization layer, and ξ(r) represents the grouped learnable parameter matrix.

[0090] Specifically, the convolutional neural network model adopts the Res2-Unet network architecture, uses multi-level 3×3 convolution kernels and LeakyReLU activation, and sets the multi-head attention pooling mechanism between the residual network layer and the weighted layer to focus information recognition on key features and ignore other interference information. The process of the multi-head attention pooling mechanism is specifically as follows:

[0091] M s (F) = σ[MLP(f 7x7 (F avg +F max ))]

[0092] Where F is the input feature map, M s(F) is the output feature map, σ represents the SIGMOD function, MLP represents the perceptron layer, and f 7x7 Represents the 7x7 convolution operation of the convolution layer, F avg ,F max They represent average pooling and maximum pooling of the input feature map respectively.

[0093] A convolutional neural network model using the Res2-Unet network architecture is used as the feedforward network, and the multi-head attention pooling mechanism is connected before the feedforward network, so that the multi-head attention pooling mechanism is integrated into the existing Res2-Unet network architecture as a plug-and-play module, so that the multi-scale features generated by the multi-head attention pooling mechanism can be input into the convolutional neural network model, so that the ice-covered area and power line area output by it can better meet the intrinsic characteristics of three-dimensional point cloud data.

[0094] Specifically, the point cloud ice coverage gradient intervals corresponding to very high ice coverage, high ice coverage, low ice coverage, and very low ice coverage are: ratio greater than or equal to 0.9, ratio less than 0.9 and greater than or equal to 0.6, ratio less than 0.6 and greater than or equal to 0.4, and ratio less than 0.4.

[0095] Furthermore, if Figure 2 As shown, the data collected by the visible light camera is visual image data; the first evidence set generation unit uses the visual image data to identify the power line area and the ice-covered area through the YOLO-based image recognition model, and generates the elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the visible light ice-covered degree gradient interval.

[0096] Specifically, YOLO, specifically YOLOv10, employs learning rate decay and data augmentation during training to enhance the model's generalization and robustness. The model is automatically trained by automatically increasing or decreasing hyperparameters, such as learning rate and batch size, by set values until a good fit is achieved and the testing phase begins. To further improve the model's generalization, we also employ sample augmentation techniques such as random rotation, scaling, cropping, and color transformation, while mitigating the risk of overfitting. Key libraries used include PyTorch, NumPy, OpenCV, and Pyside6.

[0097] Specifically, the visible light icing degree gradient intervals corresponding to very high icing degree, relatively high icing degree, relatively low icing degree, and very low icing degree are: ratio greater than or equal to 0.8, ratio less than 0.8 and greater than or equal to 0.5, ratio less than 0.5 and greater than or equal to 0.3, and ratio less than 0.3.

[0098] Furthermore, if Figure 2As shown, the data collected by the infrared camera is thermal image data; the first evidence set generation unit generates elements of the first evidence set according to the temperatures of multiple collection points on the surface of the power line in the thermal image data corresponding to the infrared icing degree gradient interval.

[0099] Specifically, the preprocessed thermal image data is used to generate a binary thermal image through the OTSU (Otsu algorithm). Since the temperature difference between the power line area and the non-power line area is large, the power line area and the collection points set in the area can be obtained only by extracting and fitting the power line boundaries.

[0100] Specifically, if Figure 2 As shown, the infrared ice coverage gradient ranges corresponding to very high ice coverage, high ice coverage, low ice coverage and very low ice coverage are: regional average temperature is less than minus 15 degrees Celsius, regional average temperature is greater than or equal to minus 15 degrees Celsius and less than minus 5 degrees Celsius, regional average temperature is greater than or equal to minus 5 degrees Celsius and less than minus 4 degrees Celsius, and regional average temperature is greater than or equal to minus 4 degrees Celsius.

[0101] In the above solution, accurate power line icing condition data can be acquired through different methods corresponding to multiple cameras.

[0102] Furthermore, if Figure 2 As shown, the task adjustment module includes a deicing power calculation unit and a moving speed calculation unit; the deicing power calculation unit is used to calculate the deicing power according to the difference between the icing degree value and the icing standard value; the moving speed calculation unit is used to generate a predicted icing generation rate according to a plurality of predicted temperature data, and calculate the moving speed according to the predicted icing generation rate and the icing degree value.

[0103] Specifically, the standard ice coverage value is the amount of ice removed by the de-icing robot under standard power operation. The de-icing amount is the moving speed multiplied by the maximum de-icing thickness, for example, 1.5 cubic meters / minute. Minutes are omitted when calculating the difference and it is regarded as 1.5 cubic meters. The difference is divided by the standard ice coverage value and multiplied by the standard power to obtain the de-icing power.

[0104] Specifically, the predicted ice formation rate corresponding to the temperature, current wind speed, and current humidity is obtained by weighting and averaging multiple predicted temperature data within the current time period. For example, the weight of the predicted temperature data for the most recent minute is set to 0.6, the weight of the predicted temperature data for the most recent two minutes is set to 0.3, and the weight of the predicted temperature data for the most recent three minutes is set to 0.1. The weighted average is calculated. For example, if the average value is negative 11.5 degrees Celsius, the corresponding ice formation rate is determined based on negative 11 degrees Celsius, the current wind speed, and the current humidity. This process is a conventional meteorological process and will not be repeated here. The moving speed calculation unit is used to add the predicted ice formation rate and the ice degree value and multiply it by a conversion coefficient to generate the moving speed. For example, the added value is mostly 1.3 to 2.5 (in cubic meters). Multiplying it by 2.1 results in a moving speed of 2.7 to 5.3 (in meters per minute).

[0105] Furthermore, if Figure 2 As shown, the task execution judgment module is used to calculate the ice coverage prediction value through a regression model using multiple ice coverage values, and to determine whether to control the de-icing robot to end the de-icing task based on the comparison result of the ice coverage prediction value and the ice coverage safety value.

[0106] The regression model can be expressed as:

[0107] P = softmax(W1x1+W2x2+B)

[0108] Where P is the predicted ice extent output by the logistic regression model, W1, W2, and B are the three training parameters, and x1 and x2 represent the two predicted ice extent values. Softmax is an activation function that forms a probability distribution of the input variables, which is the predicted ice extent value.

[0109] If the predicted ice coverage value is greater than the ice coverage safety value, the de-icing robot is controlled to terminate the de-icing task. Otherwise, no command signal is sent to the de-icing robot to terminate the de-icing task, and the de-icing task is not terminated. The ice coverage safety value is preferably 18 mm and can be adjusted according to the actual situation of the power line.

[0110] As can be understood, this embodiment reduces the workload of on-site personnel by integrating multimodal data from the drone's laser scanning camera, visible light camera, and infrared camera for power line monitoring to determine emergency icing faults and control de-icing robot maintenance. This allows for unified dispatch and management of power line icing emergency faults via a satellite communications command and dispatch platform. This multimodal data fusion, which accounts for data differences between various cameras and the impact of extreme weather on the various camera data collected, enables accurate judgment on whether to conduct de-icing tasks. Accurate power line icing condition data can be obtained through different methods corresponding to multiple cameras.

[0111] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0112] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that the present invention is susceptible to various modifications and variations. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An intelligent inspection and maintenance system for power line emergency faults, characterized in that: It includes a command and dispatch platform, which communicates with the laser scanning camera, visible light camera and infrared camera of the inspection end UAV and the de-icing robot installed on the power line at the maintenance end through a satellite communication network. The intelligent inspection and maintenance system includes a multimodal fusion module, a task adjustment module and a task execution judgment module installed on the command and dispatch platform, wherein: The multimodal fusion module is configured to generate an ice coverage value for the power line based on the collected data from the laser scanning camera, the visible light camera, and the infrared camera using an improved DS evidence theory model, wherein the improved DS evidence theory model reflects the influence of real-time weather data on each collected value and the correlation between each collected data; The task adjustment module is used to adjust the deicing power and movement speed of the deicing robot when performing the deicing task according to the ice coverage value and the predicted temperature data; The task execution judgment module is used to judge whether to control the deicing robot to start the deicing task based on the comparison result of the ice coverage degree value and the ice coverage safety value, and to judge whether to control the deicing robot to end the deicing task based on the change trend of multiple ice coverage degree values; The multimodal fusion module includes a first evidence set generation unit, and the data collected by the laser scanning camera is point cloud image data; The first evidence set generating unit extracts point cloud features of the power line area and the ice-covered area from the point cloud image data through a convolutional neural network model, and generates elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the point cloud ice coverage gradient interval; Among them, the convolutional neural network model includes a point cloud multi-head attention pooling mechanism to process the spatial relationship of three-dimensional point cloud image data.

2. The intelligent inspection and maintenance system for power line emergency faults according to claim 1 is characterized in that: The multimodal fusion module includes a coefficient generation unit, a second evidence set generation unit, a third evidence set generation unit and a calculation unit; The coefficient generating unit is used to generate a coefficient matrix according to the real-time weather data; The second evidence set generating unit is configured to calculate a second evidence set based on the first evidence set and the coefficient matrix; The third evidence set generating unit is configured to generate a third evidence set reflecting the degree of association between each collected data based on the second evidence set; The calculation unit is used to calculate the ice coverage value by using the improved DS evidence theory model to apply the third evidence set.

3. The intelligent inspection and maintenance system for power line emergency faults according to claim 2 is characterized in that: The real-time weather data includes real-time illumination data and real-time temperature data, and the coefficient generation unit includes a laser scanning reliability generation subunit, a visible light image reliability generation subunit, an infrared data reliability generation subunit, and a coefficient calculation subunit; The laser scanning reliability generating subunit is used to generate laser scanning reliability by corresponding the real-time illumination data to the illumination-darkness gradient; The visible light image reliability generating subunit is configured to generate visible light image reliability by corresponding the real-time illumination data to the illumination brightness gradient; The infrared data reliability is used to generate the infrared data reliability by corresponding the real-time temperature data to the extreme cold temperature gradient; The coefficient calculation subunit is used to generate the coefficient matrix by normalizing the laser scanning reliability, the visible light image reliability and the infrared data reliability.

4. The intelligent inspection and maintenance system for power line emergency faults according to claim 3 is characterized in that: The third evidence set generation unit includes a weight calculation subunit and a comprehensive calculation subunit; The weight calculation subunit is used to calculate the weight corresponding to each collected data through the correlation coefficient formula; The comprehensive calculation subunit performs normalization operation on the multiple weights and then performs weighted calculation on the second evidence set to generate the third evidence set.

5. The intelligent inspection and maintenance system for power line emergency faults according to claim 2, characterized in that: The multi-head attention pooling mechanism adopts the PTv2 network framework, whose weight encoding function is generated by grouping linear layers, normalization layers, activation layers and attention fully connected layers to adjust the spatial capture efficiency of three-dimensional point cloud image data.

6. The intelligent inspection and maintenance system for power line emergency faults according to claim 2, characterized in that: The data collected by the visible light camera is visual image data; The first evidence set generation unit is used to identify the power line area and the ice-covered area in the visual image data through a YOLO-based image recognition model, and generate elements of the first evidence set by corresponding the ratio of the ice-covered area to the power line area to the visible light ice coverage gradient interval.

7. The intelligent inspection and maintenance system for power line emergency faults according to claim 2, characterized in that: The data collected by the infrared camera is thermal image data; The first evidence set generating unit is used to generate elements of the first evidence set according to the temperatures of multiple acquisition points on the surface of the power line in the thermal image data corresponding to the infrared icing degree gradient interval.

8. The intelligent inspection and maintenance system for power line emergency faults according to any one of claims 1 to 7, characterized in that: The task adjustment module includes a deicing power calculation unit and a moving speed calculation unit; The deicing power calculation unit is used to calculate the deicing power according to the difference between the ice coverage value and the ice coverage standard value; The moving speed calculation unit is connected to the deicing power calculation unit, and is used to generate a predicted ice formation rate according to the plurality of predicted air temperature data, and calculate the moving speed according to the predicted ice formation rate and the ice degree value.

9. The intelligent inspection and maintenance system for power line emergency fault according to any one of claims 1 to 7, characterized in that: The task execution judgment module is used to calculate an ice coverage prediction value through a regression model using multiple ice coverage degree values, and to determine whether to control the deicing robot to end the deicing task based on a comparison result between the ice coverage prediction value and the ice coverage safety value.

Images