Power transmission channel floating foreign matter high-precision real-time identification and early warning system and method

Abstract

The invention discloses a power transmission channel floating foreign matter high-precision real-time identification and early warning system and method, and relates to the technical field of computer vision and power system safety monitoring, and the system comprises a data collection processing unit, mosaic data enhancement, mixed data enhancement, image frame interpolation mixing, and generation of an enhanced virtual sample; the foreign matter feature extraction and recognition unit is used for extracting image features through a deformable convolution mechanism and a mixed domain attention mechanism; the tracking trend prediction unit is used for predicting the motion trail, the velocity vector and the direction of the lead by using a TransT tracker and Kalman filtering to judge the foreign matter; the multi-source information self-updating unit is used for carrying out dynamic risk assessment and alarm decision making and driving the unmanned aerial vehicle to carry out linkage disposal and incremental learning; and the rolling updating unit is used for realizing smooth upgrading of the deep learning model by an algorithm through a rolling updating mechanism, and replacing the old instance with the new version instance. The method has quick response capability and provides dynamic early warning time guarantee.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and power system safety monitoring technology, specifically to a high-precision real-time identification and early warning system and method for floating foreign objects in power transmission channels. Background Technology

[0002] With the continuous expansion of the power grid and the increasing complexity of the external operating environment, the safety of power transmission channels is facing unprecedented and severe challenges. Among these challenges, line discharges, short circuits, and tripping faults caused by various lightweight floating foreign objects occur frequently and have gradually become one of the main hidden dangers threatening the safe and stable operation of the power grid. These foreign objects often intrude into transmission corridors due to wind or human factors. They are diverse in material and shape, and possess complex visual characteristics such as low contrast, semi-transparency, and non-rigidity. Under complex weather conditions, they are extremely prone to contact with conductors or causing air gap breakdown. This risk is particularly prominent in areas with frequent monsoons, urban-rural fringe areas, and around construction sites. Traditional monitoring methods mainly rely on manual inspections or video alarm systems based on fixed thresholds, which have the following problems:

[0003] (1) The perspective is singular, there are blind spots in monitoring, and it is impossible to achieve full coverage of three-dimensional space.

[0004] (2) Insufficient ability to identify small targets, semi-transparent targets, and dynamic targets.

[0005] (3) Lack of dynamic analysis and graded early warning of target behavior states (such as floating, stationary, and moving).

[0006] (4) The system is isolated and lacks intelligent linkage with disposal methods (such as drones and work order systems).

[0007] (5) Existing deep learning-based recognition methods are still insufficient in complex backgrounds, small target detection, real-time performance and multi-source information fusion, making it difficult to meet the intelligent operation and maintenance needs of power transmission lines for pre-event prevention, in-event control and post-event closed-loop.

[0008] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0009] To address the problems in related technologies, this invention proposes a high-precision real-time identification and early warning system and method for floating foreign objects in power transmission channels, in order to overcome the aforementioned technical problems existing in the existing related technologies.

[0010] Therefore, the specific technical solution adopted by the present invention is as follows:

[0011] According to a first aspect of the present invention, a high-precision real-time identification and early warning system for floating foreign objects in power transmission channels is provided, the system comprising:

[0012] The data acquisition and processing unit is used to acquire and process panoramic image samples in real time, and to perform linear interpolation and mixing on image frames with adjacent timestamps using mosaic data augmentation and hybrid data augmentation methods to generate enhanced virtual samples for training a deep learning model for foreign object recognition and segmentation.

[0013] The foreign object feature extraction and recognition unit is used to adaptively extract image features based on strong virtual samples, using deformable convolution mechanism and hybrid domain attention mechanism, so as to achieve accurate segmentation and pixel-level recognition of multi-source foreign object contours.

[0014] The tracking trend prediction unit is used to predict the motion trajectory of the multi-source foreign object based on the identified multi-source foreign object contour, and to use the visual features of the TransT tracker and the dynamic state estimation of the Kalman filter. It also judges the foreign object based on the relationship between the velocity vector and the direction of the wire, so as to realize the trajectory prediction of the multi-source foreign object and the approach trend judgment of the velocity direction angle.

[0015] The multi-source information self-updating unit is used to perform dynamic risk assessment and alarm decision-making based on the identified foreign object type, distance, wind speed and trend information, and to drive the drone linkage disposal and achieve self-updating of the deep learning model for foreign object identification and segmentation through incremental learning.

[0016] The rolling update unit is used to continuously and smoothly upgrade the deep learning model for foreign object identification and segmentation based on the rolling update mechanism, and gradually replace the old instance with the new version instance to ensure zero service interruption.

[0017] Furthermore, panoramic image samples are acquired and processed in real time. Mosaic data augmentation and hybrid data augmentation methods are used to linearly interpolate and blend image frames with adjacent timestamps to generate enhanced virtual samples for training the deep learning model for foreign object recognition and segmentation. This includes: acquiring and stitching panoramic images using a full-color lens to obtain a large-scene stitched image; enhancing the acquired continuous panoramic image samples using mosaic data augmentation, hybrid data augmentation, and virtual sample generation; selecting two samples with adjacent timestamps in the training set, denoted as the first sample and the second sample, each sample including a panoramic image and its corresponding label; sampling based on beta distribution and obtaining random interpolation coefficients within the target range, where the beta distribution parameter is a hyperparameter controlling the shape of the interpolation coefficient distribution; linearly interpolating the first and second samples according to the interpolation coefficients to generate a new image; taking the union of the label coordinates of the first and second samples to obtain a new label; and using the new image and the new label to form a complete training sample for training the deep learning model for foreign object recognition and segmentation.

[0018] Furthermore, the foreign object feature extraction and recognition unit includes:

[0019] The foreign object feature extraction module is used to adaptively capture the irregular contours of non-rigid foreign objects and achieve feature recalibration by combining deformable convolution mechanism and hybrid domain attention mechanism.

[0020] The foreign object feature recognition module is used to accurately identify the contours of foreign objects and wires through a semantic segmentation network, calculate the minimum pixel distance between the contours of foreign objects and wires and convert it into actual physical distance, and use camera calibration parameters to correct system errors, so as to achieve accurate quantitative evaluation of the distance between foreign objects and wires.

[0021] Furthermore, by combining deformable convolution mechanisms with hybrid domain attention mechanisms, the irregular contours of non-rigid foreign objects are adaptively captured and feature recalibration is achieved, including:

[0022] Image data is processed using deformable convolution and hybrid domain attention mechanisms; the learnable offset field of the deformable convolution mechanism enables the sampling points of the convolution kernel to be adaptively adjusted according to the target shape.

[0023] The standard convolution rules are used to obtain grid coordinates and network learning, and the offset field is added to obtain the deformed sampling position. Based on the deformed sampling position, the irregular contour of non-rigid foreign objects in multi-source plastic film is captured.

[0024] Feature recalibration is performed using the cascaded channel attention and spatial attention terminals in the hybrid domain attention mechanism.

[0025] Furthermore, the semantic segmentation network is used to accurately identify the contours of foreign objects and wires, calculate the minimum pixel distance between the contours and convert it into actual physical distance, and use camera calibration parameters to correct system errors, so as to achieve accurate quantitative evaluation of the distance between foreign objects and wires, including:

[0026] A semantic segmentation network is used to identify and segment the labeled foreign objects to obtain pixel-level segmentation maps, and pixel regions of foreign objects and wires are extracted based on the pixel-level segmentation maps.

[0027] The minimum Euclidean pixel distance algorithm is used to calculate the pixel region of the foreign object and the wire to obtain the minimum pixel distance; the ratio between the pixel scale and the actual scale is established by using the actual diameter of the wire and its pixel width in the image.

[0028] Establish a proportional relationship between pixel scale and actual scale by measuring the pixel value of the wire diameter, and convert the minimum pixel distance into an estimated actual distance value according to the target scale;

[0029] Camera calibration parameters are introduced to perform nonlinear correction on the actual distance estimate in order to compensate for perspective distortion and installation angle errors.

[0030] Furthermore, based on the identified multi-source foreign object contours, and utilizing the visual features of the TransT tracker correlated with the dynamic state estimation of the Kalman filter, the motion trajectory of the multi-source foreign objects is predicted. The foreign object is then identified based on the relationship between the velocity vector and the direction of the conductor, thus achieving trajectory prediction and proximity trend judgment of the velocity direction angle of the multi-source foreign objects.

[0031] The TransT tracker and Kalman filter algorithm are used in combination to predict motion trajectory and determine trend.

[0032] Based on the TransT tracker and combined with the cross-attention mechanism, the association between the template features of the tracked target and the features of the video search region is established;

[0033] The Kalman filter recursively estimates the state vector through a state-space model; the state vector includes the target's position and velocity.

[0034] Kalman filtering consists of a prediction step and an update step. Based on the prediction step, the target state and uncertainty at the current time are predicted according to the state estimate and state transition matrix of the previous time step.

[0035] Based on the update step, the Kalman gain is calculated by combining the actual observation and the predicted value at the current time, and the predicted state is corrected to obtain the optimal state estimate at the current time.

[0036] Based on the target velocity vector estimated by the Kalman filter and the direction vector of the target pointing towards the guide wire, the angle between the target velocity vector and the direction vector is calculated; if the angle is less than a set threshold, it is determined that the target has a tendency to move closer to the guide wire.

[0037] Furthermore, Kalman filtering is divided into a prediction step and an update step, including:

[0038] The calculation formula for the prediction step is:

[0039]

[0040] P(k|k-1)=F(k)P(k-1|k-1)F(k)^T+Q(k);

[0041] In the formula, Let F(k) represent the predicted state at the current moment, and let F(k) represent the state transition matrix. Let P(k|k-1) represent the optimal estimated state at the previous time step, P(k-1|k-1) represent the uncertainty covariance matrix of the predicted state, P(k-1|k-1) represent the uncertainty covariance matrix of the estimate at the previous time step, Q(k) represent the process noise covariance, and T represent the permutation operation.

[0042] The formula for calculating the update step is:

[0043] K(k)=P(k|k-1)H(k)^T(H(k)P(k|k-1)H(k)^T+R(k))^(-1);

[0044]

[0045] P(k|k)=(IK(k)H(k))P(k|k-1);

[0046] In the formula, K(k) represents the Kalman gain, P(k|k-1) is the prior estimation error covariance matrix, H(k) represents the observation matrix, and R(k) represents the observation noise covariance. This represents the optimal state estimate at the current moment. Let P(k|k) represent the prior state estimate at time k, P(k|k) represent the predicted state, and z(k) represent the actual observed value. Let I represent the observation residuals, and let I represent the identity matrix.

[0047] Furthermore, based on the identified foreign object type, distance, wind speed, and trend information, dynamic risk assessment and alarm decisions are made, driving drone-based coordinated response and enabling self-updating of the deep learning model for foreign object identification and segmentation through incremental learning. This includes:

[0048] The classification branch of the semantic segmentation network is calculated using the Softmax function to obtain the type information of foreign objects;

[0049] Based on the type of foreign object, estimated distance, wind speed and wind direction information, and based on the preset mapping relationship, dynamic risk assessment and alarm decision are made.

[0050] Based on the alarm level, nearby airport drones are activated for joint review and risk elimination; based on the construction of a data closed loop and the use of incremental learning mechanisms of new sample cross-entropy loss and knowledge distillation loss, the deep learning model for foreign object identification and segmentation is continuously optimized and self-updated.

[0051] Furthermore, based on the rolling update mechanism, the algorithm continuously and smoothly upgrades the deep learning model for foreign object identification and segmentation, and gradually replaces the old instances with new version instances to ensure zero business interruption, including: building the updated deep learning model for foreign object identification and segmentation into an application image and pushing it to the image repository;

[0052] The rolling update mechanism based on the container orchestration terminal, combined with the new version image, creates and starts new service instances;

[0053] The system verifies that the new instance is fully ready and capable of handling requests using the built-in health check interface, switches user traffic to the ready new instance, and stops the old version service instances one by one according to the preset strategy until all old instances are replaced.

[0054] According to a second aspect of the present invention, a method for high-precision real-time identification and early warning of floating foreign objects in power transmission channels is also provided, the method comprising:

[0055] Panoramic image samples are acquired and processed in real time. Mosaic data augmentation and hybrid data augmentation are used to perform linear interpolation on image frames with adjacent timestamps to generate augmented virtual samples for training a deep learning model for foreign object recognition and segmentation.

[0056] Based on strong virtual samples, deformable convolution mechanism and hybrid domain attention mechanism are used to adaptively extract image features to achieve accurate segmentation and pixel-level recognition of multi-source foreign object contours.

[0057] Based on the identified multi-source foreign object contours, and using the visual features of the TransT tracker and the dynamic state estimation of the Kalman filter, the motion trajectory of the multi-source foreign objects is predicted. The foreign objects are judged based on the relationship between the velocity vector and the direction of the wire, so as to realize the trajectory prediction of multi-source foreign objects and the approach trend judgment of the velocity direction angle.

[0058] Dynamic risk assessment and alarm decisions are made based on the identified foreign object type, distance, wind speed and trend information, and the drone linkage is driven to handle the situation. Incremental learning is used to achieve self-updating of the deep learning model for foreign object identification and segmentation.

[0059] The algorithm is based on a rolling update mechanism to continuously and smoothly upgrade the deep learning model for foreign object identification and segmentation, and gradually replace the old instance with the new version instance to ensure zero service interruption.

[0060] The beneficial effects of this invention are as follows:

[0061] 1. This invention significantly improves the system's accuracy in capturing and segmenting the contours of foreign objects by using multi-stage data augmentation, deformable convolution mechanism and attention mechanism for feature extraction. This results in a foreign object detection rate of more than 85% in complex backgrounds, while reducing the false alarm rate by more than 30%.

[0062] 2. This invention combines an optimized semantic segmentation network, a lightweight tracking algorithm, and efficient engineering deployment to ensure that the system has a rapid response capability and meets the real-time requirement of processing a single frame panoramic image for less than 800ms, thus providing time assurance for dynamic early warning.

[0063] 3. This invention, through data augmentation strategies and the adaptive feature extraction capabilities of the model, enables the system to effectively cope with different lighting, weather, and occlusion conditions. Simultaneously, the multi-source information fusion mechanism takes into account meteorological influences, thus stably supporting the detection of various complex environments (such as strong winds, rain, and fog) and foreign object types (such as plastic film, ribbons, balloons, etc.).

[0064] 4. This invention achieves advanced insight and proactive intervention in risks through full-process automation of real-time perception, accurate identification, trajectory prediction, trend judgment, risk warning and coordinated response. Based on the transformation of transmission line operation and maintenance from traditional post-event emergency response to pre-event intelligent prevention, it fundamentally improves the level of line safety protection. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is a schematic diagram of a high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to an embodiment of the present invention.

[0067] Figure 2 This is a flowchart of a high-precision real-time identification and early warning method for floating foreign objects in power transmission channels according to an embodiment of the present invention;

[0068] Figure 3 This is an overall architecture diagram of a high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to an embodiment of the present invention;

[0069] Figure 4 This is a schematic diagram of deformable convolution in a high-precision real-time identification and early warning system for floating foreign objects in a power transmission channel according to an embodiment of the present invention.

[0070] Figure 5 This is a structural diagram of the attention mechanism in a high-precision real-time identification and early warning system for floating foreign objects in a power transmission channel according to an embodiment of the present invention;

[0071] Figure 6 This is a structural diagram of a tracking algorithm in a high-precision real-time identification and early warning system for floating foreign objects in a power transmission channel according to an embodiment of the present invention;

[0072] Figure 7 This is a flowchart of a multi-source information fusion alarm system for a high-precision real-time identification and early warning system for floating foreign objects in a power transmission channel, according to an embodiment of the present invention. Detailed Implementation

[0073] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0074] According to an embodiment of the present invention, a high-precision real-time identification and early warning system and method for floating foreign objects in power transmission channels are provided.

[0075] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 and Figure 3 As shown, a high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to an embodiment of the present invention includes:

[0076] The data acquisition and processing unit 1 is used to acquire and process panoramic image samples in real time, and to perform linear interpolation and mixing on image frames with adjacent timestamps using mosaic data augmentation and hybrid data augmentation methods to generate enhanced virtual samples for training a deep learning model for foreign object recognition and segmentation.

[0077] In this optional embodiment, panoramic image samples are acquired and processed in real time, and linear interpolation is performed on image frames with adjacent timestamps using mosaic data augmentation and hybrid data augmentation methods to generate enhanced virtual samples for training a deep learning model for foreign object identification and segmentation, including:

[0078] Full-color lenses are used to capture and stitch panoramic images to obtain large-scene stitched images; mosaic data enhancement, hybrid data enhancement, and virtual sample generation are used to enhance the captured continuous panoramic image samples;

[0079] Two samples with adjacent timestamps are selected from the training set and denoted as the first sample and the second sample. Each sample includes a panoramic image and its corresponding label.

[0080] Random interpolation coefficients within the target range are obtained by sampling based on the beta distribution, where the parameters of the beta distribution are hyperparameters that control the shape of the interpolation coefficient distribution.

[0081] A new image is generated by linearly interpolating the first and second samples based on the interpolation coefficients.

[0082] The new label is obtained by taking the union of the label coordinates of the first and second samples.

[0083] A complete training sample consisting of new images and new labels is used to train a deep learning model for foreign object recognition and segmentation.

[0084] Specifically, image acquisition utilizes a 64MP 360° AR spherical eagle-eye camera with no blind spots. The panoramic view is stitched together from eight F1.0 large-aperture full-color lenses, capable of outputting 2x180° large-scene stitched images, supporting AR 3D security based on industry platforms. Dataset preprocessing and enhancement address the issues of uneven sample distribution and small targets. This paper employs Mosaic, Mixup, and virtual sample generation to enhance the samples. Virtual sample generation involves linearly interpolating two training samples and their labels during training with continuously acquired 360° panoramic image samples to generate a new virtual sample. Specifically, two samples with adjacent timestamps are selected from the training set, denoted as sample A (first sample) and sample B (second sample). Each sample includes an image (360° panoramic image) and its corresponding label. The interpolation coefficient λ is a random value between 0 and 1, typically sampled from a Beta distribution. The parameter α of the Beta distribution is a hyperparameter that controls the shape of the λ distribution. Mathematical form: λ ~ Beta(α, α). The value of λ determines the weight of sample A and sample B in the mixture. When λ = 0.5, they are mixed equally; when λ is close to 1, the new sample is closer to sample A; when λ is close to 0, it is closer to sample B. The new image is calculated by a pixel-level weighted sum: I_new = λ * I_A + (1-λ) * I_B. Where I_A and I_B are the image tensors of samples A and B (e.g., arrays of shape [H, W, C]). The operation is performed pixel-wise to ensure consistent image size. The label L_new is obtained by taking the union of the coordinates of L_A and L_B. The new image I_new and the new label L_new constitute a complete training sample.

[0085] Foreign object feature extraction and recognition unit 2 is used to adaptively extract image features based on strong virtual samples, using deformable convolution mechanism and hybrid domain attention mechanism, so as to achieve accurate segmentation and pixel-level recognition of multi-source foreign object contours;

[0086] In this optional embodiment, the foreign object feature extraction and identification unit includes:

[0087] The foreign object feature extraction module is used to adaptively capture the irregular contours of non-rigid foreign objects and achieve feature recalibration by combining deformable convolution mechanism and hybrid domain attention mechanism.

[0088] The foreign object feature recognition module is used to accurately identify the contours of foreign objects and wires through a semantic segmentation network, calculate the minimum pixel distance between the contours of foreign objects and wires and convert it into actual physical distance, and use camera calibration parameters to correct system errors, so as to achieve accurate quantitative evaluation of the distance between foreign objects and wires.

[0089] In this optional embodiment, adaptively capturing the irregular contours of non-rigid foreign objects and achieving feature recalibration by combining deformable convolution mechanisms with hybrid domain attention mechanisms includes:

[0090] Image data is processed using deformable convolution and hybrid domain attention mechanisms; the learnable offset field of the deformable convolution mechanism enables the sampling points of the convolution kernel to be adaptively adjusted according to the target shape.

[0091] The standard convolution rules are used to obtain grid coordinates and network learning, and the offset field is added to obtain the deformed sampling position. Based on the deformed sampling position, the irregular contour of non-rigid foreign objects in multi-source plastic film is captured.

[0092] Feature recalibration is performed using the cascaded channel attention and spatial attention terminals in the hybrid domain attention mechanism.

[0093] In this optional embodiment, the precise quantitative assessment of the distance between the foreign object and the wire is achieved by accurately identifying the contours of the foreign object and the wire using a semantic segmentation network, calculating the minimum pixel distance between the contours of the foreign object and the wire and converting it into the actual physical distance, and using camera calibration parameters to correct system errors.

[0094] A semantic segmentation network is used to identify and segment the labeled foreign objects to obtain pixel-level segmentation maps, and pixel regions of foreign objects and wires are extracted based on the pixel-level segmentation maps.

[0095] The minimum Euclidean pixel distance algorithm is used to calculate the pixel region of the foreign object and the wire to obtain the minimum pixel distance;

[0096] By using the actual diameter of the conductor and its pixel width in the image, the proportional relationship between the pixel scale and the actual scale is established.

[0097] Establish a proportional relationship between pixel scale and actual scale by measuring the pixel value of the wire diameter, and convert the minimum pixel distance into an estimated actual distance value according to the target scale;

[0098] Camera calibration parameters are introduced to perform nonlinear correction on the actual distance estimate in order to compensate for perspective distortion and installation angle errors.

[0099] Specifically, feature extraction and recognition; in the feature extraction stage, a combination of deformable convolutional network (DCN) and hybrid domain attention mechanism (CBAM) is used to process image data, such as... Figure 4 and Figure 5 As shown. The deformable convolution mechanism, through its learnable offset field, enables the sampling points of the convolution kernel to adaptively adjust according to the actual shape of the target. Its core operation can be expressed as:

[0100] P = P0 + ΔP;

[0101] Where P0 represents the coordinates of the regular sampling grid of the standard convolution, ΔP represents the offset field learned by the network, and P represents the sampling position after deformation, effectively capturing the irregular contours of non-rigid foreign objects such as plastic films. The hybrid domain attention mechanism achieves feature recalibration through cascaded channel attention and spatial attention modules. The channel attention weights are calculated as follows:

[0102] M_c=σ(MLP(AvgPool(FM))+MLP(MaxPool(FM)));

[0103] Spatial attention weights are calculated as follows:

[0104] M_s=σ(Conv([AvgPool(FM);MaxPool(FM)]));

[0105] Where FM represents the input feature map, AvgPool represents average pooling, MaxPool represents max pooling, σ represents the sigmoid function, MLP represents a multilayer perceptron, Conv represents a convolution operation, and M_c and M_s represent the attention weights for the channel and spatial dimensions, respectively. Based on this, the semantic segmentation network achieves pixel-level recognition through an encoder-decoder architecture, and its loss function uses cross-entropy loss.

[0106]

[0107] Where y_ij represents the true label of pixel (i,j). This represents the predicted probability, thereby enabling accurate segmentation of the foreign object's outline.

[0108] like Figure 6 As shown, the pixel regions of the foreign object and the wire are extracted based on the semantic segmentation results; the pixel distance between them is calculated and normalized to an estimated actual distance; and distance correction is performed in conjunction with camera calibration parameters.

[0109] Specific method: Extract pixel regions for foreign objects and wires respectively from the pixel-level segmentation map output by the semantic segmentation network. Calculate the minimum Euclidean pixel distance between the two regions:

[0110]

[0111] Where d_pixel represents the minimum pixel distance, (x_o, y_o) represents the pixel coordinates within the foreign object region, and (x_w, y_w) represents the pixel coordinates within the wire region. The algorithm iterates through all pixels in both regions, finding the pair of pixels that minimizes the distance formula; this pair represents the minimum pixel distance d_pixel.

[0112] The pixel distance is converted into an actual distance estimate. This method uses the wire diameter as a known reference scale for normalization calculation.

[0113] d_actual=(D_wire×f) / d_wire_pixel×(d_pixel / f);

[0114] Where D_wire represents the actual physical diameter of the wire (a known parameter), d_wire_pixel represents the average pixel width of the wire region in the segmentation image along the vertical projection direction, and f is the camera focal length (obtained through calibration). The pixel measurement value of the wire diameter is used to establish the ratio between the pixel scale and the actual scale, and then the pixel distance between the foreign object and the wire is converted into the actual distance estimate d_actual according to this ratio.

[0115] To compensate for systematic errors caused by camera perspective distortion and mounting angle, camera calibration parameters are introduced for nonlinear correction:

[0116] d_corrected=d_actual×[1+k1×(h / H)+k2×(θ-θ0)2];

[0117] Where h represents the vertical position coordinates of the target in the image, H represents the total height of the image, θ represents the actual installation pitch angle between the camera optical axis and the horizontal plane, θ0 represents the calibration reference angle, and k1 and k2 both represent distortion compensation coefficients determined through camera calibration experiments. This correction term mainly compensates for two types of errors: one is the difference in perspective scale at different heights in the image (k1 term), and the other is the distance calculation error caused by the deviation in camera installation angle (k2 term).

[0118] The tracking trend prediction unit 3 is used to predict the motion trajectory of the multi-source foreign object based on the identified multi-source foreign object contour and by using the visual features of the TransT tracker and the dynamic state estimation of the Kalman filter. It also judges the foreign object based on the relationship between the velocity vector and the direction of the wire, so as to realize the trajectory prediction of the multi-source foreign object and the approach trend judgment of the velocity direction angle.

[0119] In this optional embodiment, based on the identified multi-source foreign object contours, and utilizing the visual features of the TransT tracker correlated with the dynamic state estimation of the Kalman filter, the trajectory of the multi-source foreign object is predicted. Furthermore, the foreign object is identified based on the relationship between the velocity vector and the direction of the conductor. This achieves trajectory prediction of the multi-source foreign object and determination of its approach trend based on the velocity direction angle.

[0120] The TransT tracker and Kalman filter algorithm are used in combination to predict motion trajectory and determine trend.

[0121] Based on the TransT tracker and combined with the cross-attention mechanism, the association between the template features of the tracked target and the features of the video search region is established;

[0122] The Kalman filter recursively estimates the state vector through a state-space model; the state vector includes the target's position and velocity.

[0123] Kalman filtering consists of a prediction step and an update step;

[0124] Based on the prediction steps, the target state and uncertainties at the current moment are predicted according to the state estimate and state transition matrix of the previous moment.

[0125] Based on the update step, the Kalman gain is calculated by combining the actual observation and the predicted value at the current time, and the predicted state is corrected to obtain the optimal state estimate at the current time.

[0126] Based on the target velocity vector estimated by the Kalman filter and the direction vector of the target pointing towards the guide wire, the angle between the target velocity vector and the direction vector is calculated; if the angle is less than a set threshold, it is determined that the target has a tendency to move closer to the guide wire.

[0127] In this optional embodiment, the Kalman filter is divided into a prediction step and an update step, including:

[0128] The calculation formula for the prediction step is:

[0129]

[0130] P(k|k-1)=F(k)P(k-1|k-1)F(k)^T+Q(k);

[0131] In the formula, Let F(k) represent the predicted state at the current moment, and let F(k) represent the state transition matrix. Let P(k|k-1) represent the optimal estimated state at the previous time step, P(k-1|k-1) represent the uncertainty covariance matrix of the predicted state, P(k-1|k-1) represent the uncertainty covariance matrix of the estimate at the previous time step, Q(k) represent the process noise covariance, and T represent the permutation operation.

[0132] Specifically, F(k) represents the predicted state (position, velocity, etc.) at the current moment, and F(k) represents the state transition matrix, describing how the target moves from the previous moment to the current moment. P(k|k-1) represents the optimal estimated state at the previous time step, P(k-1|k-1) represents the uncertainty covariance matrix of the predicted state, P(k-1|k-1) represents the uncertainty covariance matrix of the previous time step, Q(k) represents the process noise covariance, and Q represents the degree of imperfection of the motion model.

[0133] The formula for calculating the update step is:

[0134] K(k)=P(k|k-1)H(k)^T(H(k)P(k|k-1)H(k)^T+R(k))^(-1);

[0135]

[0136] P(k|k)=(IK(k)H(k))P(k|k-1);

[0137] In the formula, K(k) represents the Kalman gain, P(k|k-1) represents the prior estimation error covariance matrix, H(k) represents the observation matrix, and R(k) represents the observation noise covariance. This represents the optimal state estimate at the current moment. Let P(k|k) represent the prior state estimate at time k, P(k|k) represent the predicted state, and z(k) represent the actual observed value. Let I represent the observation residuals, and let I represent the identity matrix.

[0138] Specifically, K(k) represents the Kalman gain, the weighting coefficients (between 0 and 1), H(k) represents the observation matrix, which maps the state space to the observation space, and R(k) represents the observation noise covariance, indicating the degree of inaccuracy in the sensor measurements. Let z(k) represent the optimal state estimate at the current moment, and z(k) represent the actual observed value (such as the position detected by TransT). This represents the observation residual, the difference between the prediction and the observation.

[0139] Tracking and trend prediction: The system employs a TransT tracker and a Kalman filter algorithm in synergy to predict motion trajectories and determine trends. The TransT tracker establishes a correlation between template features and search region features based on a cross-attention mechanism.

[0140]

[0141] Among them, Q r K represents the query vector of the search region. r and V r The key-value pairs represent the template, and d_k represents the dimension scaling factor. The Kalman filter performs recursive estimation through a state-space model, where the state vector is defined as x = [p_x, p_y, v_x, v_y]^T, where p_x and p_y are the target positions, and v_x and v_y are the motion velocities.

[0142] Based on the estimated velocity vector and the directional relationship between the foreign object and the conductor connection, the trend index is calculated:

[0143] θ=arccos((v·d) / (||v||·||d||));

[0144] Where v is the velocity vector, d is the direction vector of the foreign object pointing towards the wire, and when θ is less than the set threshold, it is determined to be a tendency to approach.

[0145] The multi-source information self-updating unit 4 is used to perform dynamic risk assessment and alarm decision-making based on the identified foreign object type, distance, wind speed and trend information, and drive the drone linkage disposal and achieve self-updating of the deep learning model for foreign object identification and segmentation through incremental learning.

[0146] In this optional embodiment, dynamic risk assessment and alarm decision-making are performed based on the identified foreign object type, distance, wind speed, and trend information, and the drone is driven to handle the situation in a coordinated manner. Incremental learning is also used to achieve self-updating of the deep learning model for foreign object identification and segmentation, including:

[0147] The classification branch of the semantic segmentation network is calculated using the Softmax function to obtain the type information of foreign objects;

[0148] Based on the type of foreign object, estimated distance, wind speed and wind direction information, and based on the preset mapping relationship, dynamic risk assessment and alarm decision are made.

[0149] Based on the alarm level, drones from nearby airports were activated to conduct joint inspections and risk elimination.

[0150] Based on the construction of a data closed loop and the use of incremental learning mechanisms such as new sample cross-entropy loss and knowledge distillation loss, the deep learning model for foreign object identification and segmentation is continuously optimized and self-updated.

[0151] Specifically, such as Figure 7 As shown, multi-source information and a self-updating mechanism are used; foreign object type information is output by the classification branch of the semantic segmentation network through the Softmax function:

[0152] P(c_m|x)=exp(z_m) / ∑exp(z_n);

[0153] Here, z_m represents the logits value of the m-th class, where logits represent the raw, unnormalized output value of the last fully connected layer (or output layer) of the deep learning model. P(c_m|x) represents a conditional probability, which is the probability that an object in an input image x belongs to class c_m. This is precisely the softmax-normalized probability value output by the classification branch of the semantic segmentation network for each pixel location, ranging from [0,1].

[0154] Meteorological data participates in early warning decisions through established mapping relationships:

[0155] Alert_Level=f(Y_C,D,V_w,V_d);

[0156] Where Y_C represents the type of foreign object, D represents the estimated distance, V_w represents the wind speed, and V_d represents the wind direction. The function f is a decision function encapsulating the alarm strategy. A critical alarm is triggered when the wind speed is greater than level 5 and the target distance is less than 10 meters; a severe alarm is triggered when the wind speed is greater than level 5 and the target distance is between 10 and 20 meters; and a general alarm is triggered when the wind speed is greater than level 5 and the target distance is greater than 20 meters. Under calm or light wind conditions, the alarm level is reduced accordingly based on the distance. If a target continuously approaches the conductor and its speed increases, the alarm level is increased by one level, regardless of the alarm level.

[0157] Based on the alarm, nearby airport drones are activated to conduct joint inspections, identify and eliminate risk points, such as fire or collisions, to prevent floating objects from intruding into power transmission lines.

[0158] The model self-update mechanism achieves continuous optimization by constructing a data closed loop, and its core incremental learning loss function is:

[0159] L_total=L_new+λL_distill;

[0160] Where L_new represents the cross-entropy loss of new samples, L_distill represents the knowledge distillation loss, and λ is the balancing hyperparameter.

[0161] Rolling update unit 5 is used to continuously and smoothly upgrade the deep learning model for foreign object identification and segmentation based on the rolling update mechanism, and gradually replace the old instance with the new version instance to ensure zero business interruption.

[0162] In this optional embodiment, the algorithm continuously and smoothly upgrades the deep learning model for foreign object identification and segmentation based on a rolling update mechanism, and gradually replaces old instances with new ones to ensure zero service interruption, including:

[0163] The updated deep learning model for foreign object identification and segmentation is built into an application image and pushed to the image repository;

[0164] The rolling update mechanism based on the container orchestration terminal, combined with the new version image, creates and starts new service instances;

[0165] The system verifies that the new instance is fully ready and capable of handling requests using the built-in health check interface, switches user traffic to the ready new instance, and stops the old version service instances one by one according to the preset strategy until all old instances are replaced.

[0166] Specifically, after the deep learning model for foreign object identification and segmentation is updated and verified, the system builds an application image containing the updated deep learning model for foreign object identification and segmentation (the model trained using the new data after the data update) and pushes it to the image repository. The algorithm service is continuously upgraded through the rolling update mechanism of the Kubernetes container orchestration system. One or more new service instances are created and started based on the new version image. Only after the built-in health check interface verifies that the new instance is fully ready and capable of handling requests will user traffic be switched to the new instance. The system stops the old version service instances one by one according to the preset strategy, and so on until all old instances are replaced. In this way, some service instances are kept running normally throughout the entire update process, achieving a smooth upgrade with zero business interruption.

[0167] Specifically, the data acquisition and augmentation process involves: using power transmission channel monitoring video data and augmenting it with Mosaic, Mixup, and virtual data; model training using a pre-trained semantic segmentation network yolov8-seg combined with CBAM and DCN modules for fine-tuning; real-time detection and tracking by deploying a lightweight model on embedded devices (such as Jetson Nano) to achieve real-time video stream processing; distance estimation and alarming by calculating the pixel distance between the object and the wire based on the semantic segmentation results and dynamically adjusting the alarm threshold based on meteorological data; and model updates by periodically optimizing model performance through online learning and incremental update mechanisms.

[0168] like Figure 2 As shown, according to another embodiment of the present invention, a high-precision real-time identification and early warning method for floating foreign objects in power transmission channels is also provided, the method comprising:

[0169] Step S1: Real-time acquisition and processing of panoramic image samples, and linear interpolation mixing of image frames with adjacent timestamps using mosaic data augmentation and hybrid data augmentation methods to generate enhanced virtual samples for training a deep learning model for foreign object recognition and segmentation.

[0170] Step S2: Based on strong virtual samples, adaptively extract image features using deformable convolution mechanism and hybrid domain attention mechanism to achieve accurate segmentation and pixel-level recognition of multi-source foreign object contours;

[0171] Step S3: Based on the identified multi-source foreign object contours, and using the visual features of the TransT tracker associated with the dynamic state estimation of the Kalman filter, predict the motion trajectory of the multi-source foreign object, and judge the foreign object based on the relationship between the velocity vector and the direction of the wire, so as to realize the trajectory prediction of the multi-source foreign object and the judgment of the approach trend of the velocity direction angle.

[0172] Step S4: Based on the identified foreign object type, distance, wind speed and trend information, perform dynamic risk assessment and alarm decision-making, drive the drone to handle the situation together, and achieve self-updating of the deep learning model for foreign object identification and segmentation through incremental learning.

[0173] Step S5: Based on the rolling update mechanism, the algorithm continuously and smoothly upgrades the deep learning model for foreign object identification and segmentation, and gradually replaces the old instance with the new version instance to ensure zero service interruption.

[0174] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-precision real-time identification and early warning system for floating foreign objects in power transmission channels, characterized in that, include: The data acquisition and processing unit is used to acquire and process panoramic image samples in real time, and to perform linear interpolation and mixing on image frames with adjacent timestamps using mosaic data augmentation and hybrid data augmentation methods to generate enhanced virtual samples for training a deep learning model for foreign object recognition and segmentation. The foreign object feature extraction and recognition unit is used to adaptively extract image features based on strong virtual samples, using deformable convolution mechanism and hybrid domain attention mechanism, so as to achieve accurate segmentation and pixel-level recognition of multi-source foreign object contours. The tracking trend prediction unit is used to predict the motion trajectory of the multi-source foreign object based on the identified multi-source foreign object contour, and to use the visual features of the TransT tracker and the dynamic state estimation of the Kalman filter. It also judges the foreign object based on the relationship between the velocity vector and the direction of the wire, so as to realize the trajectory prediction of the multi-source foreign object and the approach trend judgment of the velocity direction angle. The multi-source information self-updating unit is used to perform dynamic risk assessment and alarm decision-making based on the identified foreign object type, distance, wind speed and trend information, and to drive the drone linkage disposal and achieve self-updating of the deep learning model for foreign object identification and segmentation through incremental learning. The rolling update unit is used to continuously and smoothly upgrade the deep learning model for foreign object identification and segmentation based on the rolling update mechanism, and gradually replace the old instance with the new version instance to ensure zero service interruption.

2. The high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 1, characterized in that, The process of real-time acquisition and processing of panoramic image samples, and using mosaic data augmentation and hybrid data augmentation methods to perform linear interpolation and mixing on image frames with adjacent timestamps to generate enhanced virtual samples for training a deep learning model for foreign object identification and segmentation, includes: By using a full-color lens to capture and stitch panoramic images, a large-scene stitched image is obtained; The acquired continuous panoramic image samples are enhanced using mosaic data augmentation, hybrid data augmentation, and virtual sample generation methods. Two samples with adjacent timestamps are selected from the training set and denoted as the first sample and the second sample. Each sample includes a panoramic image and its corresponding label. Random interpolation coefficients within the target range are obtained by sampling based on the beta distribution, where the parameters of the beta distribution are hyperparameters that control the shape of the interpolation coefficient distribution. A new image is generated by linearly interpolating the first and second samples based on the interpolation coefficients. The new label is obtained by taking the union of the label coordinates of the first and second samples. A complete training sample consisting of new images and new labels is used to train a deep learning model for foreign object recognition and segmentation.

3. The high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 1, characterized in that, The foreign object feature extraction and identification unit includes: The foreign object feature extraction module is used to adaptively capture the irregular contours of non-rigid foreign objects and achieve feature recalibration by combining deformable convolution mechanism and hybrid domain attention mechanism. The foreign object feature recognition module is used to accurately identify the contours of foreign objects and wires through a semantic segmentation network, calculate the minimum pixel distance between the contours of foreign objects and wires and convert it into actual physical distance, and use camera calibration parameters to correct system errors, so as to achieve accurate quantitative evaluation of the distance between foreign objects and wires.

4. The high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 1, characterized in that, The method of adaptively capturing the irregular contours of non-rigid foreign objects and achieving feature recalibration by combining deformable convolution mechanisms with hybrid domain attention mechanisms includes: Image data is processed using deformable convolution and hybrid domain attention mechanisms; The learnable offset field using deformable convolution mechanism enables the convolution kernel sampling points to adaptively adjust according to the target shape; The standard convolution rules are used to obtain grid coordinates and network learning, and the offset field is added to obtain the deformed sampling position. Based on the deformed sampling position, the irregular contour of non-rigid foreign objects in multi-source plastic film is captured. Feature recalibration is performed using the cascaded channel attention and spatial attention terminals in the hybrid domain attention mechanism.

5. The high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 1, characterized in that, The process of accurately identifying the contours of foreign objects and wires using a semantic segmentation network, calculating the minimum pixel distance between the contours and converting it into actual physical distance, and using camera calibration parameters to correct system errors to achieve accurate quantitative evaluation of the distance between foreign objects and wires includes: A semantic segmentation network is used to identify and segment the labeled foreign objects to obtain pixel-level segmentation maps, and pixel regions of foreign objects and wires are extracted based on the pixel-level segmentation maps. The minimum Euclidean pixel distance algorithm is used to calculate the pixel region of the foreign object and the wire to obtain the minimum pixel distance; By using the actual diameter of the conductor and its pixel width in the image, the proportional relationship between the pixel scale and the actual scale is established. Establish a proportional relationship between pixel scale and actual scale by measuring the pixel value of the wire diameter, and convert the minimum pixel distance into an estimated actual distance value according to the target scale; Camera calibration parameters are introduced to perform nonlinear correction on the actual distance estimate in order to compensate for perspective distortion and installation angle errors.

6. A high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 1, characterized in that, The process of predicting the trajectory of multi-source foreign objects based on their identified contours, using the visual features of the TransT tracker and dynamic state estimation via Kalman filtering, and determining the foreign object based on the relationship between the velocity vector and the direction of the conductor, thereby achieving trajectory prediction and proximity trend judgment of multi-source foreign objects, includes: The TransT tracker and Kalman filter algorithm are used in combination to predict motion trajectory and determine trend. Based on the TransT tracker and combined with the cross-attention mechanism, the association between the template features of the tracked target and the features of the video search region is established; The Kalman filter recursively estimates the state vector through a state-space model; the state vector includes the target's position and velocity. Kalman filtering consists of a prediction step and an update step; Based on the prediction steps, the target state and uncertainties at the current moment are predicted according to the state estimate and state transition matrix of the previous moment. Based on the update step, the Kalman gain is calculated by combining the actual observation and the predicted value at the current time, and the predicted state is corrected to obtain the optimal state estimate at the current time. Based on the target velocity vector estimated by the Kalman filter and the direction vector of the target pointing towards the guide wire, the angle between the target velocity vector and the direction vector is calculated; if the angle is less than a set threshold, it is determined that the target has a tendency to move closer to the guide wire.

7. A high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 6, characterized in that, The Kalman filter consists of a prediction step and an update step, including: The calculation formula for the prediction step is: P(k|k-1)=F(k)P(k-1|k-1)F(k)^T+Q(k); In the formula, Let F(k) represent the predicted state at the current moment, and let F(k) represent the state transition matrix. Let P(k|k-1) represent the optimal estimated state at the previous time step, P(k-1|k-1) represent the uncertainty covariance matrix of the predicted state, P(k-1|k-1) represent the uncertainty covariance matrix of the estimate at the previous time step, Q(k) represent the process noise covariance, and T represent the permutation operation. The formula for calculating the update step is: K(k)=P(k|k-1)H(k)^T(H(k)P(k|k-1)H(k)^T+R(k))^(-1); P(k|k)=(IK(k)H(k))P(k|k-1); In the formula, K(k) represents the Kalman gain, P(k|k-1) represents the prior estimation error covariance matrix, H(k) represents the observation matrix, and R(k) represents the observation noise covariance. This represents the optimal state estimate at the current moment. Let P(k|k) represent the prior state estimate at time k, P(k|k) represent the predicted state, and z(k) represent the actual observed value. Let I represent the observation residuals, and let I represent the identity matrix.

8. A high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 1, characterized in that, The process of dynamically assessing risks and issuing alarms based on the identified foreign object type, distance, wind speed, and trend information, driving drone-based coordinated response, and achieving self-updating of the deep learning model for foreign object identification and segmentation through incremental learning includes: The classification branch of the semantic segmentation network is calculated using the Softmax function to obtain the type information of foreign objects; Based on the type of foreign object, estimated distance, wind speed and wind direction information, and based on the preset mapping relationship, dynamic risk assessment and alarm decision are made. Based on the alarm level, drones from nearby airports were activated to conduct joint inspections and risk elimination. Based on the construction of a data closed loop and the use of incremental learning mechanisms such as new sample cross-entropy loss and knowledge distillation loss, the deep learning model for foreign object identification and segmentation is continuously optimized and self-updated.

9. A high-precision real-time identification and early warning system for floating foreign objects in power transmission channels according to claim 1, characterized in that, The algorithm based on a rolling update mechanism continuously and smoothly upgrades the deep learning model for foreign object identification and segmentation, and gradually replaces old instances with new ones to ensure zero service interruption. This includes: The updated deep learning model for foreign object identification and segmentation is built into an application image and pushed to the image repository; The rolling update mechanism based on the container orchestration terminal, combined with the new version image, creates and starts new service instances; The system verifies that the new instance is fully ready and capable of handling requests using the built-in health check interface, switches user traffic to the ready new instance, and stops the old version service instances one by one according to the preset strategy until all old instances are replaced.

10. A method for high-precision real-time identification and early warning of floating foreign objects in power transmission channels, employing the high-precision real-time identification and early warning system for floating foreign objects in power transmission channels as described in any one of claims 1-9, characterized in that, The method includes the following steps: Panoramic image samples are acquired and processed in real time. Mosaic data augmentation and hybrid data augmentation are used to perform linear interpolation on image frames with adjacent timestamps to generate augmented virtual samples for training a deep learning model for foreign object recognition and segmentation. Based on strong virtual samples, deformable convolution mechanism and hybrid domain attention mechanism are used to adaptively extract image features to achieve accurate segmentation and pixel-level recognition of multi-source foreign object contours. Based on the identified multi-source foreign object contours, and using the visual features of the TransT tracker and the dynamic state estimation of the Kalman filter, the motion trajectory of the multi-source foreign objects is predicted. The foreign objects are judged based on the relationship between the velocity vector and the direction of the wire, so as to realize the trajectory prediction of multi-source foreign objects and the approach trend judgment of the velocity direction angle. Dynamic risk assessment and alarm decisions are made based on the identified foreign object type, distance, wind speed and trend information, and the drone linkage is driven to handle the situation. Incremental learning is used to achieve self-updating of the deep learning model for foreign object identification and segmentation. The algorithm is based on a rolling update mechanism to continuously and smoothly upgrade the deep learning model for foreign object identification and segmentation, and gradually replace the old instance with the new version instance to ensure zero service interruption.

Images