A machine vision-based transmission line external damage monitoring system

Abstract

This invention relates to the field of image detection technology, specifically a machine vision-based power transmission line external damage monitoring system. It includes a scene evaluation module: acquiring image data, infrared data, and point cloud data; inputting brightness and temperature parameters into an environmental analysis matrix to calculate scene evaluation values; a visual routing module: performing routing calculations based on threshold comparisons and outputting the visual features to be detected; outputting suspected bounding boxes from a visual detection model; performing spatial registration and background cropping based on these boxes to generate key image frames; and an incremental optimization module: generating confirmation results and alarm signals from a classification model, outputting difficult image samples through difference verification, aggregating incremental visual samples to update recognition parameters, and outputting an upgraded network. This invention constructs a closed loop of visual modality routing, image spatial denoising, and pattern recognition network updates, linking adaptive visual feature extraction, image semantic purification, and iteration of difficult visual samples to overcome image feature matching failures caused by representational bias.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, specifically to a machine vision-based power transmission line external damage monitoring system. Background Technology

[0002] For wide-area and weather-variable outdoor natural scenarios (such as power transmission line corridors and high-voltage power grid crossing areas), high-precision visual monitoring and image pattern recognition of large mobile mechanical targets are important technological applications for ensuring the safe operation of power systems. Existing technologies generally adopt image monitoring methods using single visible light or multimodal visual sensing. This involves continuously acquiring video streams and multimodal visual feature arrays through a visual acquisition terminal and directly inputting them into a pre-established visual target detection network. The target contour is then automatically extracted through image bounding box regression calculation or semantic mapping to confirm intrusion events. This is currently the mainstream video image analysis method.

[0003] However, due to the extreme complexity of light and color in outdoor scenes, as well as the asymmetric distribution of visual computing resources, existing image surveillance technologies face significant machine vision bottlenecks in practical applications. First, existing video analysis terminals experience drastic fluctuations in pixel-level quality of multimodal visual feature maps under extreme weather conditions or intense alternating lighting. Conventional image channel overlay processing cannot effectively suppress high-frequency visual noise, easily causing target edge feature annihilation and texture aliasing, leading to large-scale visual detection misses. Second, when high-frame-rate video streams and multi-dimensional image matrices converge concurrently, not only does it rapidly consume the limited computing power of the front-end image matrix, but the extensive stacking of massive heterogeneous visual features also results in severe frame-level delays and feature misalignments during cross-level visual feature registration and spatial alignment, hindering continuous visual tracking of dynamic targets. Third, fixed visual pattern recognition networks are highly susceptible to erosion from the distribution shift of visual representations in outdoor scenes, causing a continuous and precipitous decline in image recognition and pixel-level classification accuracy. This makes it difficult for machine vision monitoring systems to maintain good feature extraction and high-precision image pattern matching in wide-area corridor scenes, making it difficult to achieve long-term and reliable visual monitoring and analysis.

[0004] To address this, a machine vision-based system for monitoring external damage to power transmission lines is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based transmission line external damage monitoring system to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a machine vision-based transmission line external damage monitoring system, comprising: Scene assessment module: acquires image data, infrared data, and point cloud data of the transmission line corridor; extracts global brightness parameters from the image data and local temperature difference values ​​from the infrared data; inputs the global brightness parameters and local temperature difference values ​​into the environment analysis matrix for weighted calculation and output of scene assessment values; The visual routing module acquires a baseline threshold, compares it with the scene evaluation value, and outputs the comparison result. Based on the comparison result, it performs routing assembly calculations on image data, infrared data, and point cloud data to output the features to be detected. It inputs the features to be detected into the visual detection model to perform target detection and outputs suspected bounding boxes. Based on the suspected bounding boxes, it performs spatial registration on image data, infrared data, and point cloud data to output a visual alignment matrix. Based on the visual alignment matrix, it performs background cropping to output key image frames. Incremental optimization module: Input key image frames into the deep classification model to predict and calculate the output confirmation result, and determine the intrusion conditions to output early warning information; Based on the early warning information, perform difference verification on the confirmation result and key image frames to output a difficult sample set; Perform aggregation on the difficult sample set to output incremental samples, and output updated network parameters through parameter differentiation to update the visual detection model.

[0007] Preferably, the specific process for generating the scene evaluation value includes: extracting the global pixel brightness values ​​contained in the image data, performing an average value statistical operation to output a global brightness parameter; extracting the global temperature extreme values ​​contained in the infrared data, performing a subtraction operation to output a local temperature difference value; inputting the brightness parameter into a forward mapping function configured in the environment analysis matrix to perform a direct proportional calculation to output a local clarity coefficient; inputting the temperature parameter into a reverse mapping function configured in the environment analysis matrix to perform an inverse proportional calculation to output a local stability coefficient; obtaining preset brightness weight parameters and preset temperature weight parameters; performing a product correlation calculation on the local clarity coefficient and preset brightness weight parameters to output an environmental brightness evaluation value; performing a product correlation calculation on the local stability coefficient and preset temperature weight parameters to output an environmental temperature evaluation value; and performing an addition and merging calculation on the environmental brightness evaluation value and environmental temperature evaluation value to output a scene evaluation value.

[0008] Preferably, the specific generation process of the feature to be detected includes: introducing a benchmark threshold; performing a numerical comparison calculation between the scene evaluation value and the benchmark threshold to output a state comparison result; responding to the state comparison result that the scene evaluation value is lower than the benchmark threshold, inputting image data, infrared data, and point cloud data into a multimodal extractor to perform a stitching and merging operation to output a hybrid modality aggregation matrix, inputting the hybrid modality aggregation matrix into a channel attention module to perform a weighted summation calculation to output the feature to be detected; responding to the state comparison result that the scene evaluation value is not lower than the benchmark threshold, performing a feature channel mapping operation on the image data to output an independent image feature channel, inputting the independent image feature channel into a visual feature extractor to perform a convolution extraction calculation to output the feature to be detected.

[0009] Preferably, the specific construction of the visual detection model includes: a deformation space alignment layer: receiving the features to be detected, generating pixel geometric offset vectors based on the features through convolution mapping, and performing adaptive deformation convolution calculation on the features to be detected based on the pixel geometric offset vectors to output a geometric adaptive feature tensor; a cross-scale pyramid fusion layer: receiving the geometric adaptive feature tensor, performing scale downsampling extraction calculation on the geometric adaptive feature tensor to output a multi-scale semantic channel, and performing self-attention weighted association calculation on the multi-scale semantic channel to output a multi-scale feature matrix to be detected; a candidate box regression generation layer: receiving the multi-scale feature matrix to be detected, inputting the multi-scale feature matrix to be detected into a regression mapping network to perform coordinate offset decoding calculation to output a position coordinate matrix; a scale probability classification layer: receiving the multi-scale feature matrix to be detected, inputting the multi-scale feature matrix to be detected into a fully connected classification network to perform nonlinear activation mapping calculation to output a class probability matrix; and a boundary suppression decoding layer: receiving the class probability matrix and the position coordinate matrix, inputting the class probability matrix and the position coordinate matrix into an intersection-union elimination algorithm to perform redundant box filtering calculation to output a suspected bounding box.

[0010] Preferably, the specific generation process of the key image frame includes: extracting the planar pixel coordinate system vertex data of the suspected bounding box; performing coordinate system projection transformation on the planar pixel coordinate system vertex data to output infrared projection coordinate data and stereo frustum back-projection coordinate data; performing local cropping and stitching calculation on the image data, infrared data, and point cloud data based on the planar pixel coordinate system vertex data, infrared projection coordinate data, and stereo frustum back-projection coordinate data to output a visual alignment matrix; extracting the point cloud distribution data built into the visual alignment matrix; inputting the point cloud distribution data into the ground filtering module to perform background culling calculation to output non-ground target point cloud clusters; extracting depth limit values ​​based on the non-ground target point cloud clusters; generating a depth truncation filtering threshold based on the depth limit values; performing depth masking isolation calculation on the visual alignment matrix based on the depth truncation filtering threshold to output a target foreground feature matrix; and performing image encoding format conversion operation on the target foreground feature matrix to output the key image frame.

[0011] Preferably, the specific generation process of the warning information includes: inputting key image frames into the feature extraction layer of a deep classification model to perform matrix convolution calculation and output a high-dimensional semantic tensor; inputting the high-dimensional semantic tensor into a fully connected classification layer to perform probability mapping calculation and output a confirmed classification label and a deep recognition confidence value; performing a structured stitching operation on the confirmed classification label and the deep recognition confidence value to output a confirmation result; obtaining a preset intrusion risk judgment threshold value; performing a difference comparison calculation between the deep recognition confidence value and the preset intrusion risk judgment threshold value to output a crisis judgment result; and responding to the crisis judgment result that the deep recognition confidence value exceeds the preset intrusion risk judgment threshold value by performing a communication message encapsulation operation on the confirmation result and outputting the warning information.

[0012] Preferably, the specific process for generating the incremental samples includes: extracting the depth recognition confidence value built into the confirmation result; obtaining a preset absolute confidence safety threshold; subtracting the depth recognition confidence value from the preset absolute confidence safety threshold to output a confidence deviation value; obtaining a preset deviation warning threshold; comparing the confidence deviation value with the preset deviation warning threshold to output a deviation comparison result; if the deviation comparison result indicates that the confidence deviation value exceeds the preset deviation warning threshold, transmitting the key image frame to the review interaction interface for screen rendering; receiving the expert calibration bounding box and expert calibration classification label returned by the review interaction interface in response to the manual input action; combining and packaging the key image frame, expert calibration bounding box, and expert calibration classification label to output difficult image samples; obtaining a historically accumulated difficult sample set; and merging and appending the difficult image samples and the historically accumulated difficult sample set to output incremental samples.

[0013] Preferably, the specific process of updating the visual detection model includes: extracting the image feature pixel matrix, expert-calibrated bounding boxes, and expert-calibrated classification labels embedded in the incremental samples; obtaining the current running weight parameter set of the visual detection model; inputting the image feature pixel matrix into the current running weight parameter set to perform forward propagation calculation and outputting the predicted bounding box matrix and the predicted classification probability array; inputting the predicted bounding box matrix and the expert-calibrated bounding boxes into the smoothing absolute error formula to perform comparison calculation and outputting the bounding box regression loss value; inputting the predicted classification probability array and the expert-calibrated classification labels into the cross-entropy formula to perform error estimation calculation and outputting the classification cross-entropy loss value; performing addition and merging calculation on the bounding box regression loss value and the classification cross-entropy loss value to output the joint gradient loss value; performing backpropagation differentiation calculation based on the joint gradient loss value to output the weight gradient update direction data and the weight gradient update step size data; performing numerical superposition and replacement calculation on the current running weight parameter set based on the weight gradient update direction data and the weight gradient update step size data to output the updated network parameters; and performing a model overwrite storage operation based on the updated network parameters to update the visual detection model and the deep classification model simultaneously.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By extracting brightness parameters from image data and temperature parameters from infrared data, the brightness and temperature parameters are input into an environment analysis matrix to perform weighted calculations and output scene evaluation values. Based on state comparison results, routing calculations are performed on image data, infrared data, and point cloud data to output the visual features to be inspected. This effectively addresses the problem of fixed visual networks being eroded by the distribution shift of visual representations in outdoor scenes. Based on objectively acquired real brightness and temperature changes, multimodal data channels are adaptively scheduled to avoid the perception failure of a single visual channel under extreme weather or sudden changes in lighting conditions. By dynamically adapting the input structure of the underlying visual features, the attenuation of image recognition accuracy caused by drastic changes in the external physical environment is mitigated, ensuring the stability of perception capabilities under variable climate conditions.

[0015] 2. By performing spatial registration operations on image data, infrared data, and point cloud data based on suspected bounding boxes to output a visual alignment matrix, and then performing background cropping operations based on the visual alignment matrix to output key image frames, the system effectively alleviates the problem that the monitoring system struggles to maintain good feature extraction and high-precision image pattern matching in wide-area corridor scenarios. In complex outdoor power transmission line backgrounds, target features are easily interfered with by mountain or building backgrounds. By introducing spatial registration and truncation of multi-dimensional data, the system isolates background pixels at different depth levels in three-dimensional space during the generation of key image frames, purifies the semantic information of the input image, reduces the interference of spatial noise on visual feature extraction, and provides a reliable data foundation for subsequent high-precision image analysis.

[0016] 3. By performing difference verification calculations on the confirmed results and key image frames to output difficult image samples, performing aggregation operations on the difficult image samples to output incremental visual samples, performing parameter optimization calculations on the incremental visual samples to output updated recognition parameters, and performing parameter replacement operations on the updated recognition parameters to output an upgraded network, the problem of visual monitoring and analysis being difficult to maintain long-term and reliable operation is effectively solved. In response to the feature distribution shift caused by the morphological evolution of long-tail targets in wide-area scenes, the system automatically mines and re-feeds long-tail difficult samples through difference verification, and constructs an iterative update path for network parameters. This enables the recognition network to dynamically correct parameters based on the new data in actual operation, avoiding the cliff-like decline in classification accuracy of the fixed model after long-term deployment, thereby maintaining the visual monitoring capability that continuously adapts to scene changes.

[0017] 4. By constructing a closed-loop architecture that deeply couples scene perception, multi-dimensional feature purification, and incremental network updates, the problem of monitoring systems struggling to maintain long-term reliable operation in wide-area corridor environments is effectively solved. Scene-based routing calculations serve as pre-filters, adaptively matching perception modalities and allocating reasonable computing power for subsequent multi-dimensional processing. Background cropping based on the visual alignment matrix acts as a mid-level feature purification process, physically removing spatial noise and providing high-purity key image frames for the classification model. This high-purity feature directly determines the accuracy of confidence assessment in difference verification calculations, thus ensuring the true iterative value of outputting difficult image samples. This interconnected approach, from front-end dynamic perception and mid-level spatial denoising to back-end parameter optimization, achieves end-to-end synergy between data flow quality and the network evolution engine, enabling the visual monitoring system to possess inherent self-driving capabilities and continuous environmental adaptability. Attached Figure Description

[0018] Figure 1 This is a structural diagram of a machine vision-based transmission line external damage monitoring system proposed in an embodiment of this invention application; Figure 2 This is a flowchart of the environmental perception and multimodal dynamic routing proposed in an embodiment of this invention application; Figure 3 This is a flowchart of long-tailed hard sample mining and joint gradient update proposed in an embodiment of this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figures 1-3 The present invention provides a machine vision-based transmission line external damage monitoring system, the specific modules of which are as follows: Scene assessment module: acquires image data, infrared data, and point cloud data of the transmission line corridor; extracts global brightness parameters from the image data and local temperature difference values ​​from the infrared data; inputs the global brightness parameters and local temperature difference values ​​into the environment analysis matrix for weighted calculation and output of scene assessment values; The visual routing module acquires a baseline threshold, compares it with the scene evaluation value, and outputs the comparison result. Based on the comparison result, it performs routing assembly calculations on image data, infrared data, and point cloud data to output the features to be detected. It inputs the features to be detected into the visual detection model to perform target detection and outputs suspected bounding boxes. Based on the suspected bounding boxes, it performs spatial registration on image data, infrared data, and point cloud data to output a visual alignment matrix. Based on the visual alignment matrix, it performs background cropping to output key image frames. Incremental optimization module: Input key image frames into the deep classification model to predict and calculate the output confirmation result, and determine the intrusion conditions to output early warning information; Based on the early warning information, perform difference verification on the confirmation result and key image frames to output a difficult sample set; Perform aggregation on the difficult sample set to output incremental samples, and output updated network parameters through parameter differentiation to update the visual detection model.

[0021] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0022] Example 1 This application discloses a machine vision-based transmission line external damage monitoring system, see below. Figure 1 The specific modules proposed in this invention include: a scene evaluation module: acquiring image data, infrared data, and point cloud data of the transmission line corridor; extracting global brightness parameters from the image data and local temperature difference values ​​from the infrared data; inputting the global brightness parameters and local temperature difference values ​​into an environment analysis matrix for weighted calculation to output a scene evaluation value; a visual routing module: acquiring a benchmark threshold, comparing it with the scene evaluation value, outputting the comparison result, performing routing assembly calculation on the image data, infrared data, and point cloud data based on the comparison result to output the features to be detected; inputting the features to be detected into a visual detection model to perform target detection and output suspected bounding boxes; performing spatial registration on the image data, infrared data, and point cloud data based on the suspected bounding boxes to output a visual alignment matrix; performing background cropping based on the visual alignment matrix to output key image frames; and an incremental optimization module: inputting the key image frames into a deep classification model for prediction calculation to output confirmation results, and determining intrusion conditions to output early warning information; performing difference verification on the confirmation results and key image frames based on the early warning information to output a difficult sample set; performing aggregation on the difficult sample set to output incremental samples, and outputting updated network parameters through parameter differentiation to update the visual detection model.

[0023] Further, image data, infrared data, and point cloud data of the transmission line corridor are acquired; global brightness parameters from the image data and local temperature difference values ​​from the infrared data are extracted, and the global brightness parameters and local temperature difference values ​​are input into the environment analysis matrix for weighted calculation to output a scene evaluation value; this corresponds to the scene evaluation module; see [link / reference]. Figure 2 The specific implementation process includes: The process involves: extracting global pixel brightness values ​​from the image data, performing an average value calculation to output a global brightness parameter; extracting global temperature extreme values ​​from the infrared data, performing a subtraction operation to output a local temperature difference value; inputting the brightness parameter into a forward mapping function configured in the environment analysis matrix to perform a direct proportional calculation to output a local clarity coefficient; inputting the temperature parameter into a reverse mapping function configured in the environment analysis matrix to perform an inverse proportional calculation to output a local stability coefficient; obtaining preset brightness weight parameters and preset temperature weight parameters; performing a product correlation calculation on the local clarity coefficient and preset brightness weight parameters to output an environmental brightness assessment value; performing a product correlation calculation on the local stability coefficient and preset temperature weight parameters to output an environmental temperature assessment value; and performing an addition and merging calculation on the environmental brightness assessment value and environmental temperature assessment value to output a scene assessment value.

[0024] Specifically, the process for generating the scene evaluation values ​​is as follows: The field acquisition devices and hardware infrastructure supporting this system are typically deployed on or near power transmission towers, covering the transmission line corridor and the construction area below. The core of the hardware platform includes a high-resolution visible light camera, an infrared thermal imager, a solid-state lidar, and a control terminal responsible for edge computing. In addition, the system is equipped with a solar power unit, including batteries, a battery controller, and solar panels. The field devices also integrate alarm systems, such as a sound broadcast module, a buzzing vibration device, and a drone-linked communication interface. For data transmission, the system uses a 4G or 5G network communication module as the remote data transmission unit. The system's built-in status monitoring and risk prediction modules can predict risk locations based on real-time status information of the transmission line location, generate image acquisition routes accordingly, and control linked equipment for supplementary acquisition.

[0025] The system synchronously acquires three types of modal data of the transmission line corridor in each sampling period. The first type of input data is image data, acquired by a visible light camera. The data structure is a three-dimensional pixel matrix, with dimensions including the number of color channels, image height, and image width. The value of each pixel in the matrix is ​​quantized within the range of 0 to 255. The second type of input data is infrared data, acquired by an infrared thermal imager. The data structure is a single-channel temperature distribution matrix, where the values ​​in the matrix represent the radiation temperature at the corresponding spatial location. The third type of input data is point cloud data, calculated by lidar. The data structure is a discrete set of points containing three-dimensional spatial coordinates.

[0026] For image data, the module extracts the global pixel brightness values ​​and performs an average averaging operation. In this step, the system first converts the input image data into a single-channel grayscale matrix using a color space conversion process. Specifically, it extracts the three color channel values ​​for each pixel in the image matrix, multiplies the red channel value by 0.299, adds the green channel value by 0.587, and finally adds the blue channel value by 0.114. The sum of these three products is taken as the grayscale value corresponding to that pixel. After obtaining the global grayscale matrix, the arithmetic mean of the grayscale values ​​of all pixels in the matrix is ​​calculated, and the resulting data output is the global brightness parameter.

[0027] In this embodiment, before extracting the global brightness parameters, candidate highlight points are extracted and clustered into connected components. For overexposed pixel clusters with areas exceeding a threshold, the mean of the neighborhood is extracted and numerical replacement is performed to eliminate the brightness shift caused by the highlight reflection from the metal fittings, outputting the corrected global brightness parameters. Specifically, after obtaining the global grayscale matrix, the system introduces a highlight suppression mechanism to generate corrected global pixel brightness values. First, a preset highlight judgment threshold is set, fixed at 90% of the upper grayscale limit (i.e., 229.5) based on the characteristics of 8-bit images. The global grayscale matrix is ​​traversed, and pixels greater than or equal to 229.5 are extracted as candidate highlight points. Next, the candidate highlight points are clustered using an 8-neighbor connected component algorithm, and the pixel area of ​​each connected component is calculated. A highlight area filtering threshold of 20 pixels is set. This threshold is selected based on the statistical calculation of the average pixel area occupied by specular reflection spots on the surface of typical power fittings in the image at the current camera focal length. If the connected region area is less than 20, it is judged as random white noise and its original value is retained. If it is greater than or equal to 20, it is judged as an overexposed cluster of specular reflections from power fittings or insulators. For this overexposed cluster, a normal neighborhood ring extending outward by 2 pixels from its boundary is extracted, and the arithmetic mean of the grayscale values ​​of the pixels within this neighborhood ring is calculated. Subsequently, this neighborhood mean is used to replace the original highlight grayscale values ​​within the corresponding overexposed clusters one by one. After highlight replacement and noise reduction, the mean of the entire corrected grayscale matrix is ​​calculated to output the final global brightness parameters. The above process eliminates local overexposure interference caused by specular reflections on the surface of power facilities and prevents the global ambient brightness from being abnormally increased.

[0028] For infrared data, the module extracts the global temperature extremes and performs a subtraction operation. The system traverses the entire infrared temperature matrix, locates and extracts the highest and lowest temperature values, then calculates the absolute value of the difference between the two, and the resulting data output is the local temperature difference.

[0029] After successfully extracting the global brightness parameter and the local temperature difference, the system inputs these two parameters into the environmental analysis matrix for weighted calculation. The extracted global brightness parameter is then input into the forward mapping function configured in the environmental analysis matrix to perform a proportional calculation. The specific steps of this proportional calculation are as follows: obtain the previously extracted global brightness parameter, and simultaneously obtain the system's theoretical maximum brightness limit per pixel. Divide the global brightness parameter by this theoretical maximum brightness limit per pixel; the quotient is the local sharpness coefficient. The selection method for the theoretical maximum brightness limit per pixel is based on the standard attribute of the 8-bit image encoding format, and is fixed at 255.0. The local sharpness coefficient output by this calculation ranges from 0 to 1.

[0030] Secondly, the extracted local temperature difference is input into the inverse mapping function configured in the environmental analysis matrix to perform inverse proportional calculation. The specific calculation process of this inverse mapping function is as follows: the system obtains a preset inverse calibration constant as the dividend, and simultaneously obtains the extracted local temperature difference and a preset smoothing zero-prevention parameter. The sum of the local temperature difference and the smoothing zero-prevention parameter is used as the divisor. The inverse calibration constant is divided by this sum to obtain a preliminary quotient. Subsequently, a Sigmoid nonlinear activation function is used for normalization. The specific calculation process is as follows: using the natural constant as the base, the aforementioned preliminary quotient is used as the exponent for power operation to obtain the power value. One is divided by the sum of the power value and one, and the output result is the normalized local stability coefficient. It is important to note that in nighttime or extremely low-temperature boundary scenarios without obvious heat sources, the local temperature difference approaches zero. The initial quotient in the formula will become extremely large, dominated by the smoothing zero-prevention parameter. The local stability coefficient extracted after inputting into the Sigmoid function will approach zero. This can trigger a scenario evaluation value falling below the baseline threshold, forcibly activating the multimodal fusion branch (introducing infrared thermal imaging features), thus improving the reliability of nighttime monitoring. Regarding parameter selection: the inverse calibration constant is calculated based on the prior distribution of temperature differences in historical scenarios of the target area, typically chosen as the average of typical abnormal temperature differences, for example, set to 50.0, having the same numerical dimension as temperature. The smoothing zero-prevention parameter is selected as a very small positive real number to prevent the system from crashing due to a division by zero when the denominator is zero; it is typically set to 1.0 degrees Celsius or an equivalent value of 1.0, having the same numerical dimension as the temperature parameter.

[0031] Finally, preset brightness weight parameters and preset temperature weight parameters are obtained. The selection of these two weight parameters depends on historical statistical data of typical climate characteristics in the area where the device is deployed. For example, a higher temperature weight is used in foggy and rainy areas, and a higher brightness weight is used in well-lit areas. The sum of the two parameters is constant at 1.0. The system multiplies the previously obtained local clarity coefficient with the preset brightness weight parameters to output the ambient brightness assessment; simultaneously, it multiplies the local stability coefficient with the preset temperature weight parameters to output the ambient temperature assessment. The weighted calculation process is as follows: the ambient brightness assessment and the ambient temperature assessment obtained above are added and merged, and the output is the final scene assessment value.

[0032] By statistically analyzing the difference between the global average brightness and extreme temperature values, and combining this with forward and reverse mapping functions to calculate environmental clarity and stability coefficients, the system enables scene evaluation values ​​to quickly drop below the baseline threshold to trigger multimodal fusion when the lack of heat sources at night causes a sharp drop in extreme temperature differences. Conversely, in complex scenarios such as midday sun exposure where localized pseudo-temperature differences occur, the system maintains a high overall scene evaluation value by relying on extremely high environmental brightness evaluation values. This effectively filters out infrared thermal noise interference caused by sun exposure and stably calls single-modal branches. This approach transforms ambiguous natural environmental changes into quantifiable baseline data, avoiding interference from local extreme pixels and providing objective and accurate environmental state evaluation indicators for multimodal computing power routing.

[0033] Further, a baseline threshold is obtained and compared with the scene evaluation value. The comparison result is output. Based on the comparison result, routing assembly calculation is performed on image data, infrared data, and point cloud data to output the features to be detected. The features to be detected are input into the visual detection model to perform target detection and output suspected bounding boxes. Based on the suspected bounding boxes, spatial registration is performed on image data, infrared data, and point cloud data to output a visual alignment matrix. Based on the visual alignment matrix, background cropping is performed to output key image frames. This corresponds to the visual routing module. See also... Figure 2 The specific implementation process includes: A baseline threshold is introduced; the scene evaluation value is compared with the baseline threshold to output a state comparison result; in response to the state comparison result indicating that the scene evaluation value is lower than the baseline threshold, image data, infrared data, and point cloud data are input into a multimodal extractor to perform a stitching and merging operation to output a mixed modality aggregation matrix; the mixed modality aggregation matrix is ​​input into a channel attention module to perform a weighted summation calculation to output the feature to be detected; in response to the state comparison result indicating that the scene evaluation value is not lower than the baseline threshold, the image data is subjected to a feature channel mapping operation to output an independent image feature channel; the independent image feature channel is input into a visual feature extractor to perform a convolution extraction calculation to output the feature to be detected.

[0034] Specifically, the process of generating the features to be detected is as follows: The visual routing module obtains a baseline threshold and performs a numerical comparison calculation with the scene evaluation value output by the preceding calculation, outputting the state comparison result. The baseline threshold is selected based on statistical regression cross-validation analysis of the long-term system false negative rate and historical environment evaluation values, and is usually selected as a specific constant between 0.60 and 0.70. In this embodiment, it is set to 0.65.

[0035] If the response status comparison result indicates that the scene evaluation value is lower than the baseline threshold, the system inputs image data, infrared data, and point cloud data into the multimodal extractor. The point cloud data is first mapped into a two-dimensional depth matrix through depth projection. The specific projection mapping process is as follows: the system calls the pre-calibrated and stored camera intrinsic parameter matrix and LiDAR extrinsic parameter matrix, multiplies the three-dimensional spatial coordinates of each data point in the point cloud data with the extrinsic parameter matrix to transform it to the camera coordinate system, and then multiplies it with the intrinsic parameter matrix to map it to the two-dimensional pixel plane coordinate system; subsequently, the depth Z-axis value in the original three-dimensional coordinates is extracted and assigned to the corresponding mapped two-dimensional pixel plane grid cell to generate a structured two-dimensional depth matrix.

[0036] The multimodal extractor first uses the pixel coordinate system of the image data as a reference, and calls a pre-calibrated multi-sensor joint extrinsic parameter matrix to perform affine transformation and resampling interpolation on the two-dimensional depth matrix of the infrared data and point cloud data, aligning the spatial resolution and field of view of the three to the same physical reference. Then, it inputs a minimax normalization function into the image data (pixel grayscale), infrared data (Celsius), and two-dimensional depth matrix (distance in meters). The specific calculation process of this normalization function is as follows: extract the maximum and minimum terms in the current modality matrix, subtract the minimum term from the current pixel feature value to obtain the feature difference, and subtract the minimum term from the maximum term. The extreme value span difference is calculated. To prevent weak target features from being over-compressed and annihilated during normalization in extreme scenarios with severe background extreme value interference, the system introduces an extreme value span lower limit protection mechanism: For image data, infrared data, and point cloud data, a preset effective extreme value span lower limit constant corresponding to their modal dimensions is obtained (wherein, the extreme value span lower limit constant for image data is set based on the effective 8-bit grayscale contrast, for example, 50.0; the extreme value span lower limit constant for infrared data is set based on the minimum identifiable temperature difference between the target and the environment, for example, 10.0 degrees Celsius; the extreme value span lower limit constant for point cloud data is set based on the effective foreground depth distribution range, for example, 20.0 meters). The extreme value span difference calculated for the current modal feature is numerically compared with the corresponding preset effective extreme value span lower limit constant, and the maximum value of the two is taken as the final divisor span factor. Finally, the feature difference is divided by the divisor span factor to obtain a dimensionless result mapped to the interval between zero and one.

[0037] Subsequently, a feature dimension concatenation and merging operation is performed on these three aligned and dimension-neutralized modal data, i.e., stacking and combining at the data channel depth, to output a multi-channel mixed modality aggregation matrix containing three visible light image channels, one infrared temperature channel, and one spatial depth channel.

[0038] Subsequently, the system inputs the hybrid modality aggregation matrix into the channel attention module for weighted summation calculation. The specific calculation process is as follows: First, global average pooling is performed on the hybrid modality aggregation matrix containing three visible light image channels, one infrared temperature channel, and one spatial depth channel to compress it in the spatial dimension, outputting a one-dimensional channel statistical vector with a length equal to the total number of channels (i.e., 5). Next, this vector is input into a channel compression activation network consisting of two fully connected layers. The first fully connected layer compresses the channel dimension to the original number of channels divided by a preset reduction ratio (in this embodiment, the preset reduction ratio is set to 2, and rounding is used to balance computational power), and then activates it through a linear rectified function. The second fully connected layer restores the dimension to the original total number of channels. Then, the output is input into a Sigmoid activation function, mapping it to a real-valued weight parameter vector between zero and one. Finally, this weight parameter vector is multiplied point-by-point with the original hybrid modality aggregation matrix in the channel depth, performing weighted scaling, and outputting the features to be detected.

[0039] If the response state comparison result shows that the scene evaluation value is not lower than the baseline threshold, the system directly performs feature channel mapping operations on the acquired image data. Dimensionality reduction is performed using a convolutional kernel, outputting independent image feature channels. These channels are then input into a visual feature extractor, which extracts semantic information from the image through multi-layer convolutional downsampling operations. Finally, a feature tensor consistent with the multimodal branch dimension is output as the feature to be detected.

[0040] The mechanism dynamically switches between single-modal and multi-modal feature extraction paths based on scene evaluation values. In environments with good visibility, only the visual single channel is used to reduce the computational power consumption of the front-end device; in harsh environments, multi-modal fusion and channel attention calculation are triggered. This mechanism achieves a dynamic balance between computational overhead and target monitoring security.

[0041] The specific construction of the visual detection model includes: a deformation space alignment layer: receiving the features to be detected, generating pixel geometric offset vectors based on the features through convolution mapping, and performing adaptive deformation convolution calculation on the features to be detected based on the pixel geometric offset vectors to output a geometrically adaptive feature tensor; a cross-scale pyramid fusion layer: receiving the geometrically adaptive feature tensor, performing scale downsampling extraction calculation on the geometrically adaptive feature tensor to output a multi-scale semantic channel, and performing self-attention weighted association calculation on the multi-scale semantic channel to output a multi-scale feature matrix to be detected; a candidate box regression generation layer: receiving the multi-scale feature matrix to be detected, inputting the multi-scale feature matrix to be detected into a regression mapping network to perform coordinate offset decoding calculation to output a position coordinate matrix; a scale probability classification layer: receiving the multi-scale feature matrix to be detected, inputting the multi-scale feature matrix to be detected into a fully connected classification network to perform nonlinear activation mapping calculation to output a class probability matrix; and a boundary suppression decoding layer: receiving the class probability matrix and the position coordinate matrix, inputting the class probability matrix and the position coordinate matrix into an intersection-union elimination algorithm to perform redundant box filtering calculation to output a suspected bounding box.

[0042] Specifically, the internal processing of the visual inspection model is as follows: Deformation Space Alignment Layer: Receives the feature to be detected from the output of the routing module, inputs the feature to be detected into the offset prediction bypass branch composed of additional convolutional layers, and dynamically generates a pixel geometric offset vector that matches the spatial dimension of the feature to be detected by performing regular matrix convolution mapping calculation. This vector contains the two-dimensional floating-point offset values ​​of each sampling point in the feature receptive field on the horizontal and vertical coordinates.

[0043] Subsequently, adaptive deformation convolution is performed on the features to be inspected based on the pixel geometric offset vector. The calculation process is as follows: for each pixel at a specific location on the output feature map, the original regular grid coordinates on the corresponding input feature map are added with the previously extracted two-dimensional floating-point offset value to locate a new non-integer physical coordinate point; the feature pixel value at this non-integer coordinate point is obtained using a bilinear interpolation algorithm; finally, these interpolated feature pixel values ​​are multiplied one by one with the corresponding weight parameters inside the convolution kernel and summed to output a geometrically adaptive feature tensor.

[0044] Cross-scale pyramid fusion layer: Receives the geometrically adaptive feature tensor output from the upper layer and performs scale downsampling extraction calculations. Multi-scale semantic channels containing different spatial resolution levels are extracted through multiple pooling layers and convolutional operations. Self-attention weighted association calculations are then performed based on these multi-scale semantic channels. The specific calculation process is as follows: First, through bilinear interpolation upsampling, the low-resolution multi-scale semantic channels are uniformly aligned to the spatial size of the highest resolution feature map, and then stitched together along the channel dimension to form a fused feature map. Subsequently, the fused feature map is input into three independent linear mapping layers, and the projection generates a query matrix, key matrix, and value matrix with the channel dimension reduced to one-quarter of the original. The query matrix and key matrix are transposed and multiplied, and then the total number of feature channel dimensions of the aforementioned query matrix is ​​obtained as the projection dimension constant (this constant is determined based on the network channel configuration; for example, when the number of channels in the input fused feature map is 256, the projection dimension constant after dimension reduction is selected as 64). The numerical result of the aforementioned matrix multiplication is divided by the square root of the projection dimension constant (i.e., divided by 8) to prevent gradient vanishing, and then normalized by the Softmax function to generate a spatial attention weight distribution. The spatial attention weight distribution is multiplied by the value matrix, and finally added to the original fused feature map through residual connection to output a multi-scale detection feature matrix.

[0045] Candidate Box Regression Generation Layer: This layer receives the multi-scale feature matrix from deep fusion in parallel and inputs it into the regression mapping network. The specific steps for performing coordinate offset decoding calculation are as follows: Based on pre-configured anchor boxes with different fixed aspect ratios, the network calculates the horizontal and vertical offset values ​​of the center point of the current feature point target corresponding to the preset anchor box, as well as the logarithmic scaling factor values ​​relative to the width and height of the preset anchor box. These predicted horizontal and vertical offset values ​​are multiplied by the width and height of the preset anchor box respectively, and then superimposed onto the center point coordinates of the preset anchor box to calculate the actual center coordinates of the predicted box. Simultaneously, using the natural constant as the base, the logarithmic scaling factor values ​​of the width and height are used as exponents to perform power operations, yielding the corresponding physical scaling coefficients for width and height. These physical scaling coefficients are multiplied by the base width and height of the preset anchor box respectively to calculate the actual width and height of the predicted box. Finally, the actual center coordinates and the actual width and height are combined to output the position coordinate matrix.

[0046] Scale-based probabilistic classification layer: This layer receives a multi-scale feature matrix to be detected and inputs it into a fully connected classification network to perform nonlinear activation mapping calculations. An activation function maps the network's output feature vectors to a discrete, predefined intrusion category probability distribution. The output is a category probability matrix, which records the classification confidence score corresponding to the predicted bounding box generated from the location coordinates.

[0047] Boundary Suppression Decoding Layer: The system receives the class probability matrix and location coordinate matrix output from the upstream and inputs them into the Intersection over Union (IoU) elimination algorithm. The algorithm's filtering calculation process is as follows: Based on the class probability matrix, bounding boxes of the same class are sorted in descending order of score, and the baseline bounding box with the highest score is retained; subsequently, the IoU of the remaining bounding boxes with the baseline bounding box is calculated. The specific calculation method for the intersection-union ratio (IUR) is as follows: Calculate the independent physical areas of the two bounding boxes and sum them. Then, subtract the physical area of ​​the aforementioned overlapping region from the sum of these areas. Specifically, the physical area of ​​the overlapping region is calculated as follows: Compare the horizontal and vertical coordinates of the top-left corner vertices of the two bounding boxes, and take the larger value as the top-left horizontal and vertical coordinates of the overlapping region; compare the horizontal and vertical coordinates of the bottom-right corner vertices of the two bounding boxes, and take the smaller value as the bottom-right horizontal and vertical coordinates of the overlapping region; subtract the top-left horizontal coordinate from the bottom-right horizontal coordinate to obtain the overlap width, and subtract the top-left vertical coordinate from the bottom-right vertical coordinate to obtain the overlap height; in response to the condition that both the overlap width and overlap height are greater than zero, multiply them to obtain the physical area of ​​the overlapping region. The total area covered by the two bounding boxes is obtained. Finally, divide the physical area of ​​the overlapping region by this total area to obtain the IUR value.

[0048] In response to a cross-union ratio (CUI) greater than a preset overlap threshold, the bounding boxes are identified as redundant and filtered out. The preset overlap threshold is selected based on the statistical settings of the cluster density and overlap occlusion rate of target machines in historical real-world datasets, typically between 0.45 and 0.50. The final output consists of non-overlapping suspected bounding boxes, represented by a six-element data set including the minimum x-coordinate, minimum y-coordinate, maximum x-coordinate, maximum y-coordinate, category identifier, and confidence score.

[0049] Introducing a deformation space alignment layer and a self-attention mechanism into the visual detection model, deformation convolution can adaptively fit the geometric features of irregular externally broken targets such as crane arms, while the multi-scale self-attention mechanism effectively alleviates the feature confusion problem in complex mountain and forest backgrounds and improves the feature capture ability of the underlying network for irregular targets.

[0050] Extract the vertex data of the suspected bounding box in the planar pixel coordinate system; perform coordinate system projection transformation on the vertex data to output infrared projection coordinate data and stereo frustum back-projection coordinate data; perform local cropping and stitching calculation on the image data, infrared data, and point cloud data based on the vertex data, infrared projection coordinate data, and stereo frustum back-projection coordinate data to output a visual alignment matrix; extract the point cloud distribution data built into the visual alignment matrix; input the point cloud distribution data into the ground filtering module to perform background removal calculation to output non-ground target point cloud clusters; extract the depth limit value based on the non-ground target point cloud clusters; generate a depth truncation filtering threshold based on the depth limit value; perform depth masking isolation calculation on the visual alignment matrix based on the depth truncation filtering threshold to output a target foreground feature matrix; perform image encoding format conversion operation on the target foreground feature matrix to output key image frames.

[0051] Specifically, the generation process of the key image frames is as follows: The system extracts the vertex data of the planar pixel coordinate system of the suspected bounding box output by the previous module, that is, obtains the horizontal and vertical coordinate data of the upper left and lower right corners of the two-dimensional rectangle.

[0052] Subsequently, a coordinate system projection transformation calculation is performed on the vertex data of the planar pixel coordinate system. The specific calculation process is as follows: The pre-stored device intrinsic and extrinsic parameter matrices are called, and the inverse of the intrinsic parameter matrix is ​​multiplied by the input two-dimensional pixel coordinate data, projecting it back onto the normalized camera three-dimensional coordinate system space. Then, the result is multiplied by the extrinsic parameter matrix containing rotation and translation parameters, mapping it to the infrared sensor space to output infrared projection coordinate data. Simultaneously, the system utilizes the three-dimensional frustum back-projection geometry principle, extending the four boundary rays of the two-dimensional rectangle in the depth direction for calculation, outputting stereo frustum back-projection coordinate data.

[0053] Based on the three types of coordinate data acquired above, the system simultaneously performs local cropping on the original image data, infrared data, and point cloud data. The cropped multimodal data is then aligned at the pixel level under a unified physical spatial reference, and a matrix overlay stitching operation is performed along the data channel dimension to output a visual alignment matrix.

[0054] The point cloud distribution data built into the visual alignment matrix is ​​extracted and input into the ground filtering module to perform background removal calculations. The calculation process involves iteratively selecting a point set and substituting it into the 3D spatial plane equation for fitting and verification. This spatial plane equation is constructed as follows: the horizontal, vertical, and depth coordinates of the spatial data points are multiplied by their corresponding three constant plane coefficients, summed, and then a fourth constant term is added, resulting in a value of zero. These three constant plane coefficients and the fourth constant term are dynamically generated by the system through multiple rounds of iterative fitting operations using a random sampling consistency algorithm based on the extracted 3D point cloud data set, representing the optimal ground plane normal vector and intercept parameters for the current scene.

[0055] Calculate the vertical spatial distance from the point cloud data point to the optimal plane equation, obtain a preset ground fit tolerance threshold (e.g., set to 0.15 meters, calculated based on real ground undulations and radar ranging noise), and in response to the condition that the vertical spatial distance is less than or equal to the ground fit tolerance threshold, determine that the data point belongs to the ground point cloud that meets the conditions and peel it off, and output a non-ground target point cloud cluster.

[0056] In this embodiment, after acquiring the non-ground target point cloud cluster, a local search radius and density threshold are set. The number of neighboring point clouds is counted using a KD-Tree, low-density points are classified as edge points, and the Euclidean distance between edge points and high-density points is calculated to remove isolated floating noise points exceeding the threshold. Specifically, after acquiring the non-ground target point cloud cluster, the system introduces a density clustering denoising algorithm for secondary purification. The specific calculation steps are as follows: First, the local neighborhood search radius is set to a constant of 0.5 meters, and the core density point threshold is set to 15 points. The system uses the KD-Tree data structure to traverse each three-dimensional coordinate point in the non-ground target point cloud cluster, calculating the number of other point clouds contained within a spherical space with the current point as the center and a radius of 0.5 meters. If the number is greater than or equal to 15, the current point is marked as a high-density core point; if the number is less than 15, it is marked as a low-density edge point. Subsequently, the 3D Euclidean distance from all low-density edge points to the nearest high-density core point is calculated. Specifically, the coordinate differences between the two points on the horizontal, vertical, and depth axes in 3D space are calculated separately. These three differences are squared, summed, and then the square root of the sum is taken to obtain the final 3D Euclidean distance. A suspension noise isolation threshold of 1.2 meters is set. If the calculated Euclidean distance is greater than 1.2 meters, the edge point is determined to be an artifact caused by wind deflection of power transmission cables or an isolated bird crossing the field of view, and is permanently removed from the non-ground target point cloud cluster. The filtered compact point cloud is then input into the subsequent depth limit value extraction process. The selection method for the local neighborhood search radius constant and the core density point threshold is based on the statistical calibration of the scanning beam density of the lidar equipment and the 3D spatial point distance distribution of typical cable targets. The selection method for the suspension noise isolation threshold is based on the calculation of the maximum offset distance caused by the historical maximum wind deflection in the cable point cloud. The above process removes isolated suspended point clouds caused by cable wind deflection and birds, preventing the target depth truncation identification interval from being incorrectly expanded.

[0057] Based on this non-ground target point cloud cluster, the coordinate values ​​of the closest and farthest points to the camera sensor are extracted along the depth direction, and these two values ​​are recorded as depth limit values. Based on the depth limit values, a tolerance safety margin is subtracted and added to both the near and far ends to generate the effective recognition range of depth, i.e., the depth truncation filtering threshold. The tolerance safety margin is selected by comprehensively calibrating based on the average physical size error of common engineering vehicle targets and the ranging accuracy error of the LiDAR hardware; in this example, it is selected as 1.5 meters.

[0058] Subsequently, depth masking isolation calculation is performed on the visual alignment matrix based on the aforementioned depth truncation filtering threshold. The specific calculation process is as follows: traverse all data points within the visual alignment matrix, extract their corresponding depth coordinate values, and compare these depth coordinate values ​​with the previously generated depth truncation filtering threshold range. In response to a depth coordinate value not falling within this range, the image feature value and infrared feature value corresponding to that data point are forcibly modified to zero. After isolation processing, a target foreground feature matrix stripped of background features is output.

[0059] Finally, the target foreground feature matrix is ​​converted to an image encoding format to generate key image frames that meet the feature extraction criteria.

[0060] By constructing a stereo frustum back projection using 2D plane vertex data and extracting depth limit values ​​by filtering out the ground in the point cloud, a depth mask is generated for background isolation. This process removes non-target pixels from the 3D physical space level, improving the semantic purity of the image frame.

[0061] Furthermore, key image frames are input into a deep classification model to predict and calculate the output confirmation result, and intrusion conditions are determined to output early warning information. Based on the early warning information, the confirmation result and key image frames are compared to output a difficult sample set. The difficult sample set is aggregated to output incremental samples. By calculating the parameters, the updated network parameters are output to update the visual detection model; this corresponds to the incremental optimization module; see [link to module]. Figure 3 The specific implementation process includes: The key image frames are input into the feature extraction layer of the deep classification model to perform matrix convolution calculation and output a high-dimensional semantic tensor. The high-dimensional semantic tensor is input into the fully connected classification layer to perform probability mapping calculation and output a confirmed classification label and a depth recognition confidence value. The confirmed classification label and the depth recognition confidence value are then subjected to a structured stitching operation to output a confirmation result. A preset intrusion risk judgment threshold value is obtained. The depth recognition confidence value and the preset intrusion risk judgment threshold value are compared to calculate the difference and output a hazard judgment result. If the hazard judgment result is that the depth recognition confidence value exceeds the preset intrusion risk judgment threshold value, the confirmation result is subjected to a communication message encapsulation operation to output a warning message.

[0062] Specifically, the process of generating early warning information is as follows: The system inputs the aforementioned key image frames into the feature extraction layer of the deep classification model, performs multi-layer matrix convolution calculations, and outputs a high-dimensional semantic tensor containing local structural information of the target.

[0063] The high-dimensional semantic tensor is input into a fully connected classification layer to perform probability mapping calculations. The model outputs two core data points: a confirmation classification label and a deep recognition confidence score. A structured concatenation operation is then performed on the confirmation classification label and the deep recognition confidence score, combining them into a single data object, which is recorded as the confirmation result.

[0064] Obtain the preset intrusion risk threshold value. The selection method for this threshold value is based on a comprehensive assessment of the importance level of the protected transmission line assets and the acceptable false alarm frequency of the equipment at the dispatch center. For example, 0.50 is selected for high-voltage trunk lines, and 0.80 is selected for conventional distribution networks.

[0065] The system compares the currently obtained depth recognition confidence score with the preset intrusion risk threshold and outputs a crisis determination result. If the crisis determination result indicates that the depth recognition confidence score is greater than the preset intrusion risk threshold, the system will confirm the result by performing communication message encapsulation according to the network communication protocol, generating a warning message with a timestamp and device code, and transmitting it externally.

[0066] By performing feature mapping on key image frames through an independent classification model, pure classification labels and deep recognition confidence scores are output, reducing mutual interference between feature expressions. Critical judgment is performed through strict confidence threshold comparison, effectively filtering false alarms detected by the front-end network and reducing the false alarm rate of the monitoring system.

[0067] Extract the depth recognition confidence value built into the confirmation result; obtain a preset absolute confidence safety threshold; subtract the depth recognition confidence value from the preset absolute confidence safety threshold to output a confidence deviation value; obtain a preset deviation warning threshold; compare the confidence deviation value with the preset deviation warning threshold to output a deviation comparison result; if the deviation comparison result indicates that the confidence deviation value is greater than the preset deviation warning threshold, transmit the key image frame to the review interaction interface for screen rendering; receive the expert calibration bounding box and expert calibration classification label returned by the review interaction interface in response to the manual input action; combine and package the key image frame, expert calibration bounding box, and expert calibration classification label to output difficult image samples; obtain a historically accumulated difficult sample set; merge and append the difficult image samples and the historically accumulated difficult sample set to output incremental samples.

[0068] Specifically, the incremental sample generation process is as follows: Extract the deep recognition confidence score value built into the confirmation result. The system obtains the preset absolute confidence threshold. The selection method of this threshold is based on the expected lower confidence limit index of classification accuracy in the monitoring system product definition, which is usually selected as 0.90 or 0.95.

[0069] The system subtracts the depth recognition confidence score from a preset absolute confidence safety threshold. The specific steps are as follows: The preset absolute confidence safety threshold is used as the minuend, and the currently output depth recognition confidence score is used as the subtrahend; a direct subtraction operation is then performed. If the difference is negative (i.e., the current confidence score is extremely high, exceeding the safety threshold), the difference is forcibly truncated to zero; if the difference is positive, the positive difference is directly output as the confidence deviation score.

[0070] The system obtains a preset deviation warning threshold. This threshold is selected based on a balance between the daily workload limit of manual reviewers in the back-end monitoring center and the recall rate of long-tail anomaly data; the example value is set to 0.10. The system compares the calculated confidence deviation value with the preset deviation warning threshold to generate a deviation comparison result. If the confidence deviation value exceeds the preset deviation warning threshold, the system determines that the current feature is a difficult-to-identify target and transmits the key image frame to the review interaction interface.

[0071] In this embodiment, after the deviation exceeds the limit, the system extracts the confidence deviation between the current frame and the previous four historical frames to form a time-series sliding window, and calculates the fluctuation variance of the five frames of data. If the variance exceeds the limit, it is judged as occasional jitter and discarded; if the variance is lower than the threshold, it is confirmed as a real difficult sample. Specifically, in response to the condition that the confidence deviation value exceeds the preset deviation warning threshold, the system introduces a time-series fluctuation smoothing mechanism to perform secondary screening of real and fake difficult samples. The specific calculation process is as follows: the system maintains a time-series sliding window of length 5 in memory, extracts the confidence deviation values ​​corresponding to the current key image frame and the four adjacent previous consecutive key image frames, forming a time-series deviation array containing five floating-point numbers. Then, the average value of the time-series deviation array is calculated, the difference between each deviation value and the arithmetic mean is calculated and squared, the five squared values ​​are added together and divided by the difference of the total number of values ​​minus one (i.e., divided by 4), to obtain the sample fluctuation variance value. The threshold for determining pseudo-difficult fluctuation variance is selected based on the historical background confidence floor variance statistical mean of the monitoring system under non-intrusive natural wind and grass movement conditions; in this example, it is set to 0.02. If the calculated fluctuation variance is greater than 0.02, the system determines that the current high deviation is due to an occasional abrupt change caused by the target being briefly obscured by trees or by the camera's momentary mechanical shaking due to strong winds, and has no long-term feature iteration value, so the image frame is directly discarded. If the fluctuation variance is less than or equal to 0.02, the system determines that the target stably exhibits identification difficulties over a continuous time series, confirms it as a genuine distribution offset sample, and transmits it to the review interaction interface. The above process filters out pseudo-difficult samples caused by brief obstruction or camera shaking, avoiding low-quality abrupt change data contaminating the network.

[0072] The review interaction interface receives key image frames and performs screen rendering operations. It also receives data returned from the review interaction interface in response to manually entered coordinates and text options, including precise expert-calibrated bounding box coordinates and expert-calibrated classification label text.

[0073] The system performs a data assembly and packaging operation on key image frames, expert-calibrated bounding boxes, and expert-calibrated classification labels to generate a single difficult image sample. Finally, it performs a data merging and appending operation on this sample and the historically accumulated difficult sample set, outputting an updated incremental sample for subsequent processing.

[0074] By using confidence bias values ​​to screen ambiguous suspected targets and introducing an interactive review interface to obtain expert calibration results when the limit is exceeded, high-risk long-tail data that is prone to causing model drift is intercepted. A high-quality difficult sample set is constructed manually, breaking the confirmation bias caused by unsupervised closed loop.

[0075] Extract the image feature pixel matrix, expert-calibrated bounding boxes, and expert-calibrated classification labels from the incremental samples; obtain the current running weight parameter set of the visual detection model; input the image feature pixel matrix into the current running weight parameter set to perform forward propagation calculation and output the predicted bounding box matrix and predicted classification probability array; input the predicted bounding box matrix and expert-calibrated bounding boxes into the smoothing absolute error formula to perform comparison calculation and output the bounding box regression loss value; input the predicted classification probability array and expert-calibrated classification labels into the cross-entropy formula to perform error estimation calculation and output the classification cross-entropy loss value; perform addition and merging calculation on the bounding box regression loss value and the classification cross-entropy loss value to output the joint gradient loss value; perform backpropagation differentiation calculation based on the joint gradient loss value to output the weight gradient update direction data and weight gradient update step size data; perform numerical superposition and replacement calculation on the current running weight parameter set based on the weight gradient update direction data and weight gradient update step size data to output the updated network parameters; perform model overwrite storage operation based on the updated network parameters to update the visual detection model and deep classification model simultaneously.

[0076] Specifically, the process of simultaneously updating the visual detection model and the deep classification model is as follows: The system extracts the built-in image feature pixel matrix, expert-calibrated bounding boxes, and expert-calibrated classification labels from the incremental samples. It also extracts the current running weight parameter set from the visual detection model. The image feature pixel matrix is ​​input into the current running weight parameter set to perform forward propagation calculations, outputting a predicted bounding box matrix and a predicted classification probability array.

[0077] The system performs error calculations for two types of tasks separately. For localization deviation, the system inputs the predicted bounding box matrix and the expert-calibrated bounding box into the smoothed absolute error formula. The specific comparison and calculation process of this formula is as follows: the pixel coordinates of the predicted and expert-calibrated bounding boxes are divided by the absolute width and height of the image, converting them into dimensionless, relatively normalized coordinates; the values ​​of the predicted and expert-calibrated bounding boxes are compared one by one in each coordinate dimension, and the absolute error difference between the two is obtained through subtraction. If the absolute error difference is less than one, it is squared and multiplied by 0.5 to obtain the output result; if the absolute error difference is greater than or equal to one, it is subtracted by the constant 0.5 to obtain the output result. The results of each dimension are summed to output the bounding box regression loss value.

[0078] To address classification bias, the system inputs the predicted classification probability array and expert-labeled classification labels into the cross-entropy formula. The error estimation calculation process of this formula is as follows: from the predicted classification probability array output by the model, locate and extract the single probability value corresponding to the true expert-labeled category; calculate the natural logarithm of the extracted single probability value; then multiply the obtained natural logarithm by negative one and take the opposite number, and output the final result as the classification cross-entropy loss value.

[0079] The system acquires preset first task balance coefficients and second task balance coefficients. These coefficients are selected based on a dynamic adjustment of the order-of-magnitude ratio between the localization loss and classification loss values ​​during the model pre-training phase. For example, in this embodiment, the first task balance coefficient is selected as 1.0, and the second task balance coefficient as 0.5. The system performs the addition and merging calculation process as follows: multiply the bounding box regression loss value by the first task balance coefficient to obtain the first product; multiply the classification cross-entropy loss value by the second task balance coefficient to obtain the second product; and then perform an addition operation on the first and second products to obtain the joint gradient loss value.

[0080] Based on the joint gradient loss value, the system kernel engine performs backpropagation differentiation calculation. Following the chain rule, it calculates the partial derivatives of each parameter in the network with respect to the total loss, obtaining a derivative vector, which is then output as the weight gradient update direction data.

[0081] The system obtains the weight gradient update step size data. The step size data is selected based on the fine-tuning training rule of avoiding catastrophic forgetting in deep learning models, and strictly selects floating-point numbers with extremely small magnitudes. In the example, it is set to 0.00001.

[0082] Subsequently, the system performs numerical superposition and replacement calculations on the currently running weight parameter set. The specific replacement calculation steps are as follows: the system iterates through and extracts each weight value in the currently running weight parameter set, calculates the product of the aforementioned weight gradient update step size data and the corresponding dimension's weight gradient update direction data, subtracts this product from the currently extracted weight value, and outputs the result as the updated network parameters. Finally, based on these updated network parameters, a storage instruction is executed to write the new parameter series to the system's physical file, performing a model overwrite storage operation to complete the visual inspection model update process.

[0083] Simultaneously, the system performs synchronous closed-loop updates to the deep classification model based on the same incremental samples. The specific process is as follows: key image frames from the incremental samples are extracted and input into the feature extraction layer and fully connected classification layer of the deep classification model; forward propagation is performed to output the current predicted classification probability; this current predicted classification probability and the expert-calibrated classification labels from the incremental samples are input into the cross-entropy formula for calculation, outputting the deep classification update loss value; based on this deep classification update loss value, backpropagation is performed to calculate the gradient of the weight parameters of the deep classification model with the same step size of 0.00001; and the current running weight parameter set of the deep classification model is numerically superimposed and replaced, completing the network upgrade of the deep classification model. This dual-path parameter overwriting mechanism ensures the synchronous improvement of target localization and category judgment capabilities during the evolution process.

[0084] In the specific implementation, for model overlay storage operations, in order to avoid the huge instantaneous computing power overhead caused by backpropagation differentiation and the resulting memory overflow of edge computing terminals, the system adopts a time-sharing computing power scheduling mechanism: the incremental samples generated daily are temporarily stored in the local flash memory queue; during the night when visible light devices are in hibernation or when there are no suspected targets and the high-frequency computing power is idle, the system suspends the high-energy-consuming visible light inference process to release the video memory resources of the edge devices, and then loads the model copy in memory to independently perform the above-mentioned backpropagation differentiation and parameter value superposition replacement; after the update verification is passed, the main control program performs an atomic switch of the memory address pointer during the wake-up period the next morning to redirect the inference request.

[0085] By combining the smoothing absolute error formula and the cross-entropy formula to calculate the combined gradient loss of the bounding box and classification, which is consistent with the backpropagation logic of the underlying object detection network, the model can directly use expert knowledge in incremental samples to correct the front-end weights, thus realizing an online closed-loop upgrade of the system's underlying recognition capabilities.

[0086] The core visual detection model of the system adopts a multi-stage cascaded architecture, which includes: a deformation space alignment layer that receives the above-mentioned stitched features and inputs additional side-channels to generate two-dimensional floating-point offset values ​​for adaptive deformation feature extraction; a cross-scale pyramid fusion layer that performs scale downsampling through multiple pooling and convolution, calculates pixel correlations to perform self-attention weighted association, and outputs a multi-scale feature matrix to be detected; a candidate box regression generation layer that decodes preset anchor boxes by performing offset and logarithmic scaling factor decoding by a regression mapping network, and obtains the position coordinate matrix by combining exponential operation; a scale probability classification layer that performs nonlinear activation mapping by a fully connected network to obtain the class probability matrix; and a boundary suppression decoding layer that calculates the intersection-union ratio overlap area based on the independent physical area subtraction and minimax coordinate comparison algorithm to perform redundancy filtering. In addition, the deep classification model in the backend of the system consists of stacked residual convolutional blocks connected in series to form the feature extraction backbone. Each residual block contains a 3x3 regular matrix convolution, batch normalization and non-linear activation unit. Skip connections are introduced to add the input features and the residual mapping results. The high-dimensional semantic tensor output is reduced in dimensionality by global average pooling and then connected to a fully connected classification layer to perform probability mapping, outputting the final confirmed classification label.

[0087] Secondly, regarding the offline pre-training process before the model goes live. Before the system is officially deployed and the aforementioned incremental optimization operations are performed, the visual detection model and the deep classification model need to undergo complete initial model training. The specific steps are as follows: Construct an initial multimodal image and point cloud sample set containing typical historical power transmission line damage scenarios. Each training sample in this initial sample set contains visible light image frames, infrared temperature frames, and point cloud depth frames that are strictly physically synchronized and collected at the same time, and covers no less than a preset order of magnitude (e.g., two thousand per class) of typical intrusion target instances such as large wheeled cranes and tracked excavators; and manually label them with expert bounding boxes and classification labels; initialize all network weight parameters of the model using a random normal distribution; input the initial multimodal samples into the untrained network to perform forward propagation calculation, and calculate the initial bounding box regression loss and classification cross-entropy loss according to the aforementioned smoothing absolute error formula and cross-entropy formula, respectively, and sum them to form the joint gradient loss; Subsequently, an adaptive moment estimation optimizer is employed, with an initial learning rate constant set to 0.001. Backpropagation is performed based on the joint gradient loss to calculate derivatives and update all network weights. After each round of forward and backward iterations on the full initial sample set, the mean accuracy is calculated on an independent validation set. If the mean accuracy fails to improve for five consecutive rounds or reaches the preset limit for the total number of iterations, an early stopping mechanism is triggered to stop training. Finally, the network parameters at the time of training stoppage are physically stored and output as the current running weight parameter set for the visual detection model and the deep classification model, serving as the base model support for online monitoring inference and subsequent incremental optimization.

[0088] This invention provides a machine vision-based transmission line external damage monitoring system. By constructing a closed-loop architecture of "scene-aware routing - cross-modal noise reduction - long-tail sample self-evolution," it solves the problem of long-term reliable operation of monitoring systems in complex outdoor environments. First, by integrating brightness and temperature parameters to adaptively evaluate the environment, on-demand routing of the perception modality is achieved, balancing low power consumption of edge devices with high robustness under harsh weather conditions. Second, by utilizing target bounding boxes to drive multi-dimensional spatial registration and depth clipping, background noise is physically removed, improving feature purity against complex transmission line backgrounds. Finally, a difficult sample mining and parameter update mechanism based on prediction difference verification is constructed, enabling the visual network to continuously self-correct the evolving external damage morphology in a wide-area corridor, avoiding network aging and contributing to long-term stable monitoring capabilities.

[0089] Example 2 This embodiment uses a machine vision-based transmission line external damage monitoring system applied to a 500 kV ultra-high voltage transmission corridor in a mountainous area as an example to illustrate the system's lightweight operation and deep stripping mechanism under conditions of sufficient lighting but extremely complex background (with a large number of tall trees blocking the light). The specific implementation process is as follows: One day at noon, direct sunlight caused extremely high brightness in the visible light image. At the same time, a large wheeled crane drove into the woodland 30 meters below the power line, preparing to illegally carry out hoisting operations. On-site equipment simultaneously collected image data, infrared data, and point cloud data.

[0090] The system extracts a global average grayscale value of 210.0 from the image, and after direct proportional calculation, the local sharpness coefficient is 0.82. Simultaneously, the extracted infrared temperature extreme difference is 44.5 degrees Celsius, and after inverse proportional calculation, the normalized local stability coefficient is 0.75. After weighting the parameters with equal weights, the system outputs a scene evaluation value of 0.785.

[0091] The system compares the scene evaluation value of 0.785 with the preset benchmark threshold of 0.65. Since the evaluation value is significantly higher than the benchmark threshold, the system determines that the current visible light conditions are extremely favorable, automatically cutting off the computationally intensive multimodal fusion branch and entering the single-modal purification branch. The system only inputs high-resolution visible light image data into the convolutional network for dimensionality reduction and outputs independent feature channels, which significantly saves the instantaneous power consumption of the edge computing terminal.

[0092] When the crane is in operation, its 20-meter-long metal boom appears in the picture as an extremely irregular, slender, tilted shape, intertwined with the dense tree branches in the background.

[0093] After the single-modal features are entered into the deformation space alignment layer, the network extracts irregular two-dimensional pixel offset values ​​for the slender crane boom. Based on these offset values, the convolutional kernel undergoes an elongated adaptive deformation, closely conforming to the crane boom contour to extract corresponding features, effectively avoiding the inclusion of side branches into the feature matrix. After multi-scale pyramid fusion and intersection-union area elimination calculations, the system finally outputs the unique suspected bounding box surrounding the crane.

[0094] To completely eliminate interference from background tree branches, the system extracts the vertex coordinates of the 2D bounding box and, combined with the camera's intrinsic and extrinsic parameter matrices and a view frustum back-projection geometric algorithm, maps the 2D box to a 3D point cloud space. After the ground filtering module removes forest soil based on the 3D spatial plane fitting equation, the system calculates the depth limits of the crane and surrounding trees to be 28.0 meters at the near end and 35.0 meters at the far end. The system, with a 1.5-meter tolerance, generates an effective depth range, forcibly clearing all background pixel features of the tree canopy outside this range to zero, outputting only clean, focused image frames of the crane body and boom.

[0095] After key image frames are input into the deep classification model, the model can identify the unique local features of the crane's tires and hydraulic support legs because the background foliage has been effectively filtered out by depth masking. The fully connected classification layer output confirms the classification label as "large wheeled crane" with a confidence score of 0.85. Since the hazard threshold for the critical 500 kV line is strictly set at 0.50, the system immediately triggers an alarm after comparison, sending a warning message containing time and coordinates to the dispatch center and simultaneously activating on-site loudspeakers for acoustic eviction.

[0096] In subsequent closed-loop optimization, the system calculated that the difference between the confidence score of 0.85 and the preset absolute safety threshold of 0.95 was 0.10, which just touched the system's set deviation warning threshold of 0.10. The system automatically determined that the sample was a difficult sample affected by tree shadows and pushed it to the human review screen. Human experts used the interactive interface to re-select the blurred edges where the boom end overlapped with the tree branches with extremely high precision coordinates, and added expert classification labels to generate incremental samples.

[0097] During the equipment's sleep period at night, the system extracts the incremental sample and uses the smoothed absolute error formula and cross-entropy formula for forward inference and multi-task joint loss calculation. Finally, partial derivative calculation and parameter overwriting are completed.

[0098] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A machine vision-based transmission line external damage monitoring system, characterized in that, include: Scene assessment module: acquires image data, infrared data and point cloud data of the transmission line corridor; extracts global brightness parameters from the image data and local temperature difference values ​​from the infrared data, inputs the global brightness parameters and local temperature difference values ​​into the environment analysis matrix for weighted calculation and output of scene assessment value; Visual routing module: Obtains a baseline threshold, compares it with the scene evaluation value, outputs the comparison result, and performs routing assembly calculation on image data, infrared data and point cloud data based on the comparison result to output the features to be detected; Input the features to be detected into the visual detection model to perform target detection and output suspected bounding boxes; perform spatial registration on image data, infrared data and point cloud data based on suspected bounding boxes to output a visual alignment matrix; perform background cropping based on the visual alignment matrix to output key image frames; Incremental optimization module: Input key image frames into the deep classification model for prediction and calculation, output confirmation results, and determine intrusion conditions to output early warning information; Based on the early warning information, the confirmation results and key image frames will be subjected to difference verification to output a difficult sample set; Aggregate the output incremental samples for the difficult sample set, and output updated network parameters by taking the derivative of the parameters to update the visual detection model.

2. The machine vision-based transmission line external damage monitoring system according to claim 1, characterized in that, The specific process for generating the scene evaluation value includes: extracting the global pixel brightness values ​​contained in the image data, performing an average value statistical operation to output a global brightness parameter; extracting the global temperature extreme values ​​contained in the infrared data, performing a subtraction operation to output a local temperature difference value; inputting the global brightness parameter into the forward mapping function configured in the environment analysis matrix to perform a direct proportional calculation to output a local clarity coefficient; inputting the local temperature difference value into the reverse mapping function configured in the environment analysis matrix to perform an inverse proportional calculation to output a local stability coefficient; obtaining preset brightness weight parameters and preset temperature weight parameters; performing a product correlation calculation on the local clarity coefficient and preset brightness weight parameters to output an environmental brightness evaluation value; performing a product correlation calculation on the local stability coefficient and preset temperature weight parameters to output an environmental temperature evaluation value; and adding and merging the environmental brightness evaluation value and the environmental temperature evaluation value to output a scene evaluation value.

3. The machine vision-based transmission line external damage monitoring system according to claim 1, characterized in that, The specific generation process of the feature to be detected includes: introducing a benchmark threshold; performing a numerical comparison calculation between the scene evaluation value and the benchmark threshold to output a state comparison result; responding to the condition that the scene evaluation value is lower than the benchmark threshold, inputting image data, infrared data, and point cloud data into a multimodal extractor to perform a stitching and merging operation to output a mixed modality aggregation matrix, inputting the mixed modality aggregation matrix into a channel attention module to perform a weighted summation calculation to output the feature to be detected; responding to the condition that the scene evaluation value is not lower than the benchmark threshold, performing a feature channel mapping operation on the image data to output an independent image feature channel, inputting the independent image feature channel into a visual feature extractor to perform a convolution extraction calculation to output the feature to be detected.

4. The machine vision-based transmission line external damage monitoring system according to claim 1, characterized in that, The specific construction of the visual detection model includes: a deformation space alignment layer: receiving the features to be detected, generating pixel geometric offset vectors based on the features through convolution mapping, and performing adaptive deformation convolution calculation on the features to be detected based on the pixel geometric offset vectors to output a geometric adaptive feature tensor; a cross-scale pyramid fusion layer: receiving the geometric adaptive feature tensor, performing scale downsampling extraction calculation on the geometric adaptive feature tensor to output a multi-scale semantic channel, and performing self-attention weighted association calculation on the multi-scale semantic channel to output a multi-scale feature matrix to be detected; a candidate box regression generation layer: receiving the multi-scale feature matrix to be detected, inputting the multi-scale feature matrix to be detected into a regression mapping network to perform coordinate offset decoding calculation to output a position coordinate matrix; a scale probability classification layer: receiving the multi-scale feature matrix to be detected, inputting the multi-scale feature matrix to be detected into a fully connected classification network to perform nonlinear activation mapping calculation to output a class probability matrix; and a boundary suppression decoding layer: receiving the class probability matrix and the position coordinate matrix, inputting the class probability matrix and the position coordinate matrix into an intersection-union elimination algorithm to perform redundant box filtering calculation to output a suspected bounding box.

5. The machine vision-based transmission line external damage monitoring system according to claim 1, characterized in that, The specific generation process of the key image frame includes: extracting the vertex data of the suspected bounding box in the planar pixel coordinate system; performing coordinate system projection transformation on the vertex data of the planar pixel coordinate system to output infrared projection coordinate data and stereo frustum back-projection coordinate data; performing local cropping and stitching calculation on the image data, infrared data, and point cloud data based on the vertex data of the planar pixel coordinate system, infrared projection coordinate data, and stereo frustum back-projection coordinate data to output a visual alignment matrix; extracting the point cloud distribution data built into the visual alignment matrix; inputting the point cloud distribution data into the ground filtering module to perform background removal calculation to output non-ground target point cloud clusters; extracting depth limit values ​​based on the non-ground target point cloud clusters; generating a depth truncation filtering threshold based on the depth limit values; performing depth masking isolation calculation on the visual alignment matrix based on the depth truncation filtering threshold to output a target foreground feature matrix; and performing image encoding format conversion operation on the target foreground feature matrix to output the key image frame.

6. The machine vision-based transmission line external damage monitoring system according to claim 1, characterized in that, The specific process for generating the warning information includes: inputting key image frames into the feature extraction layer of a deep classification model to perform matrix convolution calculation and output a high-dimensional semantic tensor; inputting the high-dimensional semantic tensor into a fully connected classification layer to perform probability mapping calculation and output a confirmed classification label and a deep recognition confidence value; performing a structured stitching operation on the confirmed classification label and the deep recognition confidence value to output a confirmation result; obtaining a preset intrusion risk judgment threshold value; performing a difference comparison calculation between the deep recognition confidence value and the preset intrusion risk judgment threshold value to output a hazard judgment result; and responding to the condition that the hazard judgment result is that the deep recognition confidence value is greater than the preset intrusion risk judgment threshold value, performing a communication message encapsulation operation on the confirmation result and outputting the warning information.

7. The machine vision-based transmission line external damage monitoring system according to claim 1, characterized in that, The specific process for generating incremental samples includes: extracting the depth recognition confidence value built into the confirmation result; obtaining a preset absolute confidence safety threshold; subtracting the depth recognition confidence value from the preset absolute confidence safety threshold to output a confidence deviation value; obtaining a preset deviation warning threshold; comparing the confidence deviation value with the preset deviation warning threshold to output a deviation comparison result; if the deviation comparison result indicates that the confidence deviation value exceeds the preset deviation warning threshold, transmitting the key image frame to the review interaction interface for screen rendering; receiving the expert calibration bounding box and expert calibration classification label returned by the review interaction interface in response to the manual input action; combining and packaging the key image frame, expert calibration bounding box, and expert calibration classification label to output difficult image samples; obtaining a historically accumulated difficult sample set; and merging and appending the difficult image samples and the historically accumulated difficult sample set to output incremental samples.

8. The machine vision-based transmission line external damage monitoring system according to claim 1, characterized in that, The specific process of updating the visual inspection model includes: extracting the image feature pixel matrix, expert-calibrated bounding boxes, and expert-calibrated classification labels from the incremental samples; obtaining the current running weight parameter set of the visual inspection model; inputting the image feature pixel matrix into the current running weight parameter set to perform forward propagation calculation and outputting the predicted bounding box matrix and the predicted classification probability array; inputting the predicted bounding box matrix and the expert-calibrated bounding boxes into the smoothing absolute error formula to perform comparison calculation and outputting the bounding box regression loss value; inputting the predicted classification probability array and the expert-calibrated classification labels into the cross-entropy formula to perform error estimation calculation and outputting the classification cross-entropy loss value; performing addition and merging calculation on the bounding box regression loss value and the classification cross-entropy loss value to output the joint gradient loss value; performing backpropagation differentiation calculation based on the joint gradient loss value to output the weight gradient update direction data and the weight gradient update step size data; performing numerical superposition and replacement calculation on the current running weight parameter set based on the weight gradient update direction data and the weight gradient update step size data to output the updated network parameters; and performing a model overwrite storage operation based on the updated network parameters to update the visual inspection model and the deep classification model simultaneously.

Images