Cable defect detection method of binocular intelligent inspection robot

Abstract

The invention discloses a cable defect detection method of a binocular intelligent inspection robot, and relates to the field of image processing, and the method comprises the steps: firstly obtaining the state information of a cable, then carrying out the preprocessing and stereo matching of the image information of the cable, and then constructing a target detection and segmentation model; the target detection and segmentation model is adopted to detect and segment cable defects, and model sample data is updated in real time in an incremental learning mode; fusing the cable state information acquired by the infrared camera, the thermal imager and the wireless sensor network by using a multi-sensor data information fusion model; the cable is maintained and managed based on a detection result; according to the invention, the detection precision and accuracy can be improved, cables of different types and specifications can be handled, the adaptability and stability are ensured, and the real-time performance and accuracy of detection can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more particularly to a cable line defect detection method using a binocular intelligent inspection robot. Background Art

[0002] With the acceleration of industrialization and the increasing emphasis on workplace safety, the quality and safety of cables, as a fundamental material, have become a focus of attention across various industries. Cables are widely used in fields such as electricity, communications, transportation, and petrochemicals. In modern society, a wide range of equipment and devices rely on cables for connection and transmission. Therefore, cable quality issues directly impact the normal operation of various industries and the safety of people's lives and property. However, due to the complex operating environments and difficult maintenance requirements of cables, they are prone to various defects such as breakage, short circuits, and insulation degradation. Traditional inspection methods rely primarily on manual visual inspection or simple test instruments, which are subject to inefficiency, low accuracy, and a high risk of missed inspections. These issues not only reduce production efficiency but also can lead to significant economic losses and safety hazards. To address these issues, binocular intelligent inspection robots have emerged, driven by the continuous advancement of computer vision and machine learning technologies.

[0003] The current binocular vision algorithm still has certain errors in the detection of cable breakage and aging, especially when detecting cables of different depths, the detection results may be misjudgments and missed detections; moreover, the current cable defect detection method of a binocular intelligent inspection robot is affected by changes in conditions such as light, temperature, humidity, etc., resulting in a decrease in detection accuracy; the binocular intelligent inspection robot needs to complete a large amount of image acquisition and processing work in a short period of time. The current binocular intelligent inspection robot often needs time to process and analyze the collected images, so the detection results may not be real-time enough and cannot meet the timely response and processing of cable defects, and because the number of cables is large and they are widely distributed.

[0004] Therefore, the present invention discloses a cable line defect detection method of a binocular intelligent inspection robot, which can improve the detection precision and accuracy, can cope with cables of different types and specifications, ensure its adaptability and stability, and can improve the real-time and accuracy of detection. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention discloses a cable line defect detection method using a binocular intelligent inspection robot, which can realize comprehensive monitoring and defect detection of cable line status; a binocular intelligent inspection robot is used to collect cable line status information in real time without human intervention, thereby improving detection efficiency and accuracy; a real-time stream processing engine is used to detect and segment cable line defects, and edge processing equipment is deployed to disperse data processing and analysis operations, thereby improving detection efficiency and response speed; a GPU parallel processing unit is used to preprocess and stereo match binocular image information, thereby improving the accuracy of cable line defect detection; a U-Net image segmentation network structure and a clustered convolutional neural network are used to segment defect areas and extract features from cable line images, thereby realizing automatic identification and accurate segmentation of cable line defects; an incremental learning method is used to update model sample data in real time, and the target detection and segmentation model is iteratively trained, thereby improving the robustness and reliability of detection; a multi-sensor data information fusion model is used to fuse cable line status information collected by infrared cameras, thermal imagers and wireless sensor networks, thereby improving the effectiveness and accuracy of detection; the cable line is maintained and managed based on the detection results, thereby improving the safety and reliability of the cable line; and the degree of automation and intelligence is high.

[0006] The present invention adopts the following technical solutions: A cable defect detection method using a binocular intelligent inspection robot comprises the following steps: Step 1: Obtain cable status information. This information is obtained through a binocular intelligent inspection robot. The binocular intelligent inspection robot uses a binocular camera, an infrared camera, a thermal imager, and a wireless sensor network to collect cable image information, temperature change information, heat distribution information, and physical parameters in real time. The binocular camera is equipped with a dynamic feature matching algorithm, which uses the optical flow method to track the displacement of the cable vibration with an accuracy of ≤0.5mm in real time. The inertial navigation system (INS) is used to construct a millimeter-level precision spatial coordinate mapping model to solve the positioning deviation problem caused by cable deformation in complex environments. Step 2: Preprocessing and stereo matching of cable image information. A GPU parallel processing unit is used to preprocess and stereo match the binocular image information to improve the accuracy and efficiency of cable defect detection. The GPU parallel processing unit includes an image denoising module, an image enhancement module, a geometric correction module, and a stereo matching module. The image denoising module, image enhancement module, and geometric correction module work independently and in parallel. The output ends of the image denoising module, image enhancement module, and geometric correction module are connected to the input end of the stereo matching module. Step 3: Build a target detection and segmentation model. Use a real-time stream processing engine to build a cable defect target detection and segmentation model. The real-time stream processing engine uses a U-Net image segmentation network structure and a clustered convolutional neural network to segment defect areas and extract features from cable images. The real-time stream processing engine automatically identifies and accurately segments cable defects by training the target detection and segmentation model. Step 4: Defect detection and classification: Use the target detection and segmentation model to detect and segment cable defects, and deploy edge processing devices to decentralizedly perform data processing and analysis operations to improve detection efficiency and response speed; Step 5: Model update and optimization: update the model sample data in real time through incremental learning, and iteratively train the target detection and segmentation model to improve the robustness and reliability of detection; Step 6: Data fusion analysis: a multi-sensor data information fusion model is used to fuse the cable status information collected by the infrared camera, thermal imager and wireless sensor network, and a comprehensive analysis is performed on the detection results to improve the effectiveness and accuracy of the detection; Step 7: Maintenance and management of the cables. The binocular intelligent inspection robot maintains and manages the cables based on the detection results.

[0007] As a further technical solution of the present invention, the image denoising module removes different types of noise from the image through a wavelet transform denoising method, the image enhancement module increases the contrast, clarity and details of the image through adaptive histogram equalization, the geometric correction module uses a perspective transformation method to perform geometric transformation on the image, so that the lines and structures of the image are restored to the correct shape and position, and the stereo matching module performs stereo matching on the binocularly collected images through an image matching and aggregation algorithm, and calculates the disparity between images based on the image matching and aggregation results to obtain the depth information of the cable line, so as to reduce misjudgments caused by cable line depth differences and environmental factors. As a further technical solution of the present invention, the working method of the image matching and aggregation algorithm includes the following steps: Step 1: Set the association priority of image feature points. Use the modal adaptive gating unit to statistically train the key value of each feature point in the binocular image. Automatically assign fusion weights for infrared temperature features and visible light texture features to achieve dynamic modal weight balance. Establish a cross-modal association matrix for feature points. Require the mapping error between infrared thermal imaging feature points and visible light feature points in spatial coordinates to be ≤0.3 pixels. Optimize the spatial alignment accuracy of cross-modal features through the graph convolutional network (GCN) module. Scan the support of each feature point in the salient image area of ​​the binocular image, as well as the degree of influence on image matching and aggregation. Use the Pearson correlation coefficient to quantify the degree and direction of association between each feature point and the target task, and assign weights based on the quantified results. Step 2: Extract binocular image feature points through parallel multi-branch residual blocks, use corner detection and scale space extreme value detection algorithms to extract binocular image feature points, and convert the binocular image feature points into feature vector representations. The binocular image feature points include corner points, edge points, spots, scale feature points, acceleration feature points, and direction feature points. Step 3: Search for matching points of the binocular image feature points, calculate the distance or similarity between the binocular image feature points using the nearest neighbor algorithm, perform weighted matching on the distance or similarity, and sort the association priorities according to the distance or similarity of the binocular image feature points; Step 4: Eliminate incorrect matching feature points. Use the RANSAC random sampling consensus algorithm to eliminate incorrect matches. Through continuous iteration, use the correct matching points as new reference points, re-perform feature matching and association operations to find new correct matching points, delete incorrect matches, and adjust matching priorities. Step 5: Disparity calculation: Calculate the disparity value of each pixel using geometric reasoning or optimization algorithms based on the positional relationship of matching point pairs. The disparity value represents the horizontal displacement between corresponding points in the left and right images. Step 6: Disparity filtering: bilateral filtering is used to filter the calculated disparity map to remove noise and outliers and improve the quality of the depth image. Step 7: Output the image matching and aggregation results and the depth information of the cable. As a further technical solution of the present invention, the U-Net image segmentation network structure determines the defect area range by establishing a fault detection PCA and SVM-DS fusion decision model. The pre-processed cable image contains h+1 types of cable states, and the cable state data set The cable is in a defect-free state. to The cable line defect state is represented by n groups of sample data for each cable line defect state. The cable line state data training sample set is is a set of sample images under the i-th cable defect state. The defect area is determined by the following steps: (S1) the image is normalized, and the formula is expressed as: In formula (1), represents the normalization processing result of the sample image under the i-th cable defect state, express The column vector of the mean value of each dimension, represents a column vector; represents an n-dimensional full column vector; Indicates the measurement variance of cable status data; is the measurement variance of the jth image; (S2) calculates the range of the defect area and uses the Q statistic to set the defect threshold. The Q statistic represents the degree of deviation of the cable state measurement value relative to the training model. The calculation formula of the Q statistic is expressed as: In formula (2), Represents the eigenvalue of the covariance matrix of the cable line status measurement value; represents the critical value of the normal distribution at the significance level a; (S3) determines the location of normal and abnormal data, and uses the pairwise coupling method and the one-to-one multi-classification SVM method to realize the conversion from binary classification SVM to multi-classification probability type. The formula is expressed as: In formula (3), Indicates the category number of cable defects; The posterior probability that a cable defect of type k and a cable defect of type j are paired is that the cable defect of type i. The region where the cable defect occurs is determined by formula (3) to facilitate defect detection. As a further technical solution of the present invention, the clustered convolutional neural network includes an input layer, a convolutional layer, a batch normalization layer, a pooling layer, an adaptive topology structure layer, an activation function layer, a fully connected layer, and an output layer. The working method of the clustered convolutional neural network includes the following steps: (1) The image data of the cable defect area is input into the clustering convolutional neural network through the input layer; (2) Perform convolution operation on the input image through the convolution layer sliding filter to generate feature map, and extract deep features through convolution layer stacking; (3) Accelerate the network training process and improve the network convergence through the batch normalization layer, which improves the network's robustness to changes in input data by normalizing each small batch of data; (4) Reducing the spatial size and number of parameters of the feature map through a pooling layer, which uses a maximum pooling operation to select the maximum value in the input region as the output; (5) Dynamically adjust the neural network topology through an adaptive topology layer, which uses a genetic algorithm optimization method to adaptively adjust the structure and topology of the network to adapt to complex tasks and data; (6) Introducing nonlinear transformations through activation function layers to enhance the expressive power of the model; (7) Clustering the feature map through the clustering layer to cluster similar feature points together to improve the robustness and generalization ability of the model; (8) The output of the convolutional layer is mapped to classification or regression results through the fully connected layer, and the results are output through the output layer.

[0008] As a further technical solution of the present invention, the edge processing device uses a hardware-accelerated FPGA chip to implement data processing and analysis, and transmits the results to a central server via a wireless communication network, reducing network transmission time. The edge processing device improves data processing efficiency and response speed through a local cache queue.

[0009] As a further technical solution of the present invention, the incremental learning method updates the newly added data points by adding them to the data set through a time window and obtains the membership vector of the newly added data points. The time window completes the membership matrix parameter update by inputting the newly added data into the target detection and segmentation model to adapt to the newly added data points. The incremental learning method updates the target detection and segmentation model in real time based on the updated membership matrix parameters. As a further technical solution of the present invention, the multi-sensor data information fusion model uses an improved BP neural network algorithm model to predict the error of the data information after multi-sensor fusion, and the formula is expressed as: In formula (4), is the error of the fused data information, is the weight function of the qth layer network in the improved BP neural network algorithm model, x is the threshold value of the qth layer network in the neural network algorithm model, It is the data input node of the qth layer network in the neural network algorithm model. It represents the output node of the qth layer network in the neural network algorithm model, and v represents the step size in the neural network algorithm model. Due to the different feature parameters and dimensions in the input layer, the fused data information is expressed as: In formula (5), Y represents the fused data, Represented as the hidden node of the qth layer network in the neural network algorithm model, is the weighting factor in the neural network algorithm model.

[0010] Positive beneficial effects: The present invention discloses a cable line defect detection method for a binocular intelligent inspection robot, which can realize comprehensive monitoring and defect detection of cable line status; the binocular intelligent inspection robot is used to collect cable line status information in real time without human intervention, thereby improving detection efficiency and accuracy; a real-time stream processing engine is used to detect and segment cable line defects, and edge processing equipment is deployed to disperse and perform data processing and analysis operations, thereby improving detection efficiency and response speed; a GPU parallel processing unit is used to preprocess and stereo match binocular image information, thereby improving the accuracy of cable line defect detection; a U-Net image segmentation network structure and a clustered convolutional neural network are used to segment defect areas and extract features from cable line images, thereby realizing automatic recognition and accurate segmentation of cable line defects; an incremental learning method is used to update model sample data in real time, and target detection and segmentation models are iteratively trained, thereby improving the robustness and reliability of detection; a multi-sensor data information fusion model is used to fuse cable line status information collected by infrared cameras, thermal imagers and wireless sensor networks, thereby improving the effectiveness and accuracy of detection; the cable line is maintained and managed based on the detection results, thereby improving the safety and reliability of the cable line; and the degree of automation and intelligence is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a schematic diagram of the overall process of a cable defect detection method using a binocular intelligent inspection robot according to the present invention; Figure 2 This is a schematic diagram of the architecture of a GPU parallel processing unit in a cable defect detection method of a binocular intelligent inspection robot according to the present invention; Figure 3 This is a flow chart of the image matching and aggregation algorithms in a cable defect detection method using a binocular intelligent inspection robot according to the present invention; Figure 4 A schematic diagram of the workflow of the U-Net image segmentation network structure in a cable defect detection method of a binocular intelligent inspection robot according to the present invention; Figure 5 This is a schematic diagram of the architecture of a clustering convolutional neural network in a cable defect detection method of a binocular intelligent inspection robot in the present invention. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0013] like Figure 1-Figure 5As shown, a cable defect detection method using a binocular intelligent inspection robot includes the following steps: Step 1: Obtain cable status information. This information is obtained through a binocular intelligent inspection robot. The binocular intelligent inspection robot uses a binocular camera, an infrared camera, a thermal imager, and a wireless sensor network to collect cable image information, temperature change information, heat distribution information, and physical parameters in real time. The binocular camera is equipped with a dynamic feature matching algorithm, which uses the optical flow method to track the displacement of the cable vibration with an accuracy of ≤0.5mm in real time. The inertial navigation system (INS) is used to construct a millimeter-level precision spatial coordinate mapping model to solve the positioning deviation problem caused by cable deformation in complex environments. Step 2: Preprocessing and stereo matching of cable image information. A GPU parallel processing unit is used to preprocess and stereo match the binocular image information to improve the accuracy and efficiency of cable defect detection. The GPU parallel processing unit includes an image denoising module, an image enhancement module, a geometric correction module, and a stereo matching module. The image denoising module, image enhancement module, and geometric correction module work independently and in parallel. The output ends of the image denoising module, image enhancement module, and geometric correction module are connected to the input end of the stereo matching module. Step 3: Build a target detection and segmentation model. Use a real-time stream processing engine to build a cable defect target detection and segmentation model. The real-time stream processing engine uses a U-Net image segmentation network structure and a clustered convolutional neural network to segment defect areas and extract features from cable images. The real-time stream processing engine automatically identifies and accurately segments cable defects by training the target detection and segmentation model. Step 4: Defect detection and classification: Use the target detection and segmentation model to detect and segment cable defects, and deploy edge processing devices to decentralizedly perform data processing and analysis operations to improve detection efficiency and response speed; Step 5: Model update and optimization: update the model sample data in real time through incremental learning, and iteratively train the target detection and segmentation model to improve the robustness and reliability of detection; Step 6: Data fusion analysis: a multi-sensor data information fusion model is used to fuse the cable status information collected by the infrared camera, thermal imager and wireless sensor network, and a comprehensive analysis is performed on the detection results to improve the effectiveness and accuracy of the detection; Step 7: Maintenance and management of the cables. The binocular intelligent inspection robot maintains and manages the cables based on the detection results.

[0014] This method builds a full-chain detection system of "multi-source perception, intelligent processing, closed-loop management" through seven core steps. It combines dynamic feature matching, parallel computing, deep learning and other technologies to achieve accurate detection of cable defects under complex working conditions. The specific implementation method is as follows: 1. Accurate Collection of Multi-Source Heterogeneous Data (Step 1) Sensor array collaborative perception Binocular vision positioning: RGB images are collected through a binocular camera, and the optical flow method is used to track the dynamic displacement of cable surface feature points (such as hardware bolts and insulation layer texture). Combined with the inertial navigation system (INS), the robot's position is calculated in real time. A three-dimensional coordinate mapping model with an accuracy of ≤0.5mm is constructed to solve the imaging offset problem caused by cable vibration (amplitude ≤10mm). Multimodal data fusion: The infrared camera (accuracy ±0.3°C) captures areas with abnormal cable temperature, and the thermal imager generates a pseudo-color map of the heat distribution; The wireless sensor network (WSN) collects physical parameters such as vibration acceleration (resolution 0.1m / s²) and humidity (accuracy ±2% RH) to form a multidimensional dataset including vision, thermal and mechanical data. Dynamic scene adaptability In response to cable deformation (such as sag changes), the dynamic feature matching algorithm is used to update the feature point association priority in real time, and the attention mechanism is used to enhance the perception of key areas such as cable joints and insulators to ensure positioning accuracy in complex environments. 2. Parallel image preprocessing and stereo matching (step 2) GPU-accelerated preprocessing pipeline Image denoising module: This module uses bilateral filtering (edge ​​preservation ≥ 90%) combined with non-local means filtering (NLM) to suppress Gaussian and salt-and-pepper noise caused by drone vibration or lens dust during inspections. Image enhancement module: This module uses Retinex theory to enhance contrast in low-light areas and combines histogram equalization to address areas with strong light reflections, addressing uneven lighting during outdoor inspections. Geometric correction module: Uses binocular camera calibration parameters (intrinsic parameter matrix K, extrinsic parameter matrix R / T) to correct image distortion (radial distortion ≤ 0.1%) to ensure geometric consistency for subsequent stereo matching. Efficient stereo matching technology Parallelized Stereo Matching Module: Utilizes the GPU's CUDA core to parallelize the SGM (Semi-Global Matching) algorithm, increasing the matching speed to 200 frames per second (1080p resolution); Cross-modal feature fusion: Infrared thermal imaging features and visible light texture features are input into the improved ResNet network. High-frequency texture (cracks) and thermal gradient (temperature rise) features are extracted through multi-branch residual blocks, and a 2048-dimensional multi-dimensional feature vector is output, improving the matching accuracy of defect areas (≥97%). 3. Real-time defect detection and segmentation (steps 3 and 4) Lightweight deep learning model deployment U-Net image segmentation network: Using an encoder-decoder structure, the encoder uses a clustering convolutional neural network (CNN) to automatically learn the characteristic distribution of cable defects (cracks, discharge ablation, and insulation aging). It supports pixel-level segmentation of five defect types (mIoU ≥ 92%). Real-time stream processing engine: Based on TensorRT accelerated inference, it achieves detection speeds of ≥30 frames per second on edge processing devices (such as NVIDIA JetsonAGX Orin), meeting the real-time detection needs of robots in motion. Distributed computing architecture Edge-cloud collaboration: Edge devices are responsible for real-time image segmentation and preliminary defect location (response time ≤ 100ms), while cloud servers handle historical data training and model optimization, forming a layered architecture of "local rapid detection + cloud-based in-depth analysis" to reduce data transmission delays and cloud load. 4. Dynamic Model Iteration and Multi-Source Fusion (Steps 5 and 6) Model evolution driven by incremental learning Real-time sample updates: Using new defect samples discovered during robot inspections (such as new types of insulator breakage), the training set is updated using a dynamic weight allocation algorithm. Meta-learning is used to quickly adapt to new categories and avoid catastrophic forgetting. Robustness optimization: Introducing adversarial sample training (FGSM algorithm) in iterative training improves the model's ability to resist interference from noise such as sudden changes in lighting and lens contamination, increasing detection robustness by 35%. Multi-sensor fusion decision-making Spatiotemporal consistency constraints: A correlation matrix for visual features, temperature anomaly, and vibration amplitude was constructed, requiring spatiotemporal consistency between the temperature gradient (ΔT ≥ 2°C) and the vibration amplitude (> 5 mm / s) in the defect area. Multi-source evidence was integrated using the DS evidence theory to eliminate single-sensor misjudgments (reducing the false positive rate from 8% to 2.5%). 3D information enhancement: Use the disparity map generated in step 2 to calculate the spatial depth of the defect (accuracy ≤ 1mm). Combined with thermal imaging data, a 3D defect model (such as crack depth and ablation volume) is constructed to provide a quantitative basis for defect severity assessment. 5. Intelligent maintenance management (step 7) Structured output of detection results Generates an inspection report containing the defect location (accuracy ≤ 0.2m), type, and severity (S=0.4D+0.3T+0.2V+0.1C), and uploads it to the power grid GIS system in real time via a wireless communication module (4G / 5G), linking the defect location with the spatial coordinates of the power grid ledger. Automatically trigger an early warning work order for urgent defects (such as discharge ablation), including treatment suggestions (such as power outage maintenance, insulation reinforcement), with a response time of ≤3 minutes. Full life cycle management closed loop Establish a cable defect database to record historical inspection data and maintenance records. Use digital twin technology to simulate defect development trends (such as crack growth rate) to assist in formulating differentiated operation and maintenance strategies (such as frequent inspections of high-risk sections). It supports cross-validation of test results with drone inspection and manual maintenance data, forming a closed-loop management process of "robot initial inspection and manual review status assessment", improving operation and maintenance efficiency by 40%. In the above embodiment, the image denoising module removes different types of noise from the image through a wavelet transform denoising method, the image enhancement module increases the contrast, clarity and details of the image through adaptive histogram equalization, the geometric correction module uses a perspective transformation method to perform geometric transformation on the image so that the lines and structures of the image are restored to the correct shape and position, and the stereo matching module performs stereo matching on the binocularly collected images through image matching and aggregation algorithms, and calculates the disparity between images based on the image matching and aggregation results to obtain the depth information of the cable line, so as to reduce misjudgment caused by cable line depth differences and environmental factors.

[0015] In a specific embodiment, the original image is first decomposed using a wavelet transform to obtain low-frequency and high-frequency signals. The high-frequency signal is then denoised using a soft-threshold wavelet denoising method to remove noise components while preserving the main signal features. The image is then reconstructed using an inverse wavelet transform to obtain a denoised image. The image's grayscale histogram is extracted, and adaptive histogram equalization is performed based on the current brightness distribution. The equalized image is then histogram stretched to appropriately increase the overall brightness and further improve the image contrast. Local contrast enhancement is performed on the image to highlight detailed features and improve image clarity and detail information. A feature point detection algorithm is used to detect key feature points in the image, and the image's rotation angle, scaling factor, and translation vector are calculated based on the positional relationship between adjacent feature points. The image is then transformed using a perspective transformation method based on the calculated geometric transformation parameters to restore the image's lines and structures to their correct shape and position. The transformed image is then resampled and interpolated to obtain a high-quality geometric correction result. By performing epipolar correction on the binocular image, the feature points of the binocular image are mapped to the same horizontal line and the distortion of the image is removed. Then, the left and right images are matched using a matching algorithm based on grayscale similarity and disparity restriction. The matching results are optimized using an aggregation algorithm to obtain a preliminary disparity result. Finally, the preliminary disparity result is post-processed using algorithms such as median filtering and cost smoothing to reduce depth estimation errors and noise interference, thereby obtaining the depth information of the cable.

[0016] In the above embodiment, the working method of the image matching and aggregation algorithm includes the following steps: Step 1: Set the association priority of image feature points. Use the modal adaptive gating unit to statistically train the key value of each feature point in the binocular image. Automatically assign fusion weights for infrared temperature features and visible light texture features to achieve dynamic modal weight balance. Establish a cross-modal association matrix for feature points. Require the mapping error between infrared thermal imaging feature points and visible light feature points in spatial coordinates to be ≤0.3 pixels. Optimize the spatial alignment accuracy of cross-modal features through the graph convolutional network (GCN) module. Scan the support of each feature point in the salient image area of ​​the binocular image, as well as the degree of influence on image matching and aggregation. Use the Pearson correlation coefficient to quantify the degree and direction of association between each feature point and the target task, and assign weights based on the quantified results. In a specific embodiment, a binocular camera is used to acquire visible light images (resolution ≥ 1920 × 1080), and an infrared thermal imager is used to simultaneously acquire thermal imaging data of the same viewing angle (resolution ≥ 320 × 240), ensuring that the timestamp synchronization error between the two is ≤ 10ms.

[0017] Grayscale stretching is performed on the infrared image to map the temperature range (-20℃~80℃) to the grayscale interval of [0,255] to facilitate subsequent feature extraction.

[0018] Zhang’s calibration method is used to obtain the intrinsic parameters of the binocular camera (focal length, distortion coefficient) and the extrinsic parameters of the infrared camera and binocular camera (rotation matrix R, translation vector T), and establish the spatial transformation relationship of the multimodal image.

[0019] Epipolar correction is used to convert the binocular image and infrared image into epipolar alignment form, ensuring that the corresponding feature points are located on the same horizontal scan line, thus reducing the subsequent matching search range.

[0020] Step 2: Extract binocular image feature points through parallel multi-branch residual blocks, use corner detection and scale space extreme value detection algorithms to extract binocular image feature points, and convert the binocular image feature points into feature vector representations. The binocular image feature points include corner points, edge points, spots, scale feature points, acceleration feature points, and direction feature points. The SIFT algorithm is used to extract scale-invariant feature points (corner points, spots), and the 128-dimensional histogram of oriented gradients (HOG) feature vector is calculated to characterize texture details.

[0021] Infrared thermal imaging: The FAST corner detection algorithm is used to extract areas of significant temperature gradients (such as temperature rise points at cable joints). Combined with the Local Binary Pattern (LBP) algorithm, an 80-dimensional thermal distribution feature vector is generated to highlight areas of abnormal temperature. The top 30% of feature points are retained, and low-contrast noise points are filtered out to ensure that 500-1000 valid feature points are retained in both the visible light and infrared images.

[0022] The eigenvectors are Z-score normalized to eliminate the dimensional differences between different modal data (e.g., the scales of temperature values ​​and pixel grayscale values ​​are unified).

[0023] Step 3: Search for matching points of the binocular image feature points, calculate the distance or similarity between the binocular image feature points using the nearest neighbor algorithm, perform weighted matching on the distance or similarity, and sort the association priorities according to the distance or similarity of the binocular image feature points; Step 4: Eliminate incorrect matching feature points. Use the RANSAC random sampling consensus algorithm to eliminate incorrect matches. Through continuous iteration, use the correct matching points as new reference points, re-perform feature matching and association operations to find new correct matching points, delete incorrect matches, and adjust matching priorities. Step 5: Disparity calculation: Calculate the disparity value of each pixel using geometric reasoning or optimization algorithms based on the positional relationship of matching point pairs. The disparity value represents the horizontal displacement between corresponding points in the left and right images. Step 6: Disparity filtering: bilateral filtering is used to filter the calculated disparity map to remove noise and outliers and improve the quality of the depth image. Step 7: Output the image matching and aggregation results and the depth information of the cable.

[0024] In specific embodiments, image matching and aggregation algorithms are crucial steps in stereo matching, primarily matching image features and calculating depth information within binocular vision. These techniques can address misjudgments caused by differences in cable depth, thereby improving defect detection accuracy. Image matching algorithms primarily refer to the process of matching similar regions within binocular views. Image matching can be performed using feature point selection and matching, or using block matching algorithms, such as those based on convolutional neural networks, region-based matching, or image segmentation. These matching algorithms primarily involve calculating similarity, finding optimal matching points, filtering out mismatched points, and optimizing the matching. Aggregation algorithms primarily combine the matching results to generate the final depth information. Common aggregation algorithms include smoothing filtering, weighted filtering, and graph-cut-based algorithms. Smoothing filtering primarily eliminates noise interference and converts discrete depth information into a smooth depth distribution. Weighted filtering assigns different weights to points of varying reliability, comprehensively considering the contributions of different points and improving the accuracy of depth information. Graph-cut-based algorithms employ the minimum cut principle from graph theory to segment the graph according to an energy function to produce the final depth map. In summary, image matching and aggregation algorithms aim to match similar areas in the binocular view and extract depth information to reduce misjudgments caused by differences in cable depth. Common algorithms include feature point matching, block matching, weighted filtering, and graph-cut-based algorithms. Furthermore, the use of GPU parallel processing units can accelerate image matching and aggregation algorithms, improving their efficiency and accuracy.

[0025] In a specific embodiment, during the working process of the image matching and aggregation algorithm, the calculation of data information can be achieved by setting up hardware conditions. In a specific embodiment, for example, the hardware conditions for the working of the image matching and aggregation algorithm mainly include the following aspects: Processor: The processor is the core hardware behind the image matching and aggregation algorithms, responsible for performing various computational tasks. The processor requires high computing performance to quickly process large amounts of image data. Typically, processors can be CPUs, GPUs (graphics processing units), or FPGAs (field-programmable gate arrays).

[0026] Memory: Image matching and aggregation algorithms require a large amount of storage space to store image data and intermediate computation results. Therefore, memory conditions are an important hardware requirement for the algorithm to work. Memory can be selected from devices such as RAM (random access memory) or ROM (read-only memory).

[0027] Storage devices: To store large amounts of image data and computational results, appropriate storage devices are required. Common storage devices include hard drives, solid-state drives, and optical disks. Storage devices should have high storage capacity and high data transfer speeds.

[0028] Display device: For image processing tasks, a display device is essential. The display device can be a computer monitor, projector, etc., which is used to display the calculation results of image matching and aggregation algorithms in real time.

[0029] Communication interface: If you need to achieve image data transmission and collaborative processing between multiple devices, a communication interface is essential. The communication interface can be a wired communication interface (such as Ethernet port, serial port, etc.) or a wireless communication interface (such as Wi-Fi, Bluetooth, etc.).

[0030] Power supply: The hardware devices required for image matching and aggregation algorithms require a stable power supply. The power supply can be AC ​​or DC, depending on the device requirements.

[0031] In summary, the hardware conditions for the image matching and aggregation algorithm include processors, memory, storage devices, display devices, communication interfaces and power supplies. These hardware conditions jointly determine the processing power, efficiency and stability of the algorithm. In practical applications, it is necessary to select appropriate hardware devices according to specific needs. Through the above hardware construction, the application and practical operation capabilities of the algorithm can be improved. In the above embodiment, the U-Net image segmentation network structure determines the defect area range by establishing a fault detection PCA and SVM-DS fusion decision model. The pre-processed cable line image contains h+1 types of cable line states, and the cable line state data set The cable is in a defect-free state. to The cable line defect state is represented by n groups of sample data for each cable line defect state. The cable line state data training sample set is is the sample image set under the i-th cable line defect state; the hardware conditions for cable defect detection based on the U-Net and PCA-SVM-DS fusion model 1. Hardware Configuration during Training 1. High-Performance Computing Server CPU: Intel Xeon Platinum 8380 (28 cores / 56 threads, 2.3GHz+), supporting parallel data preprocessing (such as PCA dimensionality reduction and sample labeling), meeting the feature computation requirements of large-scale data (tens of thousands of samples). GPU: NVIDIA A100 PCIe 40GB x 4 (or equivalent GPU), supporting CUDA parallel acceleration for U-Net network training, with a single card delivering 19.5 TFLOPS FP32 computing power, enabling batch training of 256×256 pixel images (batch size ≥ 64). Memory: 256GB DDR4 ECC memory, supporting simultaneous loading of multimodal training data (visible / infrared images, label masks, feature vectors), reducing frequent disk I / O. Storage: System disk: 2TB NVMe SSD (read speed ≥5000MB / s) for installing deep learning frameworks (PyTorch / TensorFlow) and training toolchains; Data disk: 10TB × 2 SATA hard drives (RAID 10) for storing cable status training datasets (single sample includes RGB / infrared images, 3D point clouds, etc., with an average size of 5MB and support for storage of more than 100,000 samples).

[0032] 2. Dedicated data preprocessing equipment: High-speed data acquisition card: Supports GigE Vision interface, real-time synchronous acquisition of binocular camera (1920×1080 @ 30fps) and infrared thermal imager (320×240 @ 50Hz) data, ensuring timestamp error ≤ 10μs. Edge computing preprocessing unit: Such as NVIDIA Jetson Xavier NX, performs image denoising, geometric correction and other preprocessing before data upload, reducing server computing load (preprocessing speed ≥ 200 frames / second).

[0033] 2. Hardware Configuration for Inference 1. Embedded edge processing device main control chip: NVIDIA Jetson AGX Orin (200 TOPS INT8 computing power), supports U-Net model quantization deployment (FP32 to INT8 conversion accuracy loss ≤ 1.5%), and achieves real-time segmentation of 256×256 images (latency ≤ 30ms). Coprocessor: NPU (Neural Processing Unit): supports PCA matrix operation acceleration (eigenvalue decomposition speed ≥ 100 times / second), pre-processing 1024-dimensional feature vectors to 50 dimensions; DSP (Digital Signal Processor): hardware-accelerates evidence synthesis calculations in SVM-DS fusion decision-making (such as Dempster combination rules, single-frame processing time ≤ 10ms). Memory and storage: 16GB LPDDR4x memory supports simultaneous U-Net inference (occupies 8GB) and multi-sensor data caching (occupies 4GB); 128GB eMMC storage is used to store trained model parameters (U-Net weight files approximately 200MB, PCA-SVM-DS parameters approximately 50MB) and the detection rule library. 2. Multimodal Sensor Hardware: Binocular Camera: Resolution: 2 megapixels (1920×1080), autofocus (depth of field 5-50m), and global shutter to reduce motion blur; Interface: USB 3.1 Gen2 (transmission rate 10Gbps), supporting simultaneous binocular image acquisition (time difference ≤ 1μs). Infrared Thermal Imager: Resolution: 384×288 (thermal sensitivity ≤ 50mK), temperature measurement range -20°C to 200°C (accuracy ±2% or ±2°C); Output: Digital thermal imaging data (14-bit grayscale) + pseudo-color image, transmitted in real time to edge devices via HDMI 2.0. Inertial Navigation System (INS): Accuracy: Position error ≤ 0.5m, attitude angle error ≤ 0.1°, providing coordinate system calibration data for PCA feature space transformation. 3. Communication and Peripheral Interfaces: Wired interfaces: Gigabit Ethernet (RJ45) + USB-C (supporting 10Gbps data backhaul) for offline training data import and inspection result export; Wireless modules: 5G NR module (Sub-6GHz, peak rate 1.2Gbps), for real-time upload of defect detection results (single frame data size ≤ 10KB, latency ≤ 50ms); Wi-Fi 6 (802.11ax), supporting local debugging and OTA model updates (5GHz band, transmission rate ≥ 800Mbps). III. Auxiliary Hardware Systems: 1. Power and Cooling: Inspection robot: 24V lithium battery pack (10Ah capacity) + solar charging panel (10W, supporting simultaneous charging), power consumption ≤ 20W for edge devices (battery life ≥ 4 hours under typical operating conditions); Server: Redundant power supplies (2+1 hot backup), supporting 24 / 7 uninterrupted training (power consumption ≥ 5kW).Cooling: Embedded devices use passive heat sinks (area ≥ 500 cm²) + low-noise fans (automatically adjustable speed, noise ≤ 35 dB), adapting to ambient temperatures of -20°C to 60°C. Servers use a liquid cooling system (independent CPU / GPU water cooling circuits) to ensure core temperatures ≤ 75°C during extended high-load operation. 2. Storage and Backup: Local storage: Edge devices have a built-in 128GB SSD to cache the last 1000 frames of inspection data (including original images, defect masks, and decision evidence), supporting resumable data transfer even after network disconnection. Cloud storage uses a distributed file system (such as HDFS) to store historical training data and inspection logs. A single cluster supports petabyte-level data expansion, with a three-replica backup strategy. Matrix operations in PCA-SVM-DS fusion decision-making (such as covariance matrix calculation and SVM kernel function) are accelerated by DSP / NPU hardware, achieving 5-10 times higher efficiency than pure CPU solutions. 2. Inspection scenario adaptation: The binocular and infrared sensors deliver high resolution and high frame rate output, relying on high-speed interfaces (USB 3.1 / HDMI 2.0) to ensure zero frame loss. The embedded device's low power consumption (≤20W) and wide operating temperature range (-20°C to 60°C) meet the field operation requirements of the transmission line inspection robot. Defect areas are identified through the following steps: (S1) The image is normalized and the formula is expressed as: In formula (1), represents the normalization processing result of the sample image under the i-th cable defect state, express The column vector of the mean value of each dimension, represents a column vector; represents an n-dimensional full column vector; Indicates the measurement variance of cable status data; is the measurement variance of the jth image; (S2) calculates the range of the defect area and uses the Q statistic to set the defect threshold. The Q statistic represents the degree of deviation of the cable state measurement value relative to the training model. The calculation formula of the Q statistic is expressed as: In formula (2), Represents the eigenvalue of the covariance matrix of the cable line status measurement value; represents the critical value of the normal distribution at the significance level a; (S3) determines the location of normal and abnormal data, and uses the pairwise coupling method and the one-to-one multi-classification SVM method to realize the conversion from binary classification SVM to multi-classification probability type. The formula is expressed as: In formula (3), Indicates the category number of cable defects; The posterior probability that the k-th cable defect and the j-th cable defect belong to the i-th cable defect when they are paired is expressed. The area where the cable defect occurs is determined by formula (3) to facilitate defect detection. In a specific embodiment, the U-Net image segmentation network structure is a deep learning model with good image segmentation effect. The model adopts an encoder-decoder structure, in which the input image is extracted through multi-layer convolution operations in the encoder, and the feature image is restored to its original size through deconvolution operations in the decoder. At the same time, U-Net also adopts skip connection technology, in which the feature map of the corresponding layer in the encoder is connected with the feature map of the corresponding layer in the decoder, so that the model can better retain low-level and high-level feature information, further improving the segmentation accuracy. The working hardware environment of the U-Net image segmentation network structure can include the following: GPU (Graphics Processing Unit): The U-Net network structure usually requires a lot of computing and data processing. Using a GPU with powerful parallel computing capabilities can accelerate the network training and inference process.

[0034] CPU (Central Processing Unit): The CPU is mainly used to control functions such as task scheduling, network management, and data transmission. It can work together with the GPU to improve overall performance.

[0035] Storage devices: The U-Net network needs to store a large amount of network parameters and training sample data. During training and inference, data needs to be read and written quickly. Therefore, high-speed storage devices such as SSDs (solid-state drives) or NVMe (non-volatile memory express) are required.

[0036] Memory: To support large-scale image segmentation tasks, a sufficiently large memory capacity is required to store the network's intermediate feature maps and gradient information, as well as input and output data.

[0037] Network connection: To achieve real-time stream processing, a high-bandwidth and low-latency network connection is required to transmit image data from the binocular intelligent inspection robot to the real-time stream processing engine and return the results to the robot for further operation.

[0038] In a further specific embodiment, U-Net is a deep learning network architecture widely used in image segmentation tasks, proposed by the UCL Medical Image Computing Group. Its main features are the encoder-decoder structure and skip connections.

[0039] The main components of U-Net include: Encoder: The encoder extracts multi-scale feature representations from the input image. It includes multiple convolutional and pooling layers to reduce spatial resolution and increase feature dimensionality. In the final encoder layer, the feature maps are compressed to a smaller size to reduce computational effort.

[0040] Decoder: The decoder is responsible for recovering the original image size from the encoder's feature representation. It includes multiple convolutional layers and upsampling layers to increase spatial resolution and restore feature maps. The decoder input is the encoder output and low-level features generated by skip connections.

[0041] Skip connection: Skip connection fuses the output of the encoder with the input of the decoder to utilize low-level feature information in the decoder. This connection helps preserve the details in the image, thereby improving the accuracy of the segmentation results.

[0042] The U-Net network architecture has achieved excellent performance in many medical image segmentation tasks, such as tumor detection and organ segmentation. Its main advantage is its ability to leverage multi-scale feature information for accurate segmentation while preserving image details through skip connections. This model architecture improves the algorithm's computational power.

[0043] The algorithm's effectiveness was analyzed by establishing a cable transmission simulation environment, and the actual effectiveness of the cable defect simulation device was analyzed through experimental results. The laboratory configuration used a Core i9 64-bit computer with 128GB of RAM, and the simulation environment was established using power cable simulation software. The on-site experimental environment was set up with a medium-voltage cable transmission voltage level. The simulation data accuracy was 95%, and the algorithm's operational error did not exceed 2.5%. A combination of wireless and proprietary networks was used. The experimental parameters are shown in Table 2: This experiment completed data transmission under China Unicom's 5G wireless communication and RS232 proprietary network, and used a Windows x86 computer for data input and algorithm programming. The collected simulated defect data was used as experimental samples, and power transmission was completed according to the erected TRF cable line. The experiment was divided into two simulation cases: Case A: The U-Net image segmentation network structure was used for image segmentation before defect feature extraction and detection; Case B: The SegNet convolutional segmentation network structure was used for image segmentation before defect feature extraction and detection; Case C: Defect feature extraction and detection were performed directly. The total amount of image data detected within 20 minutes in Cases A, B, and C, as well as the speed and monitoring accuracy of detecting 18100KB of image data, were compared, and the records are shown in Table 3. The comparison shows that the total amount of data processed, the speed and the accuracy of case A are much greater than those of cases B and C, which proves the practicability and effectiveness of this algorithm.

[0044] In the above embodiment, the clustering convolutional neural network includes an input layer, a convolution layer, a batch normalization layer, a pooling layer, an adaptive topology structure layer, an activation function layer, a clustering layer, a fully connected layer and an output layer. The working method of the clustering convolutional neural network includes the following steps: (1) The image data of the cable defect area is input into the clustering convolutional neural network through the input layer; (2) Perform convolution operation on the input image through the convolution layer sliding filter to generate feature map, and extract deep features through convolution layer stacking; (3) Accelerate the network training process and improve the network convergence through the batch normalization layer, which improves the network's robustness to changes in input data by normalizing each small batch of data; (4) Reducing the spatial size and number of parameters of the feature map through a pooling layer, which uses a maximum pooling operation to select the maximum value in the input region as the output; (5) Dynamically adjust the neural network topology through an adaptive topology layer, which uses a genetic algorithm optimization method to adaptively adjust the structure and topology of the network to adapt to complex tasks and data; (6) Introducing nonlinear transformations through activation function layers to enhance the expressive power of the model; (7) Clustering the feature map through the clustering layer to cluster similar feature points together to improve the robustness and generalization ability of the model; (8) The output of the convolutional layer is mapped to classification or regression results through the fully connected layer, and the results are output through the output layer.

[0045] In a specific embodiment, the main purpose of the clustering convolutional neural network for cable line images is to perform feature extraction and defect area segmentation. Specifically, the clustering convolutional neural network used in the real-time stream processing engine can be a model that combines cluster analysis with a convolutional neural network. The clustering convolutional neural network first groups the cable line images through cluster analysis, clustering pixels with similar features together to form different clusters. The convolutional neural network then uses these clusters as input to extract the feature representation of each cluster. In this way, the network can learn better image features and more accurately segment the defect area of ​​the cable line. The clustering convolutional neural network generally consists of two main parts: a clustering layer and a convolutional layer. The clustering layer is responsible for clustering the image and dividing the pixels into different clusters. The convolutional layer is responsible for extracting feature representations from each cluster for subsequent defect area segmentation. The network model structure parameters are shown in Table 4. In the above embodiment, the edge processing device uses a hardware-accelerated FPGA chip to implement data processing and analysis, and transmits the results to the central server through a wireless communication network, reducing network transmission time. The edge processing device improves data processing efficiency and response speed through a local cache queue.

[0046] In specific embodiments, deploying edge processing devices to decentralizedly perform data processing and analysis operations can significantly improve detection efficiency and response speed. The technology essentially shifts data processing and analysis operations from central servers to edge devices, enabling more efficient data processing and analysis through faster response times and higher computing efficiency.

[0047] Specifically, collected data is pre-processed on edge devices, such as image cropping, resizing, and noise reduction. These operations reduce data transmission across the network and alleviate the burden on central servers. Pre-trained models are deployed on edge devices, leveraging hardware acceleration technologies (such as GPUs and FPGAs) for fast and efficient data processing and analysis. When model inference is performed on edge devices, the results can be directly returned to the central server, reducing network transmission time. Because collected data may contain sensitive information, edge device deployment requires a series of measures to ensure data security and prevent misuse or tampering. These include data encryption, access control, and authentication. Deploying edge devices enables real-time detection and analysis. Local caching and queuing on edge devices can achieve faster data processing and response times. This not only improves the user experience but also prevents serious consequences in specific application scenarios.

[0048] By deploying edge processing devices to decentralizedly perform data processing and analysis, detection efficiency and response speed can be significantly improved while reducing the load on central servers. This has extremely high value and practicality in practical applications.

[0049] In the above embodiment, the incremental learning method updates the newly added data points by adding them to the data set through a time window, and obtains the membership vector of the newly added data points. The time window completes the membership matrix parameter update by inputting the newly added data into the target detection and segmentation model to adapt to the newly added data points. The incremental learning method updates and iterates the target detection and segmentation model in real time according to the membership matrix parameter update.

[0050] In a specific embodiment, incremental learning is used to update the sample data of the object detection and segmentation model in real time, enabling dynamic learning and optimization of the model, thereby improving the robustness and reliability of detection. The basic idea of ​​incremental learning is to use new sample data for training based on the current model's performance, and then readjust the model's parameters to make it more consistent with the actual situation, thereby improving performance.

[0051] Specifically, the incremental learning method collects new sample data, including positive samples and negative samples, according to actual conditions. These data can come from real-time collected image data, or previous error detection results and manually labeled data. Use new sample data for training to update the parameters of the model. During the incremental training process, different algorithms can be used, such as online learning, accelerated online learning, incremental learning, etc. These algorithms can be flexibly adopted according to different model structures and specific scenarios. After the incremental training is completed, the model is evaluated to see whether its performance has been improved. Some indicators such as accuracy, recall rate, F1 value, etc. can be used for evaluation. If the model performance has been improved, the new model can be put into use. In order to ensure that the model is always updated, this process can be carried out in a continuous cycle, and incremental training can be performed each time new sample data is obtained to optimize the performance of the model. In the above embodiment, the multi-sensor data information fusion model uses an improved BP neural network algorithm model to predict the error of the data information after multi-sensor fusion, and the formula is expressed as: In formula (4), is the error of the fused data information, is the weight function of the qth layer network in the improved BP neural network algorithm model, x is the threshold value of the qth layer network in the neural network algorithm model, It is the data input node of the qth layer network in the neural network algorithm model. It represents the output node of the qth layer network in the neural network algorithm model, and v represents the step size in the neural network algorithm model. Due to the different feature parameters and dimensions in the input layer, the fused data information is expressed as: In formula (5), Y represents the fused data, Represented as the hidden node of the qth layer network in the neural network algorithm model, is a weighting factor in the neural network algorithm model. In a specific embodiment, the multi-sensor data information fusion model is a technology that uses data collected by multiple sensors and comprehensively analyzes them through appropriate algorithms and methods to improve detection effectiveness and accuracy. In cable line status monitoring, multiple sensors such as infrared cameras, thermal imagers, and wireless sensor networks can monitor different aspects of the cable, such as temperature, humidity, vibration, etc., and obtain a large amount of data. The working hardware environment of the multi-sensor data information fusion model may include the following: Infrared camera: used to obtain infrared images of cables and detect hot areas and abnormal temperatures of cables.

[0052] Thermal imagers: Used to thermally image cables, obtain temperature distribution, and detect overheating or abnormal temperatures. Wireless sensor networks: Deployed around cables, they collect information on physical quantities such as vibration, current, and voltage, as well as environmental parameters such as humidity and temperature.

[0053] Data acquisition equipment: responsible for collecting and storing the raw data received from infrared cameras, thermal imagers and wireless sensor networks.

[0054] Data processing unit: used to preprocess, extract features and fuse data from different sensors, and integrate multi-sensor data into a comprehensive status information.

[0055] Storage devices: used to store raw data and processed data, such as databases and cloud storage.

[0056] Multi-core CPU / GPU: used for data processing, algorithm calculations, and model reasoning, providing sufficient computing resource support.

[0057] Memory: used to store data to be processed, model parameters, and intermediate results, providing fast reading and writing capabilities.

[0058] Network connection: used to transmit data and model parameters, which can be wired or wireless network to ensure the real-time and stability of sensor data and models.

[0059] In summary, the hardware environment for multi-sensor data fusion models primarily includes infrared cameras, thermal imagers, wireless sensor networks, data acquisition equipment, data processing units, storage devices, multi-core CPUs / GPUs, memory, and network connections. The configuration and performance of these hardware resources directly impact the efficiency and accuracy of data acquisition, processing, and fusion. The operating system of the system in the experimental environment is WINDOWS system, and the compilation tool JDK is installed on the experimental computer. According to the actual application environment, two servers are arranged in the test environment for the system database and deployment test system respectively. One experimental computer is used to test the various business processing functions of the system. Figure 5 The configuration parameters of the equipment in the test environment are shown in Table 5. To better compare model accuracy and mitigate the impact of different datasets on model accuracy, we randomly permuted the sample datasets and conducted comparative tests using the RF model and the FCN model. The training set contained 5,000 data samples from different sensors, and the training time was set to 10 minutes.

[0060] When conducting data fusion experiments, the test set is divided into 5 groups. The fusion accuracy of the model under different data scale training conditions is shown in Table 6. Comparative test experimental results show that the research system model has the highest data fusion accuracy under different data amounts, and can avoid the impact of data randomness on detection accuracy. As the amount of training data increases, the monitoring accuracy growth rate is the fastest among all models.

[0061] Although specific embodiments of the present invention have been described above, those skilled in the art will appreciate that these specific embodiments are merely illustrative, and that those skilled in the art may omit, substitute, and modify the details of the methods and systems described above without departing from the principles and spirit of the present invention. For example, combining the above method steps to perform substantially the same functions in substantially the same manner to achieve substantially the same results falls within the scope of the present invention. Accordingly, the scope of the present invention is limited solely by the appended claims.

Claims

1. A cable defect detection method using a binocular intelligent inspection robot, characterized by: The following steps are involved: Step 1: Get cable status information Cable status information is acquired through a binocular intelligent inspection robot, which uses a binocular camera, infrared camera, thermal imager, and wireless sensor network to collect cable image information, temperature change information, heat distribution information, and physical parameters in real time. The binocular camera is equipped with a dynamic feature matching algorithm, which uses the optical flow method to track the displacement of cable vibration in real time with an accuracy of ≤0.5mm. The inertial navigation system (INS) constructs a spatial coordinate mapping model with millimeter-level accuracy to solve the positioning deviation problem caused by cable deformation in complex environments. Step 2: Cable image information preprocessing and stereo matching A GPU parallel processing unit is used to preprocess and stereo match binocular image information to improve the accuracy and efficiency of cable defect detection. The GPU parallel processing unit includes an image denoising module, an image enhancement module, a geometric correction module, and a stereo matching module. The image denoising module, image enhancement module, and geometric correction module operate independently and in parallel. The output ends of the image denoising module, image enhancement module, and geometric correction module are connected to the input end of the stereo matching module. Step 3: Build a target detection and segmentation model. Use a real-time stream processing engine to build a cable defect target detection and segmentation model. The real-time stream processing engine uses a U-Net image segmentation network structure and a clustered convolutional neural network to segment defect areas and extract features from cable images. The real-time stream processing engine automatically identifies and accurately segments cable defects by training the target detection and segmentation model. Step 4: Defect detection and classification: Use the target detection and segmentation model to detect and segment cable defects, and deploy edge processing devices to decentralizedly perform data processing and analysis operations to improve detection efficiency and response speed; Step 5: Model update and optimization: update the model sample data in real time through incremental learning, and iteratively train the target detection and segmentation model to improve the robustness and reliability of detection; Step 6: Data fusion analysis: a multi-sensor data information fusion model is used to fuse the cable status information collected by the infrared camera, thermal imager and wireless sensor network, and a comprehensive analysis is performed on the detection results to improve the effectiveness and accuracy of the detection; Step 7: Maintenance and management of the cables. The binocular intelligent inspection robot maintains and manages the cables based on the detection results.

2. The cable defect detection method using a binocular intelligent inspection robot according to claim 1, characterized in that: The image denoising module removes different types of noise from the image using a wavelet transform denoising method. The image enhancement module increases the contrast, clarity and details of the image through adaptive histogram equalization. The geometric correction module uses a perspective transformation method to perform geometric transformation on the image, so that the lines and structures of the image are restored to the correct shape and position. The stereo matching module performs stereo matching on the binocularly collected images through image matching and aggregation algorithms, and calculates the disparity between images based on the image matching and aggregation results to obtain the depth information of the cable line, so as to reduce misjudgments caused by cable line depth differences and environmental factors.

3. The cable defect detection method using a binocular intelligent inspection robot according to claim 2, characterized in that: The image matching and aggregation algorithm uses an improved ResNet network to extract cable surface texture features and thermal distribution features, and outputs a multi-dimensional feature vector; The image matching and aggregation algorithm works in the following steps: Step 1: Set the association priority of image feature points. Use the modal adaptive gating unit to statistically train the key value of each feature point in the binocular image. Automatically assign fusion weights for infrared temperature features and visible light texture features to achieve dynamic modal weight balance. Establish a cross-modal association matrix for feature points. Require the mapping error between infrared thermal imaging feature points and visible light feature points in spatial coordinates to be ≤0.3 pixels. Optimize the spatial alignment accuracy of cross-modal features through the graph convolutional network (GCN) module. Scan the support of each feature point in the salient image area of ​​the binocular image, as well as the degree of influence on image matching and aggregation. Use the Pearson correlation coefficient to quantify the degree and direction of association between each feature point and the target task, and assign weights based on the quantified results. Step 2: Extract binocular image feature points through parallel multi-branch residual blocks, use corner detection and scale space extreme value detection algorithms to extract binocular image feature points, and convert the binocular image feature points into feature vector representations. The binocular image feature points include corner points, edge points, spots, scale feature points, acceleration feature points, and direction feature points. Step 3: Search for matching points of the binocular image feature points, calculate the distance or similarity between the binocular image feature points using the nearest neighbor algorithm, perform weighted matching on the distance or similarity, and sort the association priorities according to the distance or similarity of the binocular image feature points; Step 4: Eliminate incorrect matching feature points. Use the RANSAC random sampling consensus algorithm to eliminate incorrect matches. Through continuous iteration, use the correct matching points as new reference points, re-perform feature matching and association operations to find new correct matching points, delete incorrect matches, and adjust matching priorities. Step 5: Disparity calculation: Calculate the disparity value of each pixel using geometric reasoning or optimization algorithms based on the positional relationship of matching point pairs. The disparity value represents the horizontal displacement between corresponding points in the left and right images. Step 6: Disparity filtering: bilateral filtering is used to filter the calculated disparity map to remove noise and outliers and improve the quality of the depth image. Step 7: Output the image matching and aggregation results and the depth information of the cable.

4. The cable defect detection method using a binocular intelligent inspection robot according to claim 1, characterized in that: The U-Net image segmentation network structure determines the defect area range by establishing a fault detection PCA and SVM-DS fusion decision model. The pre-processed cable line image contains h+1 types of cable line states. The cable line state data set The cable is in a defect-free state. to The cable line defect state is represented by n groups of sample data for each cable line defect state. The cable line state data training sample set is is a set of sample images under the i-th cable defect state. The defect area is determined by the following steps: (S1) the image is normalized, and the formula is expressed as: In formula (1), represents the normalization processing result of the sample image under the i-th cable defect state, express The column vector of the mean value of each dimension, represents a column vector; represents an n-dimensional full column vector; Indicates the measurement variance of cable status data; is the measurement variance of the jth image; (S2) calculates the range of the defect area and uses the Q statistic to set the defect threshold. The Q statistic represents the degree of deviation of the cable state measurement value relative to the training model. The calculation formula of the Q statistic is expressed as: In formula (2), Represents the eigenvalue of the covariance matrix of the cable line status measurement value; represents the critical value of the normal distribution at the significance level a; (S3) determines the location of normal and abnormal data, and uses the pairwise coupling method and the one-to-one multi-classification SVM method to realize the conversion from binary classification SVM to multi-classification probability type. The formula is expressed as: In formula (3), Indicates the category number of cable defects; It represents the posterior probability that the k-th cable defect belongs to the i-th cable defect when it is paired with the j-th cable defect. The area where the cable defect occurs is determined by formula (3) to facilitate defect search.

5. The cable defect detection method using a binocular intelligent inspection robot according to claim 1, characterized in that: The clustered convolutional neural network includes an input layer, a convolution layer, a batch normalization layer, a pooling layer, an adaptive topology structure layer, an activation function layer, a fully connected layer and an output layer. The working method of the clustered convolutional neural network includes the following steps: (1) The image data of the cable defect area is input into the clustering convolutional neural network through the input layer; (2) Perform convolution operation on the input image through the convolution layer sliding filter to generate feature map, and extract deep features through convolution layer stacking; (3) Accelerate the network training process and improve the convergence of the network through the batch normalization layer. The batch normalization layer improves the robustness of the network to changes in the input data by normalizing each small batch of data; (4) Reducing the spatial size and number of parameters of the feature map through a pooling layer, which uses a maximum pooling operation to select the maximum value in the input region as the output; (5) Dynamically adjust the neural network topology through an adaptive topology layer, which uses a genetic algorithm optimization method to adaptively adjust the structure and topology of the network to adapt to complex tasks and data; (6) Introducing nonlinear transformations through activation function layers to enhance the expressive power of the model; (7) Clustering the feature map through the clustering layer to group similar feature points together to improve the robustness and generalization ability of the model; (8) The output of the convolutional layer is mapped to classification or regression results through the fully connected layer, and the results are output through the output layer.

6. The cable defect detection method using a binocular intelligent inspection robot according to claim 1, characterized in that: The edge processing device uses a hardware-accelerated FPGA chip to implement data processing and analysis, and transmits the results to a central server via a wireless communication network, reducing network transmission time. The edge processing device improves data processing efficiency and response speed through a local cache queue.

7. The cable defect detection method using a binocular intelligent inspection robot according to claim 1, characterized in that: The incremental learning method updates the newly added data points by adding them to the data set through a time window, and obtains the membership vector of the newly added data points. The time window completes the membership matrix parameter update by inputting the newly added data into the target detection and segmentation model to adapt to the newly added data points. The incremental learning method updates and iterates the target detection and segmentation model in real time according to the membership matrix parameter update.

8. The cable defect detection method using a binocular intelligent inspection robot according to claim 1, characterized in that: The multi-sensor data information fusion model uses an improved BP neural network algorithm model to predict the error of the data information after multi-sensor fusion. The formula is expressed as: In formula (4), is the error of the fused data information, is the weight function of the qth layer network in the improved BP neural network algorithm model, x is the threshold value of the qth layer network in the neural network algorithm model, It is the data input node of the qth layer network in the neural network algorithm model. It represents the output node of the qth layer network in the neural network algorithm model, and v represents the step size in the neural network algorithm model. Due to the different feature parameters and dimensions in the input layer, the fused data information is expressed as: In formula (5), Y represents the fused data, Represented as the hidden node of the qth layer network in the neural network algorithm model, is the weighting factor in the neural network algorithm model.

Images