Remote Visual Monitoring Method and System for Conductor Breakage Operations
By using a remote visualization monitoring method based on 3D reconstruction and multi-source state perception, the problems of human error and safety in conductor cutting operations have been solved, and safe, controllable and precise conductor cutting operations have been achieved.
Patent Information
- Application Number
- CN202511926961.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-19
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2045-12-19
AI Technical Summary
Traditional wire cutting operations rely on manual observation, which has a high rate of misjudgment, cannot be quantified and analyzed in real time, makes it difficult to ensure safety, and lacks real-time monitoring of the wire clamp status, resulting in high risks in the wire cutting process.
By using a remote visualization monitoring method that integrates 3D reconstruction with multi-source state perception, the three-dimensional spatial information of the wire and the cutting edge is obtained. Combined with multi-dimensional state indication signals, safety decisions are made, and the path and parameters of the working equipment are dynamically adjusted to achieve wire cutting operation.
This has improved the safety, robustness, and automation of wire cutting operations, reduced the risk of human error, and ensured that the wire cutting process is completed under safe, controllable, and precise conditions.
Smart Images

Figure CN121680127B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote monitoring technology for conductor breakage, and more specifically, to a method and system for remote visual monitoring of conductor breakage operations. Background Technology
[0002] Conductor cutting is a high-risk operation in power maintenance and construction, especially dangerous in high-altitude, complex terrain, or live-line working environments. Traditional wire cutting operations typically rely on manual on-site work, judging whether conditions are suitable for cutting by manually observing the conductor's position, the direction of the cutting blade, and the status of the wire clamp. However, manual methods have significant limitations: operators must repeatedly approach the conductor in high-risk areas to determine its position, and are easily affected by spatial obstructions, limited visibility, and environmental vibrations, resulting in a high probability of misjudgment. Furthermore, during the wire cutting process, multiple status signals such as mechanical load, vibration, and blade pressure are often judged based on manual experience, lacking real-time quantitative analysis and making it difficult to identify sudden risks in a timely manner.
[0003] During wire cutting, when the conductor is affected by wind disturbance or mechanical vibration, the cutting edge cannot automatically adjust its approach path according to the instantaneous position of the conductor, which can easily lead to problems such as off-center cutting, incomplete cutting, or jamming, seriously affecting the quality and safety of wire cutting operations. Existing technology also lacks a real-time monitoring and position matching mechanism for the contact status of the wire clamp, failing to ensure correct conductor positioning and unable to automatically block the wire cutting command in case of accidental triggering, thus posing a safety vulnerability.
[0004] To address the above problems, this invention proposes a solution. Summary of the Invention
[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a remote visual monitoring method and system for conductor breakage operations. By integrating three-dimensional reconstruction with multi-source state perception, the remote visual monitoring method solves the problems of high risk of wire cutting, high misjudgment rate, and inability to achieve dynamic safety control caused by the difficulty of accurately judging the position of the conductor, the attitude of the cutting edge, and the load vibration state by humans.
[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, this application provides a remote visual monitoring method for wire breakage operations. The method includes: acquiring multi-view image data and performing three-dimensional reconstruction of the wire-cutting area to form a three-dimensional spatial structure information set including the spatial position of the wire and the relative pose of the cutting edge; extracting key structural features of the work object and the work equipment based on the three-dimensional spatial structure information set, and generating real-time spatial annotation information; acquiring multi-source status data and performing correlation analysis to generate multi-dimensional status indication signals; making safety decisions based on the real-time spatial annotation information and multi-dimensional status indication signals, generating safety control commands; and when the safety control commands permit the operation, dynamically adjusting the movement path and operation parameters of the work equipment based on the three-dimensional spatial structure information set, and executing the wire-cutting operation.
[0007] In one embodiment, a three-dimensional reconstruction of the wire-cutting region is performed to form a three-dimensional spatial structure information set including the spatial position of the wire and the relative pose of the blade. This includes: acquiring the calibration parameters of each camera and converting them to the same global coordinate system; extracting key feature points from the multi-view image data based on the calibration parameters, and performing feature matching on the corresponding targets in different views to obtain feature point matching pairs; establishing epipolar constraints between the left and right views based on the feature point matching pairs, and reading the pixel coordinates of the matching points corresponding to the same target in the left and right views respectively; calculating the pixel offset using the pixel coordinates of the matching points and forming a disparity matrix; performing depth calculation on each pixel in the disparity matrix to generate a three-dimensional depth value, and rearranging it into a two-dimensional depth matrix according to the pixel position, mapping the matrix to the global coordinate system to generate an initial depth map; generating a point cloud dataset based on the initial depth map and performing recognition and segmentation to generate a three-dimensional spatial structure information set.
[0008] In one embodiment, a point cloud dataset is generated based on the initial depth map and then identified and segmented to generate a three-dimensional spatial structure information set. This includes: based on the initial depth map, performing depth fusion on the three-dimensional depth values obtained from different perspectives in the global coordinate system and converting them into a point cloud dataset; identifying key structural features of the point cloud dataset and classifying and segmenting them to obtain a structured point cloud; and based on the structured point cloud, solving for the key parameters of the three-dimensional spatial structure information set and generating the three-dimensional spatial structure information set.
[0009] In one embodiment, key structural features are identified from the point cloud dataset. These key structural features include the conductor center trajectory, the knife-edge plane, and the boundary features of the clamping mechanism. This process includes: spatially aligning the point cloud dataset; performing noise reduction and density equalization processing on the aligned point cloud dataset to obtain point cloud density distribution features; spatially clustering the point cloud density distribution features to divide the wire-cutting region point cloud into conductor candidate areas, knife-edge candidate areas, and clamping mechanism candidate areas; extracting and fitting local linear trends from the conductor candidate area point cloud to generate a continuous conductor center trajectory; generating a knife-edge plane from the knife-edge candidate area point cloud based on a knife-edge plane model; and extracting complete clamping mechanism boundary features from the clamping mechanism candidate area point cloud.
[0010] In one embodiment, based on a three-dimensional spatial structure information set, key structural features of the work object and the work equipment are extracted, and real-time spatial annotation information is generated. This includes: mapping the three-dimensional coordinates of each key structural feature to a unified display coordinate system; projecting the key structural features onto a two-dimensional view on the PAD display based on the mapped coordinates to form a projection position; generating a corresponding annotation symbol for each key structure based on the projection position; and binding the spatial parameters of the key structure with the annotation symbol to generate real-time spatial annotation information. The real-time spatial annotation information includes the three-dimensional spatial coordinates and motion state of the wire position, the relative pose of the blade edge, and the contact point of the wire clamp.
[0011] In one embodiment, acquiring multi-source state data and performing correlation analysis to generate multi-dimensional state indication signals includes: preprocessing the multi-source state data and extracting trend features for each type of signal; based on the trend features, performing correlation analysis on mechanical load curves, blade pressure fluctuations, and vibration characteristics to construct a weighted causal relationship matrix between signals; and mapping each trend feature to a multi-dimensional state indication signal according to the weighted causal relationship matrix.
[0012] In one embodiment, based on trend characteristics, a correlation analysis is performed on the mechanical load curve, the blade pressure fluctuation, and the vibration characteristics to construct a weighted causal relationship matrix between the signals. This includes: converting various signals into multi-channel time series according to a preset sampling time interval; based on the multi-channel time series, forming a historical window sequence from the previous N time steps at the current time point as input for the attention mechanism; calculating the sensitivity vector of each signal channel at the current time step according to the trend characteristics; obtaining the second correlation data between each signal channel and constructing an initial correlation matrix; based on the initial correlation matrix, analyzing the time delay peak values between different channels to determine the dominant direction and potential causal relationship of the mechanical load, blade pressure, and vibration signals, and obtaining the time delay directional characteristics; weighting and summing the time delay directional characteristics with the zero-time-delay Pearson correlation coefficient to generate initial causal weights; and correcting the initial causal weights according to the sensitivity vectors to generate an initial causal weight matrix.
[0013] In one embodiment, safety decisions are made based on real-time spatial annotation information and multi-dimensional status indication signals to generate remote wire-cutting safety control instructions. This includes: performing time-series smoothing on the three-dimensional coordinates of the conductor, blade, and wire clamp contact, and obtaining the current position and movement trend; mapping the multi-dimensional status indication signals to the spatial position to form an operation status vector; performing a safety risk assessment on the wire-cutting operation based on the operation status vector, and generating the current wire-cutting safety level. The safety risk assessment includes spatial collision assessment, load and vibration assessment, and wire clamp status assessment. The safety level includes safe, warning, and danger. Based on the safety level, remote wire-cutting safety control instructions are generated, including allowing wire cutting, suggesting adjusting the conductor position, and prohibiting wire cutting.
[0014] In one embodiment, the motion path and operating parameters of the working device are dynamically adjusted based on a three-dimensional spatial structure information set, and a wire-cutting operation is performed. This includes: obtaining a blade approach path based on the spatial position of the conductor and the initial pose of the blade in the three-dimensional spatial structure information set; dynamically adjusting the blade approach path to obtain a planned dynamic path, wherein the dynamic adjustment includes using the trend of changes in the spatial position of the conductor within a historical time window to predict the current position and direction of movement of the conductor and correcting the blade trajectory; calculating the wire-cutting speed based on the real-time minimum safe distance between the blade and the conductor and the mapping relationship between mechanical load, vibration data and wire-cutting speed; controlling the blade to move along the path to the conductor cutting position according to the planned dynamic path and wire-cutting speed, initiating the wire-cutting action; and recording a wire-cutting completion signal after the blade contacts the conductor and completes the cut.
[0015] Secondly, this application provides a remote visual monitoring system for conductor breakage operations. This system includes: a three-dimensional spatial structure information set construction module, used to acquire multi-view image data and perform three-dimensional reconstruction of the cutting area to form a three-dimensional spatial structure information set including the spatial position of the conductor and the relative pose of the cutting edge; a labeling information generation module, used to extract key structural features of the work object and the work equipment based on the three-dimensional spatial structure information set, and generate real-time spatial labeling information; a state perception module, used to acquire multi-source state data and perform correlation analysis to generate multi-dimensional state indication signals; and a signal receiving and execution module, used to make safety decisions based on the real-time spatial labeling information and multi-dimensional state indication signals, generate safety control commands, and when the safety control commands allow the operation, dynamically adjust the movement path and operation parameters of the work equipment based on the three-dimensional spatial structure information set, and execute the wire cutting operation.
[0016] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: By combining multi-view 3D reconstruction, multi-source mechanical state perception, real-time causal correlation analysis, adaptive safety control, and dynamic path and speed planning, a fully intelligent and closed-loop control system is achieved, encompassing scene perception, state understanding, and wire-cutting action execution. Its main advantages are: it can simultaneously acquire high-precision 3D spatial information of the conductor, cutting edge, and wire clamp, as well as real-time dynamic states such as mechanical load, vibration, and pressure; and accurately identify dangerous states such as obstruction, vibration, and lack of clamping through multi-dimensional feature extraction, correlation analysis, and causal inference; through spatial collision assessment, load / vibration multi-index judgment, and wire clamp state matching, it can perform quantifiable, interpretable, and dynamically updated risk assessments of the wire-cutting action and generate graded safety control commands; during the execution phase, it can automatically adjust the cutting edge path and speed based on the real-time model to conform to the actual position of the conductor and avoid obstacles, thereby ensuring that the wire-cutting process is completed under safe, controllable, and precise conditions. This significantly improves the safety, robustness, and automation level of remote wire-cutting operations and reduces the risk of human error. Attached Figure Description
[0017] Figure 1 This is a schematic flowchart of a remote visual monitoring method for conductor breakage operations provided in an embodiment of this application.
[0018] Figure 2 This is a schematic diagram of the structure of a remote visual monitoring system for wire breakage operations provided in an embodiment of this application.
[0019] Figure 3 This is a framework diagram of a remote visual monitoring system for conductor breakage operations provided in an embodiment of this application. Detailed Implementation
[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0021] Reference Figure 1 As shown in the schematic diagram, the remote visual monitoring method for conductor breakage operations provided by the present invention includes the following steps: S1. Collect multi-view image data and perform three-dimensional reconstruction of the wire-cutting area based on the multi-view image data to form a three-dimensional spatial structure information set including the spatial position of the conductor and the relative pose of the blade.
[0022] Based on structural analysis of the wire-cutting area, several cameras at different angles are installed at key locations of the automatic wire-cutting device and the wire-clamping mechanism, enabling simultaneous acquisition of multi-view image data, including top-down, side-down, and close-up views of the blade edge. For example... Figure 3 As shown, 001 is the automatic wire cutting device, 002 is the sensor, 003 is the data acquisition and control module of the automatic wire cutting device, 004 is the camera, 005 is the gateway, 006 is the PAD operating terminal, 007 is the wire clamp, 008 is the data acquisition and control module of the wire clamp, 009 is WiFi for data transmission, 010 is the tower side, and 011 is the ground side.
[0023] The basic functions of the PAD operating terminal include: real-time playback of camera footage, operation of the forward and reverse rotation of the motor on the automatic wire cutting device, operation of the forward and reverse rotation of the motor on the wire clamp, and alarm function of the distance sensor at the blade of the automatic wire cutting device; PAD operation commands include: wire cutting, blade reset, and pause commands for the automatic wire cutting device, and unlocking and locking commands for the wire clamp; PAD screen display includes: camera footage + operation commands + signal feedback display.
[0024] In this embodiment, the wire-cutting area is reconstructed in three dimensions based on multi-view image data to form a three-dimensional spatial structure information set including the spatial position of the conductor and the relative pose of the cutting edge, including: Based on multi-view image data, calibration parameters of each camera are obtained through a structured light reference point calibration method. The calibration parameters include intrinsic parameter matrix, distortion coefficient, extrinsic parameter rotation matrix and translation vector, and the coordinate systems of all cameras are transformed to the same global coordinate system. Based on the calibration parameters, key feature points including the outer contour of the wire, the edge of the blade, the contact point of the wire clamp and the surrounding structure are extracted from the image data of each viewpoint, and feature matching is performed on the corresponding targets in different viewpoints to obtain feature point matching pairs. The feature matching is based on the Euclidean distance threshold method, which matches corresponding targets in different viewpoints with a distance less than a preset threshold. The feature point matching results provide necessary constraints for subsequent disparity estimation.
[0025] Based on the feature point matching pairs, establish epipolar constraints between the left and right (or any two cameras from multiple perspectives) views, so that each pair of matching points is confined to the same epipolar line in the corresponding image plane, in order to ensure that the pixel search range has geometric consistency in the subsequent disparity solution process. Based on epipolar constraints, the pixel coordinates of the matching points corresponding to the same target are read in the left and right views respectively; By matching the pixel coordinates of the points, the pixel offset of the same spatial target in different image planes is calculated, and the pixel offset is solved for all matching point pairs to form a disparity matrix containing the effective disparity distribution of the entire image. The specific formula for calculating the pixel offset is as follows:
[0026] In the formula, This is the pixel offset. The horizontal pixel coordinates of the target point on the left view plane. The horizontal pixel coordinates of the target point on the right view plane.
[0027] Based on the pixel offset, combined with the baseline distance and focal length of the left and right cameras, the three-dimensional depth value of each pixel is calculated. The specific formula for calculating the three-dimensional depth value is as follows:
[0028] In the formula, This is a three-dimensional depth value. Baseline distance, It is the focal length.
[0029] For each pixel in the disparity matrix, a depth value is calculated to generate a three-dimensional depth value that corresponds one-to-one with the pixel coordinates of the two-dimensional image. This assigns a physical spatial distance to each effective pixel in the image. The three-dimensional depth values are rearranged into a two-dimensional depth matrix according to the pixel position and mapped to the global coordinate system. Finally, an initial depth map of the shearing area is generated, which reflects the position of the wire, the relative spatial distance of the blade, and the three-dimensional shape of the surrounding structure. A point cloud dataset is generated based on the initial depth map and then identified and segmented to generate a set of three-dimensional spatial structure information, including the spatial position of the conductor and the relative pose of the knife edge.
[0030] Furthermore, a point cloud dataset is generated based on the initial depth map and then identified and segmented to generate a 3D spatial structure information set including the spatial location of the conductor and the relative pose of the knife edge, including: Based on the initial depth map, the 3D depth values obtained from different perspectives are fused in the global coordinate system to eliminate the depth discontinuity caused by noise and viewpoint occlusion. The fused 3D depth values are then transformed into a dense 3D point cloud to form a point cloud dataset covering the wire, knife edge and surrounding structure. For the point cloud dataset, key structural features are identified and classified and segmented so that wires, knife edges, wire clamps and obstacles form independent structural layers in three-dimensional space, resulting in a structured point cloud. The key structural features include the center trajectory of the wire, the plane of the knife edge and the boundary features of the wire clamping mechanism. Based on structured point clouds, the key parameters of the three-dimensional spatial structure information set are solved. The key parameters of the three-dimensional spatial structure information set include the actual position of the conductor in space and the three-dimensional orientation (position + orientation) of the knife edge. Among them, solving the key parameters of the three-dimensional spatial structure information set includes calculating the center trajectory of the conductor in three-dimensional space through least squares fitting to obtain the true position of the conductor in space, and solving the three-dimensional pose (including position and orientation) of the knife edge through point cloud normal vectors.
[0031] Based on the key parameters of the three-dimensional spatial structure information set, a visualized three-dimensional spatial structure information set of the wire cutting area is generated through a triangular meshing algorithm, enabling the PAD operating terminal to present the complete three-dimensional structure of the wire spatial position and the relative pose of the cutting edge in real time.
[0032] It should be noted that by acquiring images from multiple perspectives, performing precise calibration, matching feature points, calculating disparity and mapping depth, and combining point cloud generation, geometric fitting and structural recognition, the three-dimensional spatial structure information set of the wire cutting area can be reconstructed completely and accurately in a remote environment. This enables comprehensive visualization of the conductor's spatial position and the blade's posture, allowing operators to monitor the machine's status in real time, improving the safety and accuracy of wire cutting operations, while effectively compensating for the monitoring blind spots caused by single-view and traditional limited status feedback information.
[0033] Furthermore, for the point cloud dataset, key structural features are identified, including the conductor center trajectory, the knife-edge plane, and the boundary features of the wire clamping mechanism, including: For point cloud datasets, spatial alignment of point clouds is performed using camera extrinsic and intrinsic parameters, so that point clouds from different cameras can form a consistent 3D dataset in a unified coordinate system. Based on the aligned point cloud dataset, noise reduction and density equalization are performed on the point cloud to make the point cloud as a whole present continuity and the local structure clear, and the point cloud density distribution characteristics are obtained. Among them, noise reduction and density equalization can be achieved by statistical outlier detection or radius neighborhood filtering.
[0034] Spatial clustering is performed on the point cloud density distribution characteristics to divide the point cloud of the wire cutting region into candidate areas for conductors, candidate areas for knife edges, and candidate areas for wire clamping mechanisms. Based on the point cloud of the candidate traverse area, local linear trends are extracted through principal direction analysis, and a RANSAC line model is used for fitting to generate a continuous traverse center trajectory, which is used to accurately represent the actual position and shape of the traverse in three-dimensional space. Based on the point cloud of the candidate cutting edge region, a cutting edge plane model is initially established by RANSAC plane fitting. Then, residual elimination and curvature minimization are used to optimize and improve the plane accuracy. Finally, a high-precision cutting edge plane is generated to determine the attitude and cutting direction of the cutting edge in three-dimensional space. Based on the point cloud of the candidate area of the wire clamp, the boundary features of the complete wire clamping mechanism are extracted to determine the locking and unlocking status of the wire clamp and to determine the spatial position relationship of the wire in the wire clamping mechanism.
[0035] It should be noted that by performing spatial alignment, noise reduction, and density equalization on multi-view point cloud data, and combining this with geometric fitting and structural recognition algorithms for wires, cutting edges, and wire clamping mechanisms, the continuous trajectory, planar posture, and boundary contour of each key structure can be accurately extracted in three-dimensional space. This enables independent identification and spatial positioning of wires, cutting edges, and wire clamping mechanisms, thereby providing high-precision, complete, and reliable mechanical status information for remote monitoring, operational judgment, and automated control, effectively improving the safety, accuracy, and operability of wire cutting operations.
[0036] S2 extracts key structural features of the work object and work equipment based on the three-dimensional spatial structure information set, and generates real-time spatial annotation information on the PAD display, so that the operator can observe the automatically annotated cutting path, wire clamp contact, cutting position offset and wire clamp status in a multi-view screen.
[0037] In this embodiment, key structural features are extracted based on the three-dimensional spatial structure information set, and real-time spatial annotation information is generated by overlaying it on the PAD display, including: Based on the three-dimensional spatial structure information set, the key structural features are extracted and the three-dimensional coordinates of each key structural feature are mapped to the unified display coordinate system of the PAD display end, so that the spatial position of the key structural wires, knife edges and wire clamps maintains the true proportion and relative relationship on the display end. Based on the mapped coordinates, the key structural features are projected onto the two-dimensional view of the PAD display using a perspective projection method, forming the projection position under each viewpoint; Based on the projection position, corresponding annotation symbols are generated for each key structure. The annotation symbols include wire path lines, knife edge plane boundary boxes, wire clamp contact marks and related attribute information (such as cutting direction and unlocking status). By binding the spatial parameters of key structures (such as conductor curvature, knife edge normal vector, and wire clamp contact status) with annotation symbols, the annotation and real-time status information are displayed synchronously, generating real-time spatial annotation information. The real-time spatial annotation information includes the conductor position, knife edge relative pose, and the three-dimensional spatial coordinates and motion status of the wire clamp contact.
[0038] S3. Acquire multi-source state data and perform real-time correlation analysis on the mechanical load curve, blade pressure fluctuation, and vibration characteristics to generate multi-dimensional state indication signals. These signals include whether the blade is obstructed, whether the wire is clamped, whether the wire is vibrating, and whether the motor is overloaded. These state indication signals serve as constraints for determining the feasibility of the wire-cutting action. In this automatic wire cutting device, sensors for detecting load, current, vibration amplitude, and blade pressure are respectively arranged in the motor, wire clamp, and blade mechanism to form a state sensing module. This module can collect mechanical state parameters in real time to form multi-source state data, which includes mechanical load curves, blade pressure fluctuation signals, wire and mechanism vibration signals, and motor current and speed data.
[0039] In this embodiment, multi-source state data is acquired, and real-time correlation analysis is performed on the mechanical load curve, blade pressure fluctuation, and vibration characteristics to generate a multi-dimensional state indication signal, including: The multi-source state data is preprocessed, including low-pass filtering of mechanical load curves and motor data, moving average filtering of knife edge pressure fluctuation signals, band-pass filtering of vibration signals, and extraction of time and frequency domain features. Based on the preprocessed multi-source state data, trend features are extracted for each type of signal. The trend features include amplitude, gradient and trend degree, and spectral energy distribution. The trend features are used to characterize mechanical state and dynamic changes. Based on trend characteristics, correlation analysis is performed on mechanical load curves, knife edge pressure fluctuations and vibration characteristics to construct a weighted causal relationship matrix between signals. The correlation analysis is used to reveal the real-time relationship between state characteristics such as wire clamping, knife edge obstruction and abnormal vibration. Based on the weighted causal relationship matrix, each trend feature is mapped to a multi-dimensional state indication signal, which includes the states of "whether the blade is blocked", "whether the wire is clamped", "whether the wire is shaking", and "whether the motor is overloaded".
[0040] The multi-dimensional status indication signal can be displayed in real time on the PAD operating terminal and serves as a condition constraint for determining the feasibility of the wire-cutting action.
[0041] Furthermore, based on trend characteristics, a correlation analysis is performed on the mechanical load curve, the knife edge pressure fluctuation, and the vibration characteristics to construct a weighted causal relationship matrix between the signals, including: According to the preset sampling time interval, various signals are converted into multi-channel time sequences; Based on multi-channel time series, the previous N time steps of the current time point are combined into a historical window sequence, which is used as the input of the attention mechanism to ensure that the recent historical change trend can be comprehensively considered when analyzing the current signal state. Based on the trend characteristics, calculate the sensitivity vector of each signal channel at the current time step; The specific formula for calculating the sensitivity vector is as follows:
[0042] In the formula, For the sensitivity vector, For compression functions (such as the Sigmoid function). , , These are the fusion coefficients, The amplitude of the signal. The gradient of the signal. Let be the spectral energy distribution, and b be the bias term.
[0043] The second correlation data between each signal channel is obtained. The second correlation data includes cross-correlation data, maximum response time delay correlation data, and nonlinear correlation data of mutual information. An initial correlation matrix reflecting the correlation strength of different signal channels within the current window is constructed by weighting the elements. The acquisition of cross-correlation data includes: calculating the cross-correlation function within a time window T for the normalized feature sequences of signal channel i and signal channel j; The specific formula for calculating the cross-correlation function is as follows:
[0044] In the formula, Let be the cross-correlation value between channel i and channel j under time delay τ, used to characterize the degree of linear correlation and directionality between the two signals, and T be the total number of sampling points in the signal sampling sequence, i.e., the signal length. The signal value of the i-th signal channel at discrete time point t, such as mechanical load, blade pressure, or vibration. Let be the signal value of the j-th signal channel at discrete time point t. Time delay indicates that the signal... The amount by which time is shifted forward or backward.
[0045] exist Time normalization yields cross-correlation data.
[0046] The specific calculation formula for the cross-correlation data is as follows:
[0047] In the formula, This represents the cross-correlation value between the two channels at zero time delay (at the same moment). For signal The squared L2 norm is used to normalize its energy or amplitude. For signal The square of the second norm, This represents the zero-delay normalized cross-correlation data between channel i and channel j, indicating the degree of instantaneous linear correlation.
[0048] Among them, the maximum response time delay correlation data is based on the cross-correlation function. The maximum time delay value.
[0049] The nonlinear correlation data of mutual information is obtained by discretizing the histograms of signal channels i and j to obtain the joint probability distribution. With marginal distribution Then calculate the mutual information; and combine it with the signal entropy to obtain the nonlinear correlation data of the mutual information. The specific formula for calculating the mutual information is as follows:
[0050] The specific calculation formula for the nonlinear correlation data of the mutual information is as follows:
[0051] In the formula, For mutual information, Let k be the discrete value interval number for signal channel i, and let k be the range of values that depends on the number of bins. Let be the number of the discrete value interval of signal channel j. Signal channel j falls into the sub-box The marginal probability, The closer the nonlinear correlation data of mutual information is to 1, the stronger the nonlinear coupling between the two signals, which can reveal the complex nonlinear transmission relationship between mechanical load, blade pressure, and vibration. Let i be the entropy value of signal channel i. Let be the entropy value of signal channel j.
[0052] Based on the initial correlation matrix, the peak time delays between different channels are analyzed to determine the dominant direction and potential causal relationship between mechanical load, blade pressure, and vibration signal. Specifically, by identifying the maximum correlation response time delay of different signal channels, the directional time delay characteristics of each channel relative to other channels are obtained, which are used to describe the potential causal direction. When the peak time delay... When signal i dominates signal j, signal j dominates signal i; otherwise, signal j dominates signal i. The time delay peak value is normalized to the time delay directionality.
[0053] The time-delay directional features are weighted and summed with the zero-delay Pearson correlation coefficient to generate the initial causal weights of each channel to the other channels; The initial causal weights are corrected based on the sensitivity vector, and the correlation strength and time delay directionality are fused based on the corrected initial causal weights to generate an initial causal weight matrix, which is used to represent the real-time causal relationship between mechanical load, blade pressure and vibration signal.
[0054] The specific calculation formula for correcting the initial causal weights is as follows:
[0055] In the formula, The corrected initial causal weights, As the initial causal weights, For fluctuation gain parameters, This is the sensitivity vector.
[0056] It should be noted that by comprehensively introducing trend characteristics such as signal amplitude changes, gradient abrupt changes, and spectral energy characteristics under a unified time reference, and further integrating linear and nonlinear indices such as cross-correlation, maximum time delay response, and mutual information, the coupling relationship between mechanical load, blade pressure, and vibration signals can be accurately characterized from multiple dimensions, scales, and mechanisms. Simultaneously, by performing time-series modeling and extracting sensitivity vectors from historical windows, the system can not only capture real-time state changes but also dynamically identify key influencing factors of different signals at the current moment. Based on this, the causal direction is inferred using time delay directionality, and the initial causal weights are corrected using sensitivity vectors, resulting in a weighted causal relationship matrix that possesses real-time, directional, dynamic, and adaptive properties. This allows for a more accurate revelation of potential causal links between load, pressure, and vibration, providing reliable decision-making basis for anomaly warning, fault identification, and operational strategy optimization, and significantly improving the safety and operational intelligence level of the wire shearing device.
[0057] S4. Based on real-time spatial annotation information and multi-dimensional status indication signals, a safety decision is made, and a remote wire-cutting safety control command is generated. When the PAD operation terminal receives the control command to allow wire cutting, the movement path and operation parameters of the working instrument are dynamically adjusted according to the three-dimensional spatial structure information set to ensure that the movement trajectory of the blade is consistent with the actual spatial position of the wire, thereby ensuring the accuracy and safety of wire cutting, and the wire cutting operation is performed. The remote wire-cutting safety control command includes allowing wire cutting, prohibiting wire cutting, adjusting the position of the wire, and automatic pause.
[0058] In this embodiment, a safety decision is made based on real-time spatial annotation information and multi-dimensional status indication signals to generate a remote wire-cutting safety control command, including: The three-dimensional coordinates of the wire, knife edge, and wire clamp contact are smoothed over time, and the current position and motion trend are obtained by Kalman filtering. Multidimensional status indication signals are mapped to spatial locations to form a unified operation status vector; Based on the operation status vector, a safety risk assessment is performed on the wire cutting operation, and the current wire cutting safety level is generated. The safety risk assessment includes spatial collision assessment, load and vibration assessment, and wire clamp status assessment. The safety level includes safe, warning, and danger. Space collision assessment includes: Based on the three-dimensional positional relationship and movement trend of the conductor and the knife edge, the minimum Euclidean distance is calculated for each key point of the conductor and the key point of the knife edge. When the minimum Euclidean distance is less than the preset safety threshold, it is determined that there is a risk of collision or interference, otherwise the work area is safe. Load and vibration assessment, including: The system acquires the mean, variance, and peak value of mechanical load curves, vibration amplitude, and acceleration signals within a historical window. The peak value is used to detect instantaneous or short-term anomalies, such as sudden mechanical impacts, excessively high blade pressure, or vibration spikes. If the peak value exceeds the set safety threshold, a warning or operation prohibition command is immediately triggered. If the mean value exceeds the preset normal range for a long period, it indicates that the load or vibration is continuously abnormal, prompting a need to decelerate or suspend operation. If the variance exceeds the preset threshold, it indicates that the signal fluctuates violently. Even if the peak value does not exceed the threshold, there may be potential risks, requiring the introduction of warnings, deceleration, or operation suspension. Cable clamp status assessment, including: The three-dimensional position of the wire clamp contact and the contact opening and closing status signal are obtained, and the key points of the wire center trajectory are spatially matched with the position of the wire clamp contact. Calculate the Euclidean distance between the conductor and the contact. If the Euclidean distance between the conductor and the contact exceeds the preset allowable deviation, it is determined that the conductor is not correctly positioned. If the contact open / close status signal is on and the wire is correctly in place, the wire clamp is considered ready. If the contact opening / closing status signal is closed or the wire is not in place, the wire clamp is determined not to meet the wire cutting condition. If the wire clamp does not meet the conditions for cutting the wire, it is considered to be a high-risk situation.
[0059] The spatial collision assessment results, load and vibration assessment results, and wire clamp status assessment results are combined in a weighted manner to generate a comprehensive risk coefficient. Among them, the spatial collision assessment result has a value range of [0,1], where 0 indicates no collision risk and 1 indicates a collision risk; the load and vibration assessment result has a value range of [0,1]. The judgment results of peak value, mean and variance are normalized to the interval of [0,1] and then weighted and summed according to preset weights to obtain the load and vibration assessment result; the cable clamp status assessment result has a value range of [0,1], where 0 indicates that the cable clamp is safe and ready and 1 indicates that the cable clamp is abnormal.
[0060] Based on the comprehensive risk coefficient and the established safety level threshold , Generate security level; Where 0 < comprehensive risk coefficient < Then it is safe; like Overall risk coefficient < This is a warning; If the comprehensive risk coefficient If so, it is dangerous.
[0061] Generate remote wire-cutting safety control commands based on the security level; If the security level is "secure", then a "allow wire cutting" command will be generated. If the safety level is "Warning", then generate a "suggest adjusting the conductor position" or "automatic deceleration / pause" command; If the work status is assessed as "dangerous", a "Do Not Cut Wire" or "Emergency Stop" instruction will be generated.
[0062] It should be noted that by real-time fusion and dynamic analysis of the three-dimensional spatial position information of the wire, blade, and wire clamp with multi-dimensional state signals such as mechanical load, vibration, and blade pressure, the system quantifies risks using methods such as minimum safe distance calculation, load and vibration peak / mean / variance determination, and wire clamp state matching. A comprehensive risk coefficient is generated through weighted fusion, automatically classifying safety, warning, and danger levels. This provides a comprehensive and quantitative risk assessment of the wire cutting process. Furthermore, it can dynamically and accurately classify safety levels and automatically generate corresponding control commands (such as allowing wire cutting, deceleration / pause, or emergency stop) based on the safety level. This not only ensures operator safety but also improves the efficiency and reliability of remote wire cutting operations, effectively reduces operational risks, and achieves closed-loop intelligent control throughout the entire process.
[0063] Furthermore, when the PAD operating terminal receives a control command to allow wire cutting, it uses the motion path and operating parameters of the working instrument based on the three-dimensional spatial structure information set to ensure that the movement trajectory of the cutting edge is consistent with the actual spatial position of the wire, thereby ensuring wire cutting accuracy and safety, and performing the wire cutting operation, including: Based on the spatial position of the conductor and the initial pose of the knife edge in the three-dimensional spatial structure information set, the approach path of the knife edge is obtained; The method for obtaining the approach path of the blade edge is to use the shortest safe path planning algorithm (such as 3D spline fitting or RRT algorithm). The approach path of the knife edge is dynamically adjusted to obtain a planned dynamic path. The dynamic adjustment includes using the trend of the spatial position change of the conductor within the historical time window, using Kalman filtering to predict the trajectory, estimating the current position and direction of movement of the conductor, correcting the knife edge trajectory, so that the knife edge movement is consistent with the real-time spatial position of the conductor, and combining the boundary points of the wire clamp and the obstacle point cloud to plan an obstacle avoidance path to ensure the safety of the knife edge movement. Based on the real-time minimum safe distance between the cutting edge and the conductor, and the mapping relationship between mechanical load, vibration data, and wire cutting speed, a suitable wire cutting speed is calculated, including: Obtain the real-time minimum safe distance (Euclidean distance) between the cutting edge and the wire, as well as the mechanical load (load value of the wire cutting device at the current time step) and vibration data (vibration amplitude of the wire cutting device or cutting edge at the current time step). The minimum safety distance, mechanical load, and vibration data are normalized, and the mapping relationship between shearing speed and minimum safety distance, mechanical load, and vibration data is defined. The specific calculation formula for the mapping relationship is as follows:
[0064] In the formula, For the shearing speed, To allow the maximum wire cutting speed, The mapping function for the minimum safe distance increases with the minimum safe distance, and includes: ,in, The normalized value of the minimum safe distance. The preset safe distance threshold, This is a mapping function for mechanical loads, used to reduce the shearing speed under high load conditions. ,in This is the normalized value of the mechanical load. Let be the mapping function for vibration data, where ,in This is the vibration normalized value.
[0065] Specifically, when the minimum safe distance decreases or the load / vibration exceeds the preset safe range, the blade movement speed is reduced; when the load and vibration are within the safe range, the blade is allowed to move at the preset maximum speed.
[0066] Based on the planned dynamic path and wire cutting speed, the blade is controlled to move along the path to the wire cutting position, and the wire cutting action is initiated. When the blade contacts the wire and completes the cutting, the wire cutting completion signal is recorded.
[0067] Reference Figure 2As shown in the schematic diagram, the remote visual monitoring system for conductor breakage operations provided by this invention includes a three-dimensional spatial structure information set construction module, a labeling information generation module, a status perception module, and a signal receiving and execution module. The modules are interconnected. The 3D spatial structure information set construction module is used to collect multi-view image data and perform 3D reconstruction of the wire cutting area to form a 3D spatial structure information set including the spatial position of the wire, the relative pose of the blade, and surrounding obstacles. The annotation information generation module is used to extract key structural features based on the three-dimensional spatial structural information set and generate real-time spatial annotation information. The state awareness module is used to acquire multi-source state data, perform real-time correlation analysis, and generate multi-dimensional state indication signals. The signal receiving and execution module is used to generate remote wire-cutting safety control commands based on real-time spatial annotation information and multi-dimensional status indication signals. When the PAD operation terminal receives the control command that allows wire cutting, it dynamically adjusts the blade approach path and wire-cutting speed according to the three-dimensional spatial structure information set to perform the wire-cutting operation.
[0068] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.
[0069] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.
[0070] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0071] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
[0072] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0073] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for remote visual monitoring of a wire breakage operation, characterized in that, Includes the following steps: Multi-view image data is collected, and the wire-cutting area is reconstructed in three dimensions to form a set of three-dimensional spatial structure information including the spatial position of the conductor and the relative pose of the blade. Based on the three-dimensional spatial structure information set, the key structural features of the work object and the work equipment are extracted, and real-time spatial annotation information is generated. Acquire multi-source state data and perform correlation analysis to generate multi-dimensional state indication signals, including: Preprocess the multi-source state data and extract trend features from each signal; Based on trend characteristics, correlation analysis is performed on mechanical load curves, knife edge pressure fluctuations and vibration characteristics to construct a weighted causal relationship matrix between signals; Based on the weighted causal relationship matrix, each trend feature is mapped into a multi-dimensional state indication signal; Safety decisions are made based on real-time spatial annotation information and multi-dimensional status indication signals, and safety control commands are generated. When the safety control commands allow the operation, the movement path and operation parameters of the working equipment are dynamically adjusted according to the three-dimensional spatial structure information set, and the wire cutting operation is executed. The step of making security decisions based on real-time spatial annotation information and multi-dimensional state indication signals, and generating security control commands, includes: The three-dimensional coordinates of the wire, knife edge, and wire clamp contact are smoothed over time to obtain the current position and motion trend. Multidimensional status indication signals are mapped to spatial locations to form an operation status vector; Based on the operation status vector, a safety risk assessment is performed on the wire cutting operation, and the current wire cutting safety level is generated. The safety risk assessment includes spatial collision assessment, load and vibration assessment, and wire clamp status assessment. The safety level includes safe, warning, and danger. Based on the security level, a remote wire-cutting safety control command is generated, which includes allowing wire cutting, suggesting adjusting the wire position, and prohibiting wire cutting. The step of dynamically adjusting the motion path and operating parameters of the working device based on the three-dimensional spatial structure information set, and executing the wire-cutting operation, includes: Based on the spatial position of the conductor and the initial pose of the knife edge in the three-dimensional spatial structure information set, the approach path of the knife edge is obtained; The approach path of the knife edge is dynamically adjusted to obtain the planned dynamic path. The dynamic adjustment includes using the trend of the spatial position change of the conductor within the historical time window to predict the current position and direction of movement of the conductor and correct the knife edge trajectory. The wire cutting speed is calculated based on the real-time minimum safe distance between the blade and the conductor, the mapping relationship between mechanical load, vibration data and wire cutting speed; Based on the planned dynamic path and wire cutting speed, the blade is controlled to move along the path to the wire cutting position, and the wire cutting action is initiated. When the blade contacts the wire and completes the cutting, the wire cutting completion signal is recorded.
2. The method of remote visualization monitoring of wire break operations according to claim 1, characterized in that, The three-dimensional reconstruction of the sheared area forms a three-dimensional spatial structure information set including the spatial position of the conductor and the relative pose of the cutting edge, including: Obtain the calibration parameters of each camera and transform them to the same global coordinate system; Based on the calibration parameters, key feature points are extracted from the multi-view image data, and feature matching is performed on the corresponding targets in different views to obtain feature point matching pairs. Based on the feature point matching pairs, establish epipolar constraints between the left and right views, and read the pixel coordinates of the matching points corresponding to the same target in the left and right views respectively; By matching the pixel coordinates of the points, the pixel offset is calculated, and a disparity matrix is formed. Depth calculations are performed on each pixel in the disparity matrix to generate three-dimensional depth values. These values are then rearranged into a two-dimensional depth matrix according to pixel positions. The matrix is then mapped onto the global coordinate system to generate an initial depth map. A point cloud dataset is generated from the initial depth map and then identified and segmented to generate a set of three-dimensional spatial structure information.
3. The method of remote visualization monitoring of wire break operations according to claim 2, characterized in that, The process of generating a point cloud dataset based on the initial depth map and performing recognition and segmentation to generate a three-dimensional spatial structure information set includes: Based on the initial depth map, the 3D depth values obtained from different perspectives are fused in the global coordinate system and transformed into a point cloud dataset. Key structural features are identified in the point cloud dataset, and then classified and segmented to obtain a structured point cloud. Based on structured point clouds, the key parameters of the three-dimensional spatial structure information set are solved, and the three-dimensional spatial structure information set is generated.
4. The method of remote visualization monitoring of wire break operations according to claim 3, characterized in that, The key structural features identified in the point cloud dataset include the conductor center trajectory, the knife edge plane, and the boundary features of the wire clamping mechanism, including: Spatial alignment is performed on the point cloud dataset. Based on the aligned point cloud dataset, noise reduction and density equalization are performed to obtain the point cloud density distribution characteristics. Spatial clustering is performed on the point cloud density distribution characteristics to divide the point cloud of the wire cutting region into candidate areas for conductors, candidate areas for knife edges, and candidate areas for wire clamping mechanisms. For the point cloud of the candidate traverse area, local linear trends are extracted and fitted to generate continuous traverse center trajectories; A cutting edge plane is generated from the point cloud of the candidate cutting edge region based on the cutting edge plane model. Based on the point cloud of the candidate area of the wire clamping device, the boundary features of the complete wire clamping mechanism are extracted.
5. The remote visual monitoring method for conductor breakage operations according to claim 1, characterized in that, The step of extracting key structural features of the work object and work equipment based on the three-dimensional spatial structural information set, and generating real-time spatial annotation information, includes: Map the three-dimensional coordinates of each key structural feature to a unified display coordinate system; Based on the mapped coordinates, the key structural features are projected onto the two-dimensional view of the PAD display to form the projection position; Based on the projection location, generate corresponding annotation symbols for each key structure; The spatial parameters of key structures are bound to annotation symbols to generate real-time spatial annotation information, which includes the three-dimensional spatial coordinates and motion state of the wire position, the relative pose of the knife edge, and the contact point of the wire clamp.
6. The remote visual monitoring method for conductor breakage operations according to claim 1, characterized in that, The method involves correlation analysis of mechanical load curves, cutting edge pressure fluctuations, and vibration characteristics based on trend characteristics, constructing a weighted causal relationship matrix between signals, including: According to the preset sampling time interval, various signals are converted into multi-channel time sequences; Based on multi-channel time series, the previous N time steps of the current time point are combined into a historical window sequence, which is used as the input of the attention mechanism; Based on the trend characteristics, calculate the sensitivity vector of each signal channel at the current time step; Obtain the second correlation data between each signal channel and construct an initial correlation matrix; Based on the initial correlation matrix, the peak time delay between different channels is analyzed to determine the dominant direction and potential causal relationship between mechanical load, blade pressure and vibration signal, and the time delay directionality characteristics are obtained. The initial causal weights are generated by weighting and summing the time-delay directional characteristics with the zero-delay Pearson correlation coefficient. The initial causal weights are corrected based on the sensitivity vector, and an initial causal weight matrix is generated.
7. A system using the remote visual monitoring method for conductor breakage operations as described in any one of claims 1-6, characterized in that, include: The 3D spatial structure information set construction module is used to collect multi-view image data and perform 3D reconstruction of the wire cutting area to form a 3D spatial structure information set including the spatial position of the conductor and the relative pose of the blade. The annotation information generation module is used to extract key structural features of the work object and work equipment based on the three-dimensional spatial structure information set, and generate real-time spatial annotation information. The state awareness module is used to acquire multi-source state data and perform correlation analysis to generate multi-dimensional state indication signals. The signal receiving and execution module is used to make safety decisions based on real-time spatial annotation information and multi-dimensional status indication signals, generate safety control commands, and dynamically adjust the motion path and operation parameters of the working device according to the three-dimensional spatial structure information set when the safety control command allows the operation, and execute the wire cutting operation.
Citation Information
Patent Citations
System and method for measuring unmanned aerial vehicle electrical line patrol on basis of images
CN102914294A
Polymer synthetic fiber rope breakage detecting device and detecting method
CN108442154A
