A Substation Equipment Installation Guidance Control Method and System Based on Semantic Awareness and EKF Algorithm

Abstract

This invention provides a substation equipment installation guidance and control method and system based on semantic perception and EKF algorithm, belonging to the field of substation equipment installation guidance and control technology. To address the technical problems of current substation equipment installation guidance methods relying on manual experience, resulting in poor installation accuracy, dependence on stringent auxiliary markers, and inability to adapt to unstructured installation environments, this invention preprocesses the raw point cloud data of the acquired equipment, including voxel downsampling and statistical outlier removal, filtering out dust and noise from the construction site. The unordered point cloud is classified into base, equipment, ground, or background, and then subjected to perception processing and semantic segmentation. In the segmented point cloud data, the flange plane is extracted using a random sampling consistency algorithm, and the bolt hole center coordinates are identified using a clustering algorithm. Coarse and fine registration strategies are employed for virtual-real registration operations. This invention is applied to substation equipment installation guidance.

Description

Technical Field

[0001] This invention provides a substation equipment installation guidance and control method and system based on semantic awareness and EKF algorithm, belonging to the field of substation equipment installation guidance and control technology. Background Technology

[0002] In substation construction, the "last mile" of equipment installation is a critical link that determines the construction quality and safety. The equipment to be installed is usually large in size and heavy in weight, and the installation accuracy requirements are high. Typically, the alignment error of the flange, bolt holes and base must be controlled within the millimeter level.

[0003] Currently, the most common installation method used on construction sites is a crane operator-assisted manual command mode. Ground command personnel guide the crane operator through hand gestures and walkie-talkies. However, this method has serious problems with two-way blind spots and perception gaps. When the crane operator is in the control room, he cannot observe the millimeter-level installation hole positions. When the command personnel approach the base (i.e., the "last meter"), their line of sight is blocked by the equipment itself, making it impossible to simultaneously observe the relative position of the bottom of the equipment and the base. At the same time, command relies on voice instructions (such as "a little to the left"), which have low information density and response delays. This can easily lead to the equipment repeatedly "testing" in the air or overshooting, which is not only inefficient but also increases the risk of collisions between expensive electrical equipment (such as porcelain bushings) and the base.

[0004] Currently, there is also a traditional optical / laser measurement installation mode based on manual markers. To improve accuracy, existing technologies have attempted to introduce total stations or industrial photogrammetry systems. These technologies typically rely on a "target-based registration" method, which requires attaching reflective markers or spherical targets to the surface of the object being measured so that algorithms can extract features and calculate pose. However, this installation mode is difficult to implement at substation construction sites, mainly because: the power equipment is stored in a yard before hoisting, making it difficult for personnel to attach markers to the bottom of the equipment, and the markers are very easy to fall off during hoisting. Or it may be obscured; at the same time, the lighting conditions at the substation site vary drastically (strong sunlight or shadow), and the surface texture of the power equipment is uniform (usually a uniform gray anti-corrosion paint), lacking significant texture features. Traditional computer vision algorithms based on color thresholds, edge detection, or SIFT / SURF feature points are prone to feature extraction failures or mismatches when processing such weak textures and unstructured point clouds. They cannot accurately segment key installation features such as "flanges" or "bolt holes" from cluttered construction backgrounds (such as scaffolding and ground debris), resulting in poor environmental adaptability.

[0005] In summary, existing installation technologies rely on vague human experience, which cannot guarantee accuracy, or they rely on stringent auxiliary markers, which cannot adapt to the complex unstructured environment of substations. There is an urgent need to develop an automated guided installation solution for large substation equipment, so as to achieve the goal of assisting equipment installation by directly "understanding" and accurately extracting the geometric semantic features of the equipment from the messy and weakly textured 3D point cloud without the need for manual labeling. Summary of the Invention

[0006] To address the technical problems existing in the background art, the present invention adopts the following technical solution: a substation equipment installation guidance control method based on semantic awareness and EKF algorithm, comprising the following control steps:

[0007] Step S1: Preprocess the raw point cloud data of the collected installation equipment, including voxel downsampling and statistical outlier removal, and filter out dust and noise at the construction site;

[0008] Step S2: Classify the disordered point cloud into base, equipment, ground or background, and perform perception processing and semantic segmentation processing. In the segmented point cloud data, use the random sampling consistency algorithm to extract the flange plane, and use the clustering algorithm to identify the center coordinates of the bolt holes.

[0009] Step S3: Calculate the rigid body transformation matrix distance from the current pose of the device to the theoretical target pose in BIM, and perform virtual-real space registration operation using coarse registration and fine registration strategies;

[0010] Step S4: Use a Kalman filter to process the time series of the virtual-real registration results, filter out jitter, track and predict the pose state at the next moment, and ensure the smoothness of the guidance command;

[0011] Step S5: Calculate the deviation between the current pose and the target pose. Decompose it into translation quantities and rotation amount It is then mapped to AR visual symbols or voice commands for guidance and feedback.

[0012] The specific method for step S2 is as follows:

[0013] Step S2.1: Perform hierarchical feature extraction, including:

[0014] Input point set The FPS algorithm is used to iteratively select a subset of points, so that the distance between each selected point and the set of selected points is maximized.

[0015] For each sampled center point, the algorithm operates at different radii. Within the sphere, a sphere query is performed, and multiple local point cloud neighborhoods are constructed based on multi-scale grouping;

[0016] For the center point Construct neighborhood set ;

[0017] Perform PointNet feature encoding:

[0018] Applying a multilayer perceptron (MLP) and max pooling operation to the point set within each group, local feature vectors are extracted, expressed as follows:

[0019] ;

[0020] in, It is an MLP function;

[0021] Step S2.2: After multiple layers of abstraction, the network maps high-dimensional features back to the original point cloud through the feature propagation layer, and finally outputs the class probability score for each point. The class definition expression is:

[0022] ;

[0023] We employ a weighted cross-entropy loss function, assigning higher weights to smaller sample classes to address the imbalanced sample problem.

[0024] The specific method for step S3 is as follows:

[0025] Step S3.1: Use the Fast Point Feature Histogram (FPFH) descriptor for points. and points in its neighborhood Construct a local coordinate system Calculate the angle of the normal vector deviation between the two. ;

[0026] Simplified Point Feature Histogram (SPFH): Statistically analyzes the angular distribution of all point pairs within a neighborhood;

[0027] Point The SPFH of the given point is weighted and fused with the SPFHs of its neighbors to generate the FPFH, expressed as:

[0028] ;

[0029] in, It is a point arrive Distance weights;

[0030] Step S3.2: After calculating the FPFH features of the source point cloud of the equipment and the target point cloud of the BIM base, the system uses the random sampling consensus algorithm to solve for the transformation matrix, including:

[0031] Feature matching: Search for the closest pair of points in the feature space based on the FPFH distance and establish a set of potential correspondences;

[0032] Random sampling: Randomly select 3 pairs of points from the corresponding set;

[0033] Model assumptions: The rigid body transformation matrices of these three point pairs are calculated using Singular Value Decomposition (SVD). ;

[0034] Verification: Applied to the source point cloud, it calculates the number of interior points that overlap with the target point cloud;

[0035] Iteration: Repeat the above process k times, selecting the one with the largest number of interior points. As ;

[0036] Step S3.3: Design an algorithm for the objective function, which minimizes the points in the source point cloud. To the corresponding point in the target point cloud The perpendicular distance to the tangent plane, and the formula for calculating the objective function are:

[0037] ;

[0038] in: For a point in the source cloud of the device; For the corresponding point in the target point cloud of the BIM base; For point Surface normal vector at that location; Let be the rotation matrix and translation vector to be solved;

[0039] Step S3.4: Due to the rotation matrix Including trigonometric functions, the objective function is nonlinear. To achieve real-time calculation, a small-angle approximation assumption is used. For rotation matrix By performing a linearization approximation, the nonlinear least squares problem is transformed into a system of linear equations. The Gauss-Newton method can be used to solve this problem quickly.

[0040] The specific method for step S4 is as follows:

[0041] Step S4.1: Define the state vector:

[0042] First, model the hovering device as a rigid body and define its position... State vector at time step For a 13-dimensional vector, the expression is:

[0043] ;

[0044] in:

[0045] These are three-dimensional position coordinates;

[0046] It represents the three-dimensional linear velocity;

[0047] The unit quaternion for the three-dimensional pose;

[0048] It is a three-dimensional angular velocity;

[0049] Step S4.2: Motion Model and Prediction:

[0050] Assuming a very short time interval Inside, the hoisting equipment moves at a constant speed, and its position is predicted using the following expression:

[0051] ;

[0052] Attitude prediction is performed using quaternion differential equations to update the attitude, expressed as:

[0053] ;

[0054] in, Represents quaternion multiplication;

[0055] For covariance prediction, the expression is:

[0056] ;

[0057] in, This is the process noise matrix, used to simulate uncertainties caused by wind or crane vibration;

[0058] Step S4.3: Measurement Update:

[0059] When the virtual-real registration module outputs new pose observation values At that time, the EKF algorithm is used to perform the update step, including:

[0060] The Kalman gain is calculated using the following formula:

[0061] ;

[0062] in, It is the measurement noise covariance matrix;

[0063] When the ICP registration has a low fit, the point cloud quality is poor, so it is automatically increased. The value of makes the filter trust the predictions of the physical model more than the unreliable observations, thus preventing the guidance command from jumping.

[0064] To perform state correction, the expression is:

[0065] ;

[0066] Through the EKF algorithm, the system can output smooth pose data at a frequency higher than the sensor frame rate, and maintain guidance by relying on the prediction model.

[0067] The control system used to implement the substation equipment installation guidance control method based on semantic awareness and EKF algorithm includes:

[0068] Mobile perception subsystem: It consists of a quadruped robot equipped with a high-precision LiDAR and a depth camera, configured to autonomously navigate to the blind spots of the construction site and collect real-time environmental point cloud data;

[0069] Semantic computation and guidance server: It is equipped with a deep learning inference engine to run the PointNet++ semantic segmentation model to remove environmental noise, and a registration and tracking engine to perform FPFH coarse registration, Point-to-Plane ICP fine registration and EKF state estimation, and output high-frequency 6D pose deviation data.

[0070] Human-machine collaboration terminal: Includes AR smart glasses and bone conduction headphones, configured to receive deviation data generated by semantic computing and guidance server, and overlay and display "theoretical base ghost model" and real-time guidance instructions in the physical field of view to achieve virtual-real fusion auxiliary operation.

[0071] The advantages of this invention compared to existing technologies are as follows: This invention provides a substation equipment installation guidance and control method based on semantic perception and EKF algorithm. It adopts a deep learning semantic segmentation algorithm to directly process unstructured point cloud data, effectively improving the robustness of recognition in unstructured environments. At the same time, it adopts a coarse-to-fine virtual-real registration strategy, combining FPFH global feature matching and point-to-surface iterative nearest point fine registration algorithm, effectively eliminating civil construction errors and cumulative measurement errors. This invention also uses the extended Kalman filter (EKF) algorithm to construct a 13-dimensional state space model, realizing real-time prediction and smoothing of the 6D pose of hovering equipment, enabling the system to output continuous and smooth pose data, achieving stable tracking of dynamically moving targets, and converting the calculated deterministic 6D deviation data into intuitive AR visualization commands, eliminating uncertainties in human-machine collaboration. Attached Figure Description

[0072] The present invention will be further described below with reference to the accompanying drawings:

[0073] Figure 1This is a flowchart illustrating the steps of the substation equipment installation guidance and control method based on semantic awareness and EKF algorithm of the present invention.

[0074] Figure 2 This is a flowchart of the data processing steps in an embodiment of the present invention. Detailed Implementation

[0075] like Figure 1 and Figure 2 As shown, this invention provides a substation equipment installation guidance and control method based on semantic perception and EKF algorithm, involving the application of artificial intelligence and robotics in substation engineering construction. Specifically, it proposes an equipment installation guidance method that can be applied to the on-site installation of large equipment such as transformer accessories, GIS, and reactors in substations.

[0076] This invention mainly adopts a high-precision guided installation method for substation equipment based on semantic perception. This method uses a mobile sensing terminal to collect the original three-dimensional point cloud flow of the construction site and introduces a semantic scene understanding mechanism based on deep learning: the original point cloud is input into a pre-trained hierarchical point cloud neural network (such as PointNet++), and geometric features are extracted through farthest point sampling (FPS) and multi-scale grouping (MSG) to accurately segment the disordered point cloud into "the set of points of the equipment to be installed", "the set of points of the installation base", and "the set of points of environmental noise". Then, "coarse to fine" virtual-real space registration is performed: first, the fast point feature histogram (FPFH) of the equipment point set and the base point set is calculated to achieve initial global alignment, and then the point-to-plane iterative nearest point (Point-to-Plane ICP) algorithm is used to calculate the precise pose of the equipment relative to the base, thereby obtaining the millisecond-level real-time six degrees of freedom (6D) pose deviation.

[0077] This invention also employs a dynamic pose tracking and command generation method based on Extended Kalman Filter (EKF). Addressing the nonlinear dynamic disturbances during hoisting, this method constructs a 13-dimensional state vector containing position, velocity, quaternion attitude, and angular velocity. The EKF algorithm is then used to perform temporal fusion and smoothing prediction on the pose data based on semantic perception output. Based on the smoothed 6D pose output by the EKF algorithm, the system calculates in real-time the deviation vector between the current position of the equipment and the theoretical target position in the BIM model. This vector is then mapped to augmented reality (AR) visualization symbols (such as dynamic guide arrows and virtual overlap color blocks) and natural language voice commands (such as "adjust 10mm to the south"), which are then fed back to on-site personnel in real-time through AR smart glasses and an audio system.

[0078] Furthermore, to implement the above-mentioned substation equipment installation guidance and control method, the present invention also provides a corresponding substation equipment installation collaborative guidance and control system, including:

[0079] Mobile perception subsystem: It consists of a quadruped robot equipped with a high-precision LiDAR and a depth camera, configured to autonomously navigate to the blind spot of human vision and collect real-time environmental point cloud data;

[0080] Semantic computation and guidance server: It is equipped with a deep learning inference engine to run the PointNet++ semantic segmentation model to remove environmental noise; and a registration and tracking engine to perform FPFH coarse registration, Point-to-Plane ICP fine registration and EKF state estimation, and output high-frequency 6D pose deviation data.

[0081] Human-machine collaborative terminal: Includes AR smart glasses and bone conduction headphones, configured to receive deviation data generated by the server, and overlay and display the "theoretical base ghost model" and real-time guidance instructions in the physical field of vision to achieve virtual-real fusion auxiliary operation.

[0082] The substation equipment installation guidance and control system provided by this invention is designed as a closed-loop feedback control system, consisting of a perception layer, a computing layer, and an interaction layer. It can transform unstructured environmental perception data (point cloud) into structured engineering semantic information (BIM model deviation), and further into deterministic control commands. This is mainly achieved using the following algorithm modules:

[0083] The deep learning-based semantic scene understanding module utilizes a mobile sensing terminal (quadruped robot) in the perception layer. Equipped with a high-precision LiDAR and RGB-D depth camera, it enters blind spots to collect point cloud data, including real-time 3D point cloud streams. Using spatial coordinates (x, y, z) and reflection intensity I, and deep neural networks such as PointNet++, key installation features of components such as flanges and bolt holes are accurately segmented from the messy point cloud of the construction site, solving the problem of target recognition in complex backgrounds;

[0084] The coarse-to-fine registration module combines global feature matching (FPFH) with point-to-plane integrated circuit (ICP) to achieve millimeter-level alignment between the BIM theoretical model and on-site measured data, eliminating construction errors. The BIM database stores pre-designed IFC format models, including precise geometric parameters and theoretical coordinates T for the target and source equipment. design ;

[0085] The Dynamic 6D Pose Tracking module based on Kalman filtering introduces the Extended Kalman Filter (EKF) algorithm to construct a state space model, which smoothly predicts and estimates the pose of the hovering device, overcoming the effects of sensor noise and dynamic occlusion.

[0086] The substation equipment installation guidance and control method adopted in this invention follows the logical steps of "sensing-understanding-registration-tracking-feedback", mainly including:

[0087] Step S1: Initialization and Data Loading

[0088] The preprocessing module is used to process the raw point cloud data of the installed equipment, including voxel grid downsampling and statistical outlier removal (SOR), to filter out dust and noise at the construction site.

[0089] Step S2: Perform further perceptual processing and semantic segmentation on the processed data:

[0090] Specifically, the disordered point cloud is classified into base, equipment, ground, or background. This step is a prerequisite for subsequent high-precision calculations and prevents the algorithm from mistaking background noise for installation features.

[0091] Feature extraction and fitting module: In the segmented point cloud data, the RANSAC (Random Sample Consensus) algorithm is used to extract the flange plane, and the clustering algorithm is used to identify the center coordinates of the bolt holes.

[0092] Step S3: Calculate the rigid body transformation matrix from the current pose of the device to the theoretical target pose in BIM. To balance speed and accuracy, coarse and fine registration strategies are used for virtual-real registration operations.

[0093] Step S4: Use a Kalman filter to process the time series of the registration results, filter out jitter, track and predict the pose state at the next moment, and ensure the smoothness of the guidance command.

[0094] Step S5: Calculate the deviation between the current pose and the target pose. Decompose it into translation quantities and rotation amount It is then mapped to AR visual symbols or voice commands for guidance and feedback.

[0095] Furthermore, in traditional industrial measurement, it is usually necessary to manually affix reflective markers. However, at substation construction sites, the equipment surface may be covered with oil and protective film, and it is difficult for personnel to carry out labeling operations. Therefore, step S2 specifically adopts markerless AI semantic perception technology to realize the semantic perception of industrial components through deep learning, including the application of PointNet++ network architecture and semantic segmentation and classification.

[0096] To identify the equipment to be installed and the mounting base from a cluttered point cloud, this invention employs the PointNet++ network architecture. Unlike traditional methods that convert point clouds into volumetric pixels, PointNet++ directly processes unstructured point clouds, better preserving the geometric details of the power equipment. Specifically, it includes:

[0097] Step S2.1: Hierarchical Feature Extraction:

[0098] The core of this network lies in the "aggregate abstraction layer," which consists of three key steps and can effectively address the problem of uneven point cloud density at construction sites (e.g., dense point clouds around nearby bases and sparse point clouds around distant equipment).

[0099] Farthest Point Sampling (FPS):

[0100] Input point set The FPS algorithm iteratively selects a subset of points to maximize the distance between each selected point and the set of selected points.

[0101] Technical effect: This ensures that the sampling points uniformly cover the entire surface of the device, and can capture the overall skeletal structure of the device even in the case of missing or obscured data, which is superior to random sampling.

[0102] Multi-scale grouping (MSG):

[0103] For each sampled center point, the algorithm operates at different radii. BallQuery is performed to construct multiple local point cloud neighborhoods.

[0104] Mathematical description: For the center point Construct neighborhood set For bolt holes on flanges (small-scale features) and equipment bodies (large-scale features), MSG can extract both micro and macro geometric features simultaneously, greatly improving the recognition rate of subtle installation features.

[0105] PointNet feature encoding:

[0106] For each group of points, apply multilayer perceptron (MLP) and max pooling operations to extract local feature vectors.

[0107] Formula expression: ,in It is an MLP function.

[0108] Step S2.2: After multiple layers of abstraction, the network maps high-dimensional features back to the original point cloud through the Feature Propagation layer, and finally outputs the class probability score of each point.

[0109] The category is defined as:

[0110] .

[0111] Loss function: The weighted cross-entropy loss function is used to give higher weights to smaller sample classes (such as bolt hole points) in order to solve the sample imbalance problem.

[0112] Furthermore, after identifying key components in step S2, it is necessary to accurately align the point cloud of the physical world with the BIM model of the digital world. Since the equipment may be far from the target location and have large attitude differences in the early stage of hoisting, directly using a fine registration algorithm (such as ICP) is prone to getting trapped in a local optimum. Therefore, in step S3, this invention designs a coarse-to-fine virtual-physical space registration scheme. The coarse registration adopts global alignment based on FPFH features, with the goal of calculating an initial transformation matrix. The device point cloud is roughly transformed to the vicinity of the base. After obtaining the initial attitude, fine registration is performed using an improved point-to-plane ICP algorithm for millimeter-level fine adjustment. Since traditional point-to-point ICP is prone to slippage in the tangential direction when facing flat surfaces (such as flange planes), resulting in slow convergence speed, this application adopts a point-to-plane variant, which can significantly improve the registration accuracy for industrial parts.

[0113] The coarse-to-fine virtual-real space registration scheme specifically includes:

[0114] Step S3.1: Use the Fast Point Feature Histogram (FPFH) descriptor. FPFH is a mathematical tool for describing the local geometric properties of point clouds. It has rotation invariance and is very suitable for handling arbitrary rotational postures of equipment during hoisting.

[0115] The calculation logic is as follows: for a point and points in its neighborhood Construct a local coordinate system Calculate the angle of the normal vector deviation between the two. .

[0116] Simplified Point Feature Histogram (SPFH): Statistically analyzes the angular distribution of all point pairs within a neighborhood.

[0117] FPFH generation: [Point] The SPFH of the given point is weighted and fused with the SPFH of its neighbors:

[0118] ;

[0119] in, It is a point arrive Distance weights.

[0120] Step S3.2: RANSAC-based correspondence matching:

[0121] After calculating the FPFH features of the source point cloud (equipment) and the target point cloud (BIM base), the system uses the RANSAC (Random Sample Consensus) algorithm to solve for the transformation matrix, including:

[0122] Feature matching: Search for the closest pair of points in the feature space based on the FPFH distance and establish a set of potential correspondences;

[0123] Random sampling: Randomly select 3 pairs of points from the corresponding set;

[0124] Model assumptions: The rigid body transformation matrices of these three point pairs will be calculated using SVD (Singular Value Decomposition). ;

[0125] Verification: Applied to the source point cloud, it calculates the number of inliers that overlap with the target point cloud;

[0126] Iteration: Repeat the above process k times (e.g., 1000 times), selecting the one with the largest number of interior points. As .

[0127] This step can tolerate a large number of mismatches and can accurately find the initial posture of the equipment even if there are obstructions or interference from temporary scaffolding on the construction site.

[0128] Step S3.3: Objective function design. This algorithm minimizes the points in the source point cloud. To the corresponding point in the target point cloud The perpendicular distance to the tangent plane, not the Euclidean distance.

[0129] The formula for calculating the objective function is:

[0130] ;

[0131] in: For a single point in the source point cloud (device); For the corresponding point in the target point cloud (BIM base); For point Surface normal vector at that location; Let be the rotation matrix and translation vector to be solved.

[0132] Step S3.4: Linearization and Iteration:

[0133] Due to the rotation matrix Including trigonometric functions, the objective function described above is nonlinear. To achieve real-time computation, it is assumed that the rotation angle in each iteration is small, utilizing the small-angle approximation assumption. right By performing a linearization approximation, the nonlinear least squares problem is transformed into a system of linear equations. The Gauss-Newton method can be used to solve this problem quickly.

[0134] Point-to-surface ICP utilizes the normal vector information of the flange to quickly lock the deviation of the equipment in the vertical direction (Z-axis) and tilt angle (Pitch / Roll), which is crucial for bolt hole alignment.

[0135] Furthermore, during the hoisting process, sensor data often contains noise and may be temporarily obstructed (e.g., a worker passing by and blocking the robot's line of sight). To ensure the continuity and smoothness of guidance commands, the system cannot rely solely on the registration results of a single frame, but must introduce a temporal filtering algorithm. In step S4 of this invention, an extended Kalman filter (EKF) is used to achieve 6D pose dynamic tracking, specifically including:

[0136] Step S4.1: Define the state vector:

[0137] First, model the hovering device as a rigid body and define its position... State vector at time step A 13-dimensional vector:

[0138] ;

[0139] in:

[0140] : Three-dimensional position coordinates;

[0141] : Three-dimensional linear velocity;

[0142] : A unit quaternion representing three-dimensional attitude. Compared to Euler angles, quaternions avoid the gimbal lock problem and are more suitable for describing omnidirectional rotation;

[0143] : Three-dimensional angular velocity.

[0144] Step S4.2: Motion Model and Prediction (Prediction Step):

[0145] Assuming a very short time interval Inside, the hoisting equipment moves at a constant velocity (Constant Velocity Model).

[0146] Perform location prediction:

[0147] ;

[0148] Attitude prediction: Updating attitude using quaternion differential equations:

[0149] ;

[0150] in, This represents quaternion multiplication.

[0151] Perform covariance prediction: ,in This is the process noise matrix, used to simulate uncertainties caused by wind or crane vibration.

[0152] Step S4.3: Measurement Update:

[0153] When the virtual-real registration module outputs new pose observation values At that time, the EKF algorithm is used to perform the update step:

[0154] Calculate the Kalman gain:

[0155] ;

[0156] in, It is the measurement noise covariance matrix.

[0157] When the ICP registration fit score is low (indicating poor point cloud quality), it is automatically increased. The value of makes the filter trust the predictions of the physical model more than unreliable observations, thus preventing the bootstrap command from jumping.

[0158] Perform state correction:

[0159] ;

[0160] Through the EKF algorithm, the system can output smooth pose data at a frequency higher than the sensor frame rate (e.g., 50Hz), and can maintain guidance by relying on the prediction model even when the sensor data is temporarily interrupted (<1 second).

[0161] This invention employs a deep learning semantic segmentation algorithm based on PointNet++ to directly process unstructured point cloud data, rather than converting it into images or voxels. This effectively improves the robustness of recognition in unstructured environments, enabling the system to overcome interference factors such as drastic changes in lighting at substation construction sites, uniform equipment surface textures (e.g., gray anti-corrosion paint), and oil stains. Compared to traditional machine vision technologies that rely on color or edge features, this invention can accurately extract key installation features such as flanges and bolt holes from cluttered backgrounds (e.g., scaffolding, ground debris), solving the problem that existing measurement methods cannot accurately identify or find targets under harsh conditions.

[0162] This invention designs a coarse-to-fine virtual-real registration strategy, combining FPFH global feature matching and Point-to-Plane ICP (point-to-plane iterative nearest point) fine registration algorithm. This scheme uses the BIM model as the sole ground truth, effectively eliminating civil construction errors and cumulative measurement errors. Compared with the centimeter-level errors estimated by human experience, this invention can stably achieve millimeter-level (<5mm) installation accuracy, ensuring that the equipment is accurately placed in one go.

[0163] This invention introduces an extended Kalman filter (EKF) to construct a 13-dimensional state space model, enabling real-time prediction and smoothing of the 6D pose of hovering equipment. This solves the problem of irregular swaying (micro-motion) of equipment caused by wind load and wire rope elasticity during hoisting, as well as the problem of temporary sensor obstruction caused by personnel movement. The system can output continuous and smooth pose data, avoiding the left and right fluctuations of guidance commands due to measurement value jumps, and achieving stable tracking of dynamically moving targets, which is something that traditional static measurement tools (such as total stations) cannot do.

[0164] Furthermore, this invention utilizes a quadruped robot equipped with sensors to penetrate into blind spots of human vision and transforms the calculated deterministic 6D deviation data into intuitive AR visualization commands. This eliminates the need for commanders to risk approaching the hovering heavy object for observation, completely eliminating the safety hazards of falling objects and collisions. It also upgrades traditional "vague language commands" (such as "move a little to the left") to "digital precision guidance" (such as "move 15mm to the left"), eliminating ambiguity and delays in human-machine communication and transforming time-consuming "repeated probing" into efficient "one-time guidance," significantly shortening the hoisting cycle of large equipment.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A substation equipment installation guide control method based on semantic perception and EKF algorithm, characterized in that, The control steps include the following: Step S1: Preprocess the raw point cloud data of the collected installation equipment, including voxel downsampling and statistical outlier removal, and filter out dust and noise at the construction site; Step S2: Classify the disordered point cloud into base, equipment, ground or background, and perform perception processing and semantic segmentation processing. In the segmented point cloud data, use the random sampling consistency algorithm to extract the flange plane, and use the clustering algorithm to identify the center coordinates of the bolt holes. Step S3: Calculate the rigid body transformation matrix distance from the current pose of the device to the theoretical target pose in BIM, and perform virtual-real space registration operation using coarse registration and fine registration strategies; Step S4: Use a Kalman filter to process the time series of the virtual-real registration results, filter out jitter, track and predict the pose state at the next moment, and ensure the smoothness of the guidance command; Step S5: Calculate the deviation value of the current pose and the target pose , decompose it into translation and rotation , and map it to AR visualization symbols or voice instructions for guidance feedback.

2. The substation equipment installation guidance control method based on semantic awareness and EKF algorithm according to claim 1, characterized in that, The specific method for step S2 is as follows: Step S2.1: Perform hierarchical feature extraction, including: Input point set The FPS algorithm is used to iteratively select a subset of points, so that the distance between each selected point and the set of selected points is maximized. For each sampled center point, the algorithm operates at different radii. Within the sphere, a sphere query is performed, and multiple local point cloud neighborhoods are constructed based on multi-scale grouping; For the center point Construct neighborhood set ; Perform PointNet feature encoding: Applying a multilayer perceptron (MLP) and max pooling operation to the point set within each group, local feature vectors are extracted, expressed as follows: ； in, It is an MLP function; Step S2.2: After multiple layers of abstraction, the network maps high-dimensional features back to the original point cloud through the feature propagation layer, and finally outputs the class probability score for each point. The class definition expression is: ； We employ a weighted cross-entropy loss function, assigning higher weights to smaller sample classes to address the imbalanced sample problem.

3. The substation equipment installation guidance control method based on semantic awareness and EKF algorithm according to claim 2, characterized in that, The specific method for step S3 is as follows: Step S3.1: Use the Fast Point Feature Histogram (FPFH) descriptor for points. and points in its neighborhood Construct a local coordinate system Calculate the angle of the normal vector deviation between the two. ; Simplified Point Feature Histogram (SPFH): Statistically analyzes the angular distribution of all point pairs within a neighborhood; Point The SPFH of a given point is weighted and fused with the SPFHs of its neighbors to generate the FPFH, expressed as: ； in, It is a point arrive Distance weights; Step S3.2: After calculating the FPFH features of the source point cloud of the equipment and the target point cloud of the BIM base, the system uses the random sampling consensus algorithm to solve for the transformation matrix, including: Feature matching: Search for the closest pair of points in the feature space based on the FPFH distance and establish a set of potential correspondences; Random sampling: Randomly select 3 pairs of points from the corresponding set; Model assumptions: The rigid body transformation matrices of these three point pairs are calculated using Singular Value Decomposition (SVD). ; Verification: Applied to the source point cloud, it calculates the number of interior points that overlap with the target point cloud; Iteration: Repeat the above process k times, selecting the one with the largest number of interior points. As ; Step S3.3: Design an algorithm for the objective function, which minimizes the points in the source point cloud. To the corresponding point in the target point cloud The perpendicular distance to the tangent plane, and the formula for calculating the objective function are: ； in: For a point in the source cloud of the device; For the corresponding point in the target point cloud of the BIM base; For point Surface normal vector at that location; Let be the rotation matrix and translation vector to be solved; Step S3.4: Due to the rotation matrix Including trigonometric functions, the objective function is nonlinear. To achieve real-time calculation, a small-angle approximation assumption is used. For rotation matrix By performing a linearization approximation, the nonlinear least squares problem is transformed into a system of linear equations. The Gauss-Newton method can be used to solve this problem quickly.

4. The substation equipment installation guidance control method based on semantic awareness and EKF algorithm according to claim 3, characterized in that, The specific method for step S4 is as follows: Step S4.1: Define the state vector: First, model the hovering device as a rigid body and define its position... State vector at time step For a 13-dimensional vector, the expression is: ； in: These are three-dimensional position coordinates; It represents the three-dimensional linear velocity; The unit quaternion for the three-dimensional pose; It is a three-dimensional angular velocity; Step S4.2: Motion Model and Prediction: Assuming a very short time interval Inside, the hoisting equipment moves at a constant speed, and its position is predicted using the following expression: ； Attitude prediction is performed using quaternion differential equations to update the attitude, expressed as: ； in, Represents quaternion multiplication; For covariance prediction, the expression is: ； in, This is the process noise matrix, used to simulate uncertainties caused by wind or crane vibration; Step S4.3: Measurement Update: When the virtual-real registration module outputs new pose observation values At that time, the EKF algorithm is used to perform the update step, including: The Kalman gain is calculated using the following formula: ； in, It is the measurement noise covariance matrix; When the ICP registration has a low fit, the point cloud quality is poor, so it is automatically increased. The value of makes the filter trust the predictions of the physical model more than the unreliable observations, thus preventing the guidance command from jumping. To perform state correction, the expression is: ； Through the EKF algorithm, the system can output smooth pose data at a frequency higher than the sensor frame rate, and maintain guidance by relying on the prediction model.

5. The control system used to implement the substation equipment installation guidance control method based on semantic perception and EKF algorithm as described in claim 4, characterized in that, include: Mobile perception subsystem: It consists of a quadruped robot equipped with a high-precision LiDAR and a depth camera, configured to autonomously navigate to the blind spots of the construction site and collect real-time environmental point cloud data; Semantic computation and guidance server: It is equipped with a deep learning inference engine to run the PointNet++ semantic segmentation model to remove environmental noise, and a registration and tracking engine to perform FPFH coarse registration, Point-to-Plane ICP fine registration and EKF state estimation, and output high-frequency 6D pose deviation data. Human-machine collaboration terminal: Includes AR smart glasses and bone conduction headphones, configured to receive deviation data generated by semantic computing and guidance server, and overlay and display "theoretical base ghost model" and real-time guidance instructions in the physical field of view to achieve virtual-real fusion auxiliary operation.

Images