Method for intelligent management and control of special vehicle operation process based on virtual space

Abstract

This invention discloses a method for intelligent control of special vehicle operation processes based on virtual space. It utilizes only external sensing devices—a lidar and an optical camera with overlapping detection fields deployed at the special vehicle operation site—to synchronously collect operational environment data. Based on this data, a three-dimensional virtual space containing semantic information of electrical equipment is constructed, and a virtual electronic fence is set up to form a safety zone. Based on real-time lidar point cloud data, point cloud clusters of different rigid components of the special vehicle are identified through segmentation and clustering to generate a theoretical three-dimensional contour. Based on optical camera image data, two-dimensional edges of vehicle components are extracted through semantic segmentation to generate a three-dimensional semantic contour. The theoretical three-dimensional contour and the three-dimensional semantic contour are fused to obtain a high-precision dynamic three-dimensional contour of the moving parts of the special vehicle. The shortest distance between the high-precision dynamic three-dimensional contour and the boundary of the virtual electronic fence is calculated as a benchmark for implementing graded early warnings or operational interventions on the special vehicle.

Description

Technical Field

[0001] This invention relates to the field of substation maintenance and construction management technology, and in particular to a method for intelligent control of special vehicle operation processes based on virtual space. Background Technology

[0002] The core requirement for safety management of special vehicles (such as cranes and aerial work platforms) working near electrical equipment at substation maintenance sites is to achieve accurate monitoring, real-time early warning, and risk prevention of the safe distance between dynamic working equipment and live equipment. However, existing technologies are limited by their design principles and scenario adaptability, and still have many unavoidable technical shortcomings, resulting in insufficient reliability, accuracy, and practicality of safety management.

[0003] From the perspective of protection dimensions and dynamic adaptability, existing mainstream technologies mostly focus on two-dimensional protection in the horizontal plane. For example, traditional two-dimensional laser fences construct planar warning areas through fan-shaped scanning, which can only detect horizontal object intrusions. They lack effective detection capabilities for vertical movements or complex contour changes such as the lifting and rotating of crane booms, and are prone to missed detections due to overlooking vertical risk points. At the same time, most protection systems adopt static protection logic, with fixed protection area boundaries. They cannot dynamically adapt to the movement of special vehicles or the posture of operating equipment. When the boom extension trajectory exceeds the preset static area, a protection vacuum or false alarm may occur. In addition, some point detection technologies (such as ultrasonic and infrared sensors) can only monitor distance data in a single direction and at a single point, and cannot construct a comprehensive three-dimensional safety domain. The protection range has obvious blind spots and it is difficult to cover the entire trajectory of key risk parts such as the boom end and lifting components.

[0004] In terms of target recognition and anti-interference capabilities, existing technologies generally lack effective target classification and scene understanding capabilities. For example, while video analytics-based monitoring systems can achieve preliminary target detection, they struggle to accurately distinguish between operation-related targets such as special vehicles and lifting equipment, and interfering objects such as birds, falling objects, and passing personnel. This results in a false alarm rate often exceeding 5%. Frequent false alarms not only reduce the trust of on-site personnel in warning information but may also disrupt normal operational processes. Furthermore, existing sensors exhibit extremely poor environmental adaptability: video analytics relies on lighting conditions, and feature extraction is prone to failure in strong light, backlight, nighttime, or rainy / foggy weather; ultrasonic sensors are significantly affected by temperature and airflow, while infrared sensors are easily interfered with by dust and rain / fog. These environmental factors directly lead to distorted ranging data. Moreover, obstructions, shadows, and similar-colored backgrounds in complex operational scenarios further exacerbate false alarms and missed alarms, especially in the unstructured environment of substations with multiple devices and structures, where existing technologies struggle to operate stably.

[0005] Insufficient accuracy and 3D modeling capabilities are key bottlenecks restricting the effectiveness of safety management. Existing positioning and ranging technologies generally fall short of the centimeter-level accuracy requirements for near-electrical work: UWB positioning systems typically have an accuracy of 10-30cm, while ultrasonic and infrared sensors are mostly at the decimeter level, neither of which can accurately determine safe distances. Furthermore, most technologies can only acquire the center point location data of the target, failing to accurately perceive the complex contours of irregular objects such as booms and lifting components, leading to deviations in the calculation of the actual minimum safe distance and creating collision risks. In addition, 3D reconstruction technologies such as monocular / binocular vision are insufficient in the complex, unstructured environment of substations due to equipment occlusion and texture loss, resulting in inadequate reconstruction accuracy and robustness. This makes it difficult to construct accurate 3D models of the work scene, failing to provide reliable spatial references for safe distance calculations and further limiting the accuracy of distance monitoring.

[0006] From the perspectives of deployment practicality and cost control, some existing high-precision systems suffer from complex deployment and poor stability. For example, some vehicle-mounted collision avoidance systems require the installation of tags or sensors on target objects such as crane booms and vehicle bodies, which not only increases the workload of pre-construction preparation but may also affect data transmission due to equipment vibration and electromagnetic interference. The base station deployment of wireless positioning systems such as UWB and Bluetooth is susceptible to the multipath effect and electromagnetic interference of metal structures and high-voltage equipment within substations, leading to doubts about positioning stability. At the same time, existing high-precision, large-scale control systems often rely on complex hardware combinations and algorithm support, resulting in high costs and making it difficult to promote and apply them on a large scale in substation operation scenarios of different voltage levels and sizes, thus limiting the universality of the technology. Summary of the Invention

[0007] In view of the shortcomings and deficiencies of existing technologies, such as limitations of two-dimensional protection, poor adaptability of static protection, high false alarm and false alarm rates, reliance on vehicle-mounted sensors or pre-built three-dimensional models for deployment, and insufficient ranging accuracy, this invention provides a real-time intelligent control method and system for special vehicles operating near electric fields based on virtual space.

[0008] This method does not require the installation of any additional sensors or tags on special vehicles. It only uses a three-dimensional lidar and an optical camera, which are rigidly fixed to the same base and have overlapping detection fields, deployed at the maintenance site of substations with voltage levels of 500kV and above, to simultaneously collect work environment data. After joint fusion calibration, the coordinate systems of the two are unified into a dynamically defined world coordinate system. Combined with point cloud preprocessing and image pixel-level semantic segmentation, a three-dimensional virtual space containing semantic information of the electrical equipment is generated, and a virtual electronic fence is set up in this space to form a safety zone.

[0009] For multi-joint moving parts of special vehicles, point cloud clusters of rigid parts are identified by segmentation and clustering. The joint rotation angle is calculated and a parametric kinematic model is driven to generate a theoretical 3D contour. At the same time, the 2D edges of the parts are extracted based on optical camera images, and the 3D semantic contour is generated by back projection combined with the depth information of LiDAR. The two are fused to obtain a high-precision dynamic 3D contour. Simultaneously, dynamic occlusion is handled by motion state prediction, target re-identification and trajectory interpolation strategies to ensure tracking continuity.

[0010] Based on the spatial index structure, a fast nearest neighbor search is performed. A hierarchical ranging strategy is adopted according to the distance range between special vehicles and detection equipment to calculate the shortest distance between the dynamic three-dimensional contour and the boundary of the defense zone. Based on this distance, a graded early warning or operational intervention is triggered. The early warning is synchronously sent to the smart alarm wristband of the operator through low power wide area wireless communication. In the dangerous situation, after the countdown is started, the special vehicle operation loop is locked, forming a complete closed loop of "perception-modeling-tracking-ranging-early warning-intervention".

[0011] This invention breaks through the application limitations of traditional technologies by using pure external sensing high-precision contour modeling and dynamic protection design. While improving the accuracy of near-electric work safety distance monitoring and anti-interference capabilities, it simplifies the deployment process, adapts to the temporary control needs of complex unstructured working environments in substations, and effectively reduces the risk of collisions during near-electric work.

[0012] The present invention specifically adopts the following technical solution:

[0013] A method for intelligent control of special vehicle operation processes based on virtual space involves deploying lidar and optical cameras with overlapping detection fields at the special vehicle operation site as external sensing devices to synchronously collect operation environment data without relying on any additional sensors or tags installed on the special vehicles.

[0014] Based on the aforementioned work environment data, a three-dimensional virtual space containing semantic information of electrical equipment is constructed, and a virtual electronic fence is set up in the three-dimensional virtual space to form a safety zone;

[0015] Based on real-time point cloud data from LiDAR, point cloud clusters of different rigid components of special vehicles are identified through segmentation and clustering. The rotation angles of the joints connecting each component are calculated and input into a parametric kinematic model to generate the theoretical three-dimensional contours of the moving parts of the special vehicle. Simultaneously, based on optical camera image data, two-dimensional edges of vehicle components are extracted through semantic segmentation. Combined with LiDAR depth information, the two-dimensional edges are back-projected into three-dimensional space to generate three-dimensional semantic contours. The theoretical three-dimensional contours and the three-dimensional semantic contours are fused to obtain high-precision dynamic three-dimensional contours of the moving parts of the special vehicle.

[0016] The shortest distance between the high-precision dynamic three-dimensional profile and the boundary of the virtual electronic fence is calculated as a benchmark for implementing graded early warning or operational intervention on special vehicles.

[0017] Furthermore, the lidar and optical camera are deployed together through a rigid support structure. The position of the rigid support structure is determined according to the geometry of the special vehicle's operating area to ensure that the combined field of view of the lidar and optical camera covers the entire operating area and the boundary of the electrical equipment, and that the overlapping area of ​​their detection fields meets the requirements for jointly acquiring calibration objects. The lidar is a three-dimensional lidar, and the optical camera is a high-definition camera. The deployment height of the rigid support structure is adapted to the field of view requirements of the operating area to avoid obstruction.

[0018] Furthermore, when the lidar and optical camera synchronously collect operational environment data, hardware synchronization methods are used to ensure that the data collection time is consistent. The hardware synchronization methods include GPS PPS pulses or dedicated synchronizers. The main control unit maintains time consistency with the lidar and optical camera through a time synchronization protocol. The synchronization accuracy of the time synchronization protocol meets the requirement of data fusion without misalignment.

[0019] Furthermore, the construction of the three-dimensional virtual space containing semantic information of the electrical equipment specifically includes:

[0020] Preprocessing of the initial point cloud data acquired by lidar includes denoising, filtering, and segmentation;

[0021] The lidar and optical camera are jointly calibrated. The camera intrinsic parameter matrix, distortion coefficient and the transformation matrix from lidar to optical camera are obtained sequentially through the corner extraction function, camera calibration function and pose solving function of the image processing software. The coordinate systems of the two are unified into a dynamically defined world coordinate system. The origin of the world coordinate system is set as the geometric center of lidar, and the coordinate axis direction is oriented according to the operation scene.

[0022] The calibrated lidar point cloud is transformed to the world coordinate system, and semantic labels are assigned to the point cloud by combining the pixel-level semantic segmentation results of the optical camera image. The semantic labels include at least the background, special vehicles and electrical equipment categories. Then, the data volume is reduced by downsampling while preserving the spatial structure, generating a three-dimensional virtual space with semantic information.

[0023] Furthermore, the calculation of the rotation angle of the joints connecting each component is specifically as follows: based on the characteristics of the multi-rigid-body system of the special vehicle, the hinge position of each moving joint is virtually determined, and then the rotation angle of each joint is calculated in reverse by analyzing the geometric relationship of the point cloud clusters of adjacent components. The geometric relationship includes the center distance and attitude angle difference of the point cloud clusters of adjacent components.

[0024] The semantic segmentation to extract the two-dimensional edges of vehicle components uses a trained deep learning pixel-level semantic segmentation model, and the edge extraction uses an edge extraction operator that can achieve sub-pixel accuracy.

[0025] Furthermore, during the generation of high-precision dynamic 3D contours, a dynamic occlusion processing strategy is simultaneously employed to maintain the continuity of target tracking, specifically including:

[0026] Motion state prediction: Based on the filtering algorithm, a motion model is established for the moving parts of the tracked special vehicle. When the part is briefly occluded, the position of the part in the subsequent acquisition frame is predicted based on the motion parameters before occlusion.

[0027] Target re-identification: When the occluded part reappears in the detection field of view, the point cloud shape features, motion continuity and semantically segmented visual features of the part are compared with the predicted trajectory to restore continuous tracking;

[0028] Trajectory interpolation: If the occlusion time of a component meets a preset short time threshold, an interpolation algorithm is used to complete the trajectory data during the occlusion period to avoid abrupt changes in subsequent distance calculations.

[0029] Further, calculating the shortest distance between the high-precision dynamic three-dimensional contour and the boundary of the virtual electronic fence specifically includes:

[0030] The surface of the virtual electronic fence is discretized into a dense point cloud of preset density to form a defense zone boundary point cloud. A spatial index structure is constructed based on the defense zone boundary point cloud, and the preset density meets the distance calculation accuracy requirements.

[0031] Based on the distance range between the special vehicle and the lidar, different point cloud processing methods are used to calculate the distance. The closer the distance range, the more refined the point cloud processing method is, in order to improve the ranging accuracy.

[0032] A fast nearest neighbor search is performed using the spatial index structure to traverse all points of the high-precision dynamic 3D contour, find the minimum Euclidean distance to the point cloud of the defense zone boundary, and use this minimum Euclidean distance as the shortest distance.

[0033] Furthermore, the tiered early warning is achieved through wireless communication between the main control unit and the smart alarm bracelet. The wireless communication is a low-power wide-area communication method, and the smart alarm bracelet is worn by key personnel at the work site.

[0034] When the shortest distance reaches the preset warning threshold, the smart alarm bracelet triggers the first-level alarm response, including vibration reminder and information display;

[0035] When the shortest distance reaches the preset alarm threshold, the smart alarm bracelet triggers a second-level alarm response. The intensity of the second-level alarm response is higher than that of the first-level alarm response, and an on-site audible and visual alarm is triggered simultaneously.

[0036] The wireless communication uses a broadcast mode to ensure that all smart alarm bracelets respond synchronously, and supports an alarm command retransmission mechanism to improve reliability. Higher-level alarm commands cover the execution tasks of lower-level alarm commands.

[0037] Furthermore, the specific intervention in the operation of special vehicles is as follows: when the shortest distance reaches a preset danger threshold, a locking command is sent to the locking controller installed on the special vehicle through the main control unit, and the locking command triggers a countdown of a preset duration; if the special vehicle still does not stop the dangerous action after the countdown ends, the locking controller automatically locks the operation circuit of the special vehicle to prevent the dangerous action from continuing; at the same time, the collected data and decision data during the control process are recorded for subsequent traceability analysis.

[0038] Furthermore, the calculation of the rotation angle of the joint connecting each component specifically includes: segmenting the point cloud cluster of special vehicle components using the DBSCAN clustering algorithm, extracting the main direction vector of the component using principal component analysis, solving the rotation angle by vector cross product and dot product based on the virtual positioning results of the joint hinge point, and solving the pitch angle by the angle between the vector and the vertical direction; when the component is briefly occluded, the extended Kalman filter is used to predict the joint angle and its angular velocity.

[0039] Furthermore, the fusion of the theoretical 3D contour and the 3D semantic contour specifically includes: using the directed Hausdorff distance variant as the optimization objective, constructing optimization parameters that include joint angles, local translations, and scale fine-tuning; using the Levenberg-Marquardt algorithm to minimize the sum of squared residual distances from the semantic contour points to the theoretical contour; fusing the optimized theoretical contour and the semantic contour point cloud; and outputting a high-precision dynamic 3D contour.

[0040] And, a system for intelligent control of special vehicle operation processes based on virtual space, comprising:

[0041] External perception module, main control module, 3D modeling module, contour tracking module, and risk management module;

[0042] The external sensing module includes a lidar and an optical camera, both rigidly fixed to the same base and with overlapping detection fields. The external sensing module achieves synchronous data collection of the working environment only by being deployed at the special vehicle operation site, without relying on any additional sensors or tags installed on the special vehicle.

[0043] The 3D modeling module is communicatively connected to the external sensing module and is used to construct a 3D virtual space containing semantic information of electrical equipment based on the working environment data, and to set up a virtual electronic fence in the 3D virtual space to form a safety zone.

[0044] The contour tracking module is communicatively connected to the external perception module and the 3D modeling module. It is used to identify point cloud clusters of different rigid components of special vehicles based on real-time point cloud data from LiDAR, calculate the rotation angle of the joints connecting each component and input the parameterized kinematic model to generate a theoretical 3D contour. At the same time, it extracts the 2D edges of vehicle components based on optical camera image data and back-projects them to generate a 3D semantic contour. The theoretical 3D contour and the 3D semantic contour are fused to obtain a high-precision dynamic 3D contour of the moving parts of the special vehicle.

[0045] The risk management module is communicatively connected to the contour tracking module and the 3D modeling module, and is used to calculate the shortest distance between the high-precision dynamic 3D contour and the boundary of the virtual electronic fence, and to perform graded early warning or operational intervention on special vehicles based on the shortest distance;

[0046] The main control module is communicatively connected to and powered by the external perception module, 3D modeling module, contour tracking module, and risk management module, and is used to coordinate and control the collaborative work of each module.

[0047] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:

[0048] First, it significantly improves deployment convenience and scenario adaptability. There is no need to install any additional sensors or tags on special vehicles, nor is it necessary to pre-build a 3D model of the work site. Deployment can be completed simply by using lidar and optical cameras that can be quickly deployed on-site. This avoids the need to modify work vehicles and do a lot of preparatory work, and is more in line with the temporary and flexible control needs of substation near-electricity operations.

[0049] Secondly, it significantly improves the accuracy and reliability of safe distance monitoring. By fusing and optimizing the pure external kinematic chain model with semantic back projection contours, it can accurately capture the dynamic contours of multi-joint moving parts of special vehicles, breaking through the limitations of traditional two-dimensional protection, point detection, or coarse model representation. Combined with dynamic occlusion processing strategies and layered ranging logic, it effectively reduces the risk of false alarms and missed alarms caused by environmental interference and target occlusion, providing high-precision data support for safe distance determination.

[0050] Furthermore, it achieves a closed-loop safety management process, constructing a complete management and control link from environmental perception, 3D modeling, dynamic tracking, distance assessment to graded early warning and mandatory intervention. It not only ensures timely response of operators through multi-source synchronous alarms, but also automatically locks the operation of special vehicles in dangerous situations, avoiding management loopholes caused by relying solely on passive alarms, and comprehensively improving the safety assurance capability of near-electric work.

[0051] Finally, to enhance the system's stability and versatility in complex environments, through designs such as radar-visual joint calibration and low-power wide-area communication, it adapts to the multi-device, strong electromagnetic, and unstructured operating environment of substations. It can achieve high-precision control without complex hardware combinations, making it easy to promote and apply in substation operating scenarios of different voltage levels and scales. Attached Figure Description

[0052] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0053] Figure 1 This is a schematic diagram of the principle architecture of an embodiment of the present invention;

[0054] Figure 2 This is a deployment diagram of an embodiment of the present invention;

[0055] Figure 3 This is a flowchart illustrating an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram illustrating the definition of joint hinge points and angle parameters for special vehicles according to an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram illustrating the fusion and optimization of theoretical three-dimensional contours and semantic three-dimensional contours in an embodiment of the present invention. Detailed Implementation

[0058] In the following, specific embodiments of this application will be described in detail with reference to the accompanying drawings. Based on these detailed descriptions, those skilled in the art will be able to clearly understand and implement this application. Without departing from the principles of this application, features from various embodiments can be combined to obtain new implementations, or certain features from some embodiments can be substituted to obtain other preferred implementations.

[0059] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings:

[0060] like Figure 1As shown, to achieve real-time intelligent control of special vehicle operation processes based on virtual space, this embodiment first installs a lidar and an optical camera at the same base within the work area where the special vehicle will operate, ensuring overlapping areas in their detection fields. This allows the detection data from the lidar and optical camera to be fused, forming a three-dimensional voxel-based work area map with color and semantic labels. This map serves as a virtual space generated from a three-dimensional scene model and is stored on a host computer. The host computer then sets up virtual electronic fences within this virtual space to create multi-level defense zones. When the special vehicle is operating, the host computer uses the lidar and optical camera to determine whether each defense zone will pose a risk due to the vehicle's intrusion. Based on the determined risk level, it issues alarm messages or locks the special vehicle's operation. This implementation scheme can be deployed at substation maintenance sites to monitor vehicle movements and prevent vehicles from accidentally touching energized areas of the substation.

[0061] The special vehicles mentioned in this invention refer to those used for near-electrical work at substations, including cranes and aerial work platforms. The work sites are substations with voltage levels of 500kV and above undergoing renovation and expansion projects, or equipment maintenance and replacement work. In this invention, a virtual electronic fence is deployed at the boundary of the special vehicle's work area. Outside the special vehicle's work area, there are operating live equipment (such as busbars). The lidar is a three-dimensional lidar, and the optical camera is a high-definition camera.

[0062] A control system (hardware component) capable of fully implementing all the functions described in the embodiments of this invention typically includes: one lidar unit, one camera, three smart alarm wristbands, two sets of vehicle lock controllers, and one main control unit including a host computer; the base is a tripod; the lidar and high-definition camera are supported by a tripod, with a deployment height of 1.5 meters above the ground; the setup methods include:

[0063] The monitoring area of ​​the lidar and cameras covers the entire work area.

[0064] The main control unit is powered by a power supply and connected to a communication line.

[0065] The smart alarm bracelet is worn by the person in charge of the work site, crane operators, and workers working at heights.

[0066] The vehicle locking controller is installed on cranes and aerial work platforms to lock special vehicles and prevent improper operation of them.

[0067] Its basic functions include: when the substation near-electric work safety management and control system is deployed, the main control unit connects to the power supply and communication lines, and after the equipment is powered on, it performs system self-test and sensor calibration;

[0068] The system self-test process includes checking sensor status, communication links, and power supply status.

[0069] In this embodiment, when setting up an electronic fence, the method is to set up multi-level defense zones on a three-dimensional point cloud map. The multi-level defense zones include warning zones, alarm zones, and danger zones.

[0070] The smart alarm bracelet incorporates a microprocessor, a LoRa wireless communication module, a vibration motor (such as an eccentric rotor motor), an LCD display, a voice module, and a battery. The bracelet connects to the main control unit via a LoRa wireless network.

[0071] The main control unit of the substation near-electric work safety management system communicates with the smart alarm wristband via a LoRa communication link. The LoRa communication link adopts a star or mesh topology, with a communication distance of ≥1km, low power consumption, and is suitable for large-scale deployment in substations.

[0072] When the substation near-electric work safety management system is working, the main control unit calculates the risk level in real time (such as whether a special vehicle is close to entering the warning zone); and broadcasts alarm commands through the LoRa network. The alarm commands include the target smart alarm wristband ID, alarm type, and vibration intensity.

[0073] After receiving the command, the smart alarm bracelet's microprocessor parses the command and drives the vibration motor to vibrate (intensity adjustable), while simultaneously triggering a voice prompt or LCD display.

[0074] When a special vehicle's movement intrudes into the warning zone, such as coming within 7 meters of live equipment, the substation's near-electrical work safety control system will cause the smart alarm wristband to vibrate slightly as a reminder.

[0075] When the movement of a special vehicle intrudes into the alarm zone, such as when it is 6.5m away from live equipment, the substation near-electrical work safety control system will activate an audible and visual alarm and cause the smart alarm wristband to vibrate strongly.

[0076] When a special vehicle's actions intrude into a danger zone, such as being 6 meters away from live equipment, the substation's near-electrical work safety control system will activate the vehicle locking controller to lock the special vehicle to prevent the dangerous action, and will execute the locking after a countdown (3 seconds).

[0077] The sensor calibration process includes joint calibration of lidar and camera, and aligning their coordinate systems with a dynamically defined world coordinate system to adapt to the dynamic changes in the special vehicle's operating site and ensure data consistency for each deployment.

[0078] Special vehicles are used for near-electric work at substations, with work hours during power outage intervals, and the work area is usually rectangular.

[0079] like Figure 2As shown in this embodiment, the tripod is positioned on the extended diagonal of the rectangle in the special vehicle's working area, with a distance of 5-10m from the apex of the rectangle. This position maximizes the combined field of view coverage of the lidar and optical camera, reduces blind spots, and ensures that there is no significant obstruction between the lidar sensor and the special vehicle.

[0080] In a dynamically defined world coordinate system, the coordinate axes are defined as follows: the Z-axis (Z_W) is perpendicular to the horizontal plane and pointing upwards, the X-axis (X_W) and Y-axis (Y_W) are parallel to the ground plane, and the X-axis is set to point in the main working direction; the main working direction includes the main travel direction of the crane or the orientation based on the layout of the main equipment on site (such as parallel to the busbar direction), and the Y-axis is determined by the right-hand rule.

[0081] The specific registration method of the world coordinate system in the substation near-electric work safety management system is as follows: During system initialization, the radar's own position is directly set to (0,0,0) to complete the establishment of the world coordinate system; all sensor data, the spatial position of the virtual fence, and the trajectory of moving targets are uniformly converted and calculated under this world coordinate system.

[0082] Based on the above design, this embodiment provides a method for joint fusion calibration of lidar and camera, including the following steps;

[0083] Step 1: Hardware preparation, specifically:

[0084] 1. Secure Installation: Rigidly fix the lidar and camera to the same base to ensure that their relative poses do not change during calibration. This is the physical basis of the entire calibration process;

[0085] The selection principles for the base location are as follows: The base location should ensure that the combined field of view of the lidar and camera can cover the entire work area, including the movement range of special vehicles, the boundaries of energized equipment, and key risk points in the work area (such as the boom trajectory); there should be no large obstructions (such as structures or other equipment) between the base and the special vehicles to avoid blocking sensor signals; choose a location with higher elevation or a wide field of view; the base should be deployed in a safe area, away from high-voltage energized equipment, traffic lanes, and the movement paths of work vehicles to prevent collisions or interference. The base should be built on solid, flat ground (such as a concrete foundation), avoiding soft soil or vibrating areas, to ensure that the lidar and camera sensors will not shift after being rigidly fixed.

[0086] The preferred selection method in this embodiment includes:

[0087] Based on the geometry of the work area: The work area is usually a rectangular area (such as a power outage interval); the base should be located on the extension of the diagonal of the rectangular area, 5 to 10 meters away from the vertex of the rectangle; for example, for a rectangular interval with a length and width of 30m × 20m, the base can be set at the outer edge of one corner of the rectangle to ensure that the sensor's field of view can cover the diagonal area.

[0088] Field of view overlap: Ensure that the field of view (FOV) of the lidar and camera has sufficient overlap to observe common calibration objects;

[0089] Synchronous triggering: Hardware synchronization (such as GPS PPS pulses or a dedicated synchronizer) is used to ensure that each frame of LiDAR point cloud and camera image is acquired at the same time to avoid errors caused by sensor or object movement.

[0090] Step 2: Camera intrinsic parameter calibration, specifically including:

[0091] 1. Data Acquisition: Take multiple (usually 15-20) images of the checkerboard calibration board from different angles and positions using a camera; the shooting area should cover all regions of the checkerboard calibration board image (center, edges, and four corners);

[0092] 2. Corner Extraction: Use the cv2.findChessboardCorners() function of image processing software (generally, software with image acquisition, preprocessing, edge extraction and feature matching functions is required, and the software must support the calling of mainstream image processing algorithms to meet the data processing adaptation requirements of this solution, the same below) to automatically detect the chessboard corners in each image.

[0093] 3. Optimization and solution: Using the cv2.calibrateCamera() function of the image processing software, input the pixel coordinates of all corner points of all images and their corresponding real-world 3D coordinates (assuming Z=0), and use the maximum likelihood method to estimate and optimize the camera's intrinsic parameter matrix K and distortion coefficients D;

[0094] The intrinsic parameter matrix K = [[fx, 0, cx], [0, fy, cy], [0, 0, 1]],

[0095] The distortion coefficient D = [k1, k2, p1, p2, k3, ...].

[0096] Step 3: Joint extrinsic parameter calibration, used to solve the transformation matrix T_{lidar}^{cam} between the radar and the camera, including:

[0097] 1. Data Acquisition: Place the checkerboard calibration board within the common field of view of the radar and camera; simultaneously acquire one frame of radar point cloud and one camera image. Ensure the calibration board remains stationary. Change the orientation (translation, rotation) and position (far, near, left, right, up, down) of the calibration board, repeatedly acquiring dozens of such data pairs; the more diverse the orientation and position changes, the more accurate and reliable the calibration results.

[0098] 2. Image Corner Detection: For each acquired image, the 2D pixel coordinates p_{image} of the chessboard corner points are accurately extracted using image processing software (cv2.findChessboardCorners()).

[0099] 3. Point cloud plane extraction: For each frame of point cloud, the point cloud clusters belonging to the checkerboard plane are segmented manually or by automatic algorithm.

[0100] The manual method described above is as follows: In the point cloud visualization tool, select the point cloud area where the checkerboard pattern is located;

[0101] The automatic algorithm specifically works as follows: A plane segmentation algorithm (using SACMODEL_PLANE+RANSAC integrated in a software library with point cloud data processing capabilities) is used to fit a checkerboard plane. From the extracted checkerboard point cloud plane, the 3D coordinates P_{lidar} of the four outer corner points of the checkerboard are calculated. This is because images detect all corner points, while radar can only see a plane and cannot "see" the internal corner points. The calculation method is: fitting the plane equation to calculate the bounding box of the point cloud plane; based on the orientation of the checkerboard, the four corner points of the bounding box are taken as estimated values.

[0102] 4. Construct the correspondence, namely: for each pose i of the calibration board, set a set of 2D pixels: the four outer corner points p_i^{image} detected on the image. A set of 3D radar points: the corresponding four outer corner points P_i^{lidar} extracted from the point cloud, as well as the known camera intrinsic parameters K and distortion coefficients D;

[0103] 5. Solve the PnP problem: For each set of data, abstract it into a Perspective-n-Point problem PnP: Assume the world coordinate system is the LiDAR coordinate system. Given the coordinates of a set of 3D points in the world coordinate system, the corresponding 2D pixel coordinates in the image, and the camera intrinsic parameters, find the camera pose relative to the world coordinate system, i.e., T_{lidar}^{cam}. Use the cv2.solvePnP() or cv2.solvePnPRansac() functions of image processing software to solve it. Through iterative optimization, calculate the rotation vector rvec (which can be converted to the rotation matrix R using cv2.Rodrigues()) and the translation vector tvec, which together constitute the transformation matrix T_{lidar}^{cam}. cv2.solvePnPRansac() uses the RANSAC algorithm, which is not sensitive to outliers and has stronger robustness.

[0104] 6. Optimization and Refinement: Extrinsic parameters obtained using only a set of data may have significant noise. This embodiment provides an optimization and refinement approach: Calculate an initial solution using a set of data, and then use this initial solution as a starting point to construct a nonlinear optimization problem (Bundle Adjustment). Simultaneously optimize the extrinsic parameters and (optionally) 3D point positions, minimizing the reprojection error; the reprojection error is the gold standard for evaluating calibration results. It projects the radar point P_{lidar} onto the image using the currently estimated extrinsic parameters T and intrinsic parameters K, obtaining the calculated pixel coordinates p_{projected}, and then calculates the distance difference with the actually detected pixel coordinates p_{image}; the optimization objective is to minimize the sum of this distance difference for all point pairs.

[0105] Through the above methods and their preferred solutions, the embodiments of the present invention achieve high-precision fusion calibration of three-dimensional lidar and two-dimensional camera, providing an accurate multi-sensor data fusion foundation for subsequent solutions.

[0106] Furthermore, the method for three-dimensional scene modeling in this embodiment is as follows: the substation near-electric work safety management system scans the work area, collects initial point cloud data, and performs preprocessing on the point cloud data, including denoising, filtering, and segmentation. Then, a three-dimensional voxel semantic map of the work environment is generated through three-dimensional reconstruction, and the live equipment, grounding equipment, and work area are identified in the three-dimensional voxel semantic map.

[0107] When generating a 3D voxel semantic map, LiDAR and camera data are fused within the 3D voxel semantic map to imbue the LiDAR's 3D geometric point cloud with the camera's texture and semantic information. Specifically, this includes:

[0108] (1) Data synchronization and acquisition: Hardware triggering (PPS pulse) ensures that each frame of point cloud of the lidar and the camera image are acquired at the same time to avoid fusion misalignment caused by motion.

[0109] (2) Point cloud coordinate transformation: Transform the lidar point cloud P_L of the current frame to the world coordinate system through the fixed lidar extrinsic parameter T_{W}^{L_fixed}, and obtain P_WP_W=T_{W}^{L_fixed}·P_L.

[0110] (3) Image semantic segmentation: The trained deep learning model is used to perform pixel-level semantic segmentation on the synchronized image; the model outputs the category label for each pixel, and the category label is set according to the current work site (e.g., "background", "crane", "personnel", "electrical equipment").

[0111] (4) Point cloud semantic annotation (core fusion step): For each point P_W(x,y,z) in the world coordinate system, perform the following operations:

[0112] a. World-to-Camera System: Utilizing the camera's extrinsic parameter matrix, which is composed of the radar-to-camera transformation T_{L}^{C} and the world-to-radar transformation T_{W}^{L}, it is expressed as: T_{W}^{C} = T_{L}^{C}·(T_{W}^{L})^{-1}.

[0113] Transform P_W into the camera coordinate system {C} to obtain P_C;

[0114] b. Camera system → Image pixel system: Project P_C onto the image plane using the camera intrinsic parameter matrix K to obtain its corresponding pixel coordinates (u,v)[u,v,1]^T=K·P_C;

[0115] c. Label mapping: Query the category label of the semantic segmentation image at pixel coordinates (u,v);

[0116] d. Assignment: Assign the category label to point P_W in the 3D point cloud;

[0117] If a 3D point is projected outside the image, it is marked as "unknown".

[0118] (5) Generate a three-dimensional semantic map: The point cloud P_W with semantic information labeled in multiple frames is accumulated and downsampled using a voxel grid filter to reduce the amount of data while maintaining the structure, and finally a three-dimensional voxel semantic map with color and semantic labels is formed, i.e., a three-dimensional point cloud map; the map is stored in the world coordinate system {W}.

[0119] When special vehicles are in operation, the host computer monitors each defense zone using lidar and optical cameras as follows: the lidar continuously scans the defense zone to collect real-time point cloud data of the defense zone status, and processes the point cloud data through motion distortion removal, point cloud filtering and segmentation, target clustering and feature extraction. Then, it identifies the moving parts of the special vehicle and performs multi-target tracking through target association (using data association algorithms, which can be selected according to actual needs, such as nearest neighbor method, semantically constrained probabilistic data association method, etc.), state estimation (preferably using Kalman filtering), and trajectory prediction (linear or nonlinear prediction). Finally, it calculates the shortest distance between the moving parts of the target special vehicle and the defense zone boundary based on KD tree fast nearest neighbor search.

[0120] The specific implementation of the above process includes:

[0121] A dynamic occlusion handling mechanism is adopted when monitoring each defense zone. Specifically, to address the issue of temporary target loss or misjudgment caused by vehicles, personnel, or equipment obstructing the sensor's field of view during operations, the following multi-layered strategy is employed to ensure the continuity of tracking and the accuracy of judgment, including an intelligent predictive tracking algorithm:

[0122] Strategy A, Motion State Prediction: Based on the Kalman Filter algorithm, a motion model is established for the tracked target (such as the end of a crane arm); when the target is briefly occluded (usually <2 seconds), the intelligent prediction tracking algorithm does not immediately delete its trajectory, but predicts its most likely position in subsequent frames based on its velocity, acceleration and angular velocity before being occluded.

[0123] Strategy B, Re-identification: When the target reappears in the sensor's field of view, the system compares the target's point cloud shape features (volume, outline, height), motion continuity, and visual features after semantic segmentation (such as vehicle type, color) with the predicted trajectory for association matching. If it is confirmed that the target is the same before and after the occlusion, continuous tracking is resumed.

[0124] Strategy C, Trajectory Smoothing and Interpolation: For extremely short periods of complete occlusion (<1 second), after the target is recaptured, the system uses an interpolation algorithm (such as spline interpolation) to smooth and complete the trajectory data during the occlusion period, so as to ensure the continuity of the trajectory and avoid distance calculation jumps due to missing data.

[0125] If the target is suddenly lost in a high-risk area (such as the edge of a danger zone) and cannot be quickly recovered using the methods and strategies described above, the substation near-electric work safety management system will exit the protection scan, notify relevant personnel, and manual protection will take over.

[0126] As an important preferred solution to the above design, this invention, in order to achieve centimeter-level precision contour tracking of moving parts of special vehicles (especially cranes, aerial work platforms and other multi-joint equipment), while minimizing system complexity and deployment costs, also innovatively proposes a modeling method that relies entirely on external master sensors, without the need to install any additional sensors or tags on the work vehicle.

[0127] 1. Construction and solution of a pure exogenous kinematic chain model

[0128] (1) Model definition: The special vehicle is abstracted as a multi-rigid-body system consisting of multiple rigid links connected by kinematic joints, and a parameterized kinematic chain model is constructed for it.

[0129] (2) Component-level point cloud segmentation and tracking:

[0130] a. Using the point cloud data obtained from the above steps in real time, the point cloud is separated into point cloud clusters {C_chassis, C_turntable, C_boom, ...} belonging to different rigid body components such as vehicle chassis, turntable, boom (main boom, auxiliary boom), and hook through semantic segmentation (based on point cloud features or semantic results projected onto the image) and clustering algorithms.

[0131] b. For each component's point cloud cluster, calculate its 3D bounding box (OBB) or the principal orientation of the point cloud cluster in real time. By continuously tracking the center position and attitude angle of these bounding boxes, obtain the six-degree-of-freedom pose change sequence of each component.

[0132] (3) Virtual solution of joint hinge points:

[0133] a. Based on the known vehicle kinematics model, virtually calculate the hinge points of each motion joint. For example, the hinge point of the slewing joint is located at the geometric center of the turntable enclosure, and the hinge point of the luffing joint is located at the corner of the enclosure where the boom connects to the turntable.

[0134] b. By analyzing the pose changes of adjacent components (such as the turntable and the boom) and combining the geometric relationship of their bounding boxes, the key joint angles θ_joint(t), such as the rotation angle and pitch angle, are calculated inversely. This process is essentially an inverse kinematics problem based on point cloud observations. Preferably, the geometric relationship of the bounding boxes of adjacent components is as follows: the pitch angle of the luffing joint is calculated inversely using the center distance and attitude angle difference between the bounding boxes of the turntable and the boom.

[0135] (4) Model-driven and state smoothing: The joint angles θ_joint(t) calculated in real time are input into the parameterized kinematic model to drive it to generate the theoretical 3D contour at the current moment in the world coordinate system. At the same time, Kalman filtering is applied to the joint angle sequence to smooth observation noise and ensure the continuity and stability of the model motion.

[0136] 2. Semantic Backprojection Contour Enhancement and Verification

[0137] (1) Pixel-level semantic segmentation and edge extraction: The synchronously acquired visible light images are subjected to pixel-level semantic segmentation by a deep learning model to accurately identify components such as "crane arm", and the edge extraction algorithm is applied to obtain its sub-pixel precision two-dimensional contour.

[0138] As a preferred option, the deep learning model can be a pre-trained semantic segmentation model (such as Mask R-CNN, U-Net).

[0139] (2) Contour back projection and fusion:

[0140] a. Project the aforementioned two-dimensional contour points back into three-dimensional space using a camera model. Utilizing depth information obtained from the LiDAR point cloud as a priori, calculate the precise positions of these contour points in three-dimensional space, generating a high-density, high-precision three-dimensional semantic contour point cloud P_semantic.

[0141] b. Perform data fusion between this semantic contour point cloud P_semantic and the theoretical contour generated by the kinematic model.

[0142] (3) Two-way verification and optimization:

[0143] a. Model validation images: Kinematic models provide powerful spatial priors for semantic segmentation, which can effectively avoid misidentification in complex backgrounds.

[0144] b. Image sharpening model: The three-dimensional contour obtained by semantic back projection is used to verify and sharpen the theoretical contour of the kinematic model, especially to accurately capture the subtle changes of key parts such as the end of the boom.

[0145] c. Finally, through a nonlinear optimization process, the difference between the theoretical contour and the semantic contour is minimized, and a verified and optimized high-precision dynamic 3D contour model is output.

[0146] 3. Final Output and Application

[0147] The optimized high-precision contour model will serve as the optimal representation of the target for subsequent KD-tree nearest neighbor distance calculation. This preferred solution fully leverages the collaborative advantages of the main sensor, achieving stable, accurate, and real-time perception of the contours of complex moving parts without any onboard modules, providing a reliable data foundation for core safety management.

[0148] As an important preferred embodiment, the host computer calculates the shortest distance between the moving parts of the target special vehicle and the boundary of the defense zone based on KD tree fast nearest neighbor search, including the following steps;

[0149] Step 1: Construction of the KD-tree in the preprocessing stage:

[0150] The electronic fence within the defense zone is set as a geometric model. The system discretizes its surface to generate a dense point cloud composed of thousands of points to approximate its surface, called the "defense zone boundary point cloud" S_fence; the preferred discretization density is ≥50 points per square meter.

[0151] We then use S_fence to construct a KD-tree; a KD-tree is a spatial partitioning data structure that can greatly accelerate the operation of "finding the nearest point in the set of points given a point".

[0152] Step 2: Calculate the corresponding sensor data based on the distance between the target vehicle and the base, specifically including:

[0153] For long-range targets (>20m): pure lidar point cloud processing is used. The camera does not participate in ranging; it is only used to provide preliminary visual features of the target to assist in classification. The distance is calculated directly using all the point clouds P_target after clustering the target. Because the distance is long, the target point cloud is sparse, but the defense zone is also large, so the accuracy is sufficient.

[0154] For medium-range targets (10m-20m): LiDAR is the primary method, with camera assistance for classification; the target point cloud is downsampled to balance computational efficiency and accuracy, i.e., voxel downsampling is performed on P_target to reduce the number of points, and then distance calculation is performed.

[0155] Close-range / high-risk targets (<10m, especially <6m): High-precision fusion calculations are employed, specifically including:

[0156] a. Point cloud refinement: Do not downsample P_target, use all original points or even increase the point cloud density through interpolation (especially at the target edge).

[0157] b. Image-assisted edge extraction: Project the target onto the image and use the Canny operator to extract precise image edges;

[0158] c. Back projection: The extracted image edge points, combined with the camera's optical image model and the depth information mapped from the point cloud, are back projected into three-dimensional space to generate a series of additional, high-precision three-dimensional edge points, which are added to P_target. The high resolution of the image is used to "sharpen" the edges of the LiDAR point cloud, thereby obtaining a more accurate target contour representation.

[0159] (3) Nearest neighbor search and distance calculation:

[0160] For each point P_i in the processed target point cloud P_target, use the previously constructed zone KD tree to query and find the point Q_j in S_fence that is closest to P_i;

[0161] Calculate the Euclidean distance between these two points: d_i = dist(P_i, Q_j);

[0162] Iterate through all points in P_target and find the minimum value among all d_i, i.e., min(d_i);

[0163] min(d_i) is taken as the shortest distance between the moving part of the special vehicle and the boundary of the electronic fence.

[0164] As a preferred embodiment, a multi-source alarm is generated by simultaneously vibrating the smart alarm bracelets of all relevant personnel (e.g., all bracelets vibrate slightly at a distance of 7m). The following method is used:

[0165] A. Time synchronization protocol: The entire system adopts the IEEE1588-2008 (PTPv2) time synchronization protocol. The main control unit, sensors and wristband are all synchronized to a unified time source (such as a GPS clock). The smart alarm wristband has a built-in hardware clock, which periodically performs time calibration with the main control unit to ensure that the time error between the two is less than 1ms.

[0166] B. Using a broadcast communication mechanism: When the main control unit issues an alarm command, it adopts the LoRa broadcast mode (one-to-many) to send the command to all registered smart alarm bracelets in the system at the same time, instead of sending it one by one; the command includes a timestamp, and the bracelet will vibrate at the predetermined time according to the timestamp to avoid network latency differences;

[0167] Redundancy design: If a bracelet does not receive a command, the system will resend the alarm (up to 3 times) to ensure reliability; the smart alarm bracelet stores local logic: after receiving a high-level alarm (such as a danger zone), its execution task will immediately override the execution task of the low-level alarm.

[0168] The advantages of the solutions provided in the embodiments of the present invention include:

[0169] 1. Ease of deployment and use

[0170] There is no need to collect on-site data and build a dedicated 3D model before the operation, nor is it necessary to install additional sensors on special vehicles. Safety control and defense can be completed simply by using the integrated radar and vision sensing equipment (LiDAR and optical camera) that can be quickly deployed on-site. This simplifies the on-site preparation process, reduces reliance on pre-operation preparation work, and avoids resource consumption caused by repeated construction. It is also suitable for the temporary and flexible control needs in near-electricity operation scenarios at substations.

[0171] 2. Precision and adaptability of sensing and protection

[0172] By integrating "lightning and vision" hardware and calibration technology, and combining the three-dimensional geometric perception capability of lidar with the semantic recognition capability of cameras, the accuracy and robustness of target monitoring in complex operating environments are improved, reducing the interference of factors such as birds, falling objects, and changes in ambient light on monitoring results. It breaks through the limitations of traditional two-dimensional planar protection and constructs a three-dimensional dynamic safety domain, which can not only cover the monitoring needs of the vertical movement and complex contours of special vehicles (such as crane booms), but also adjust the safety boundary in real time according to the vehicle's operating posture and movement trajectory, avoiding blind spots or misjudgments caused by static protection, and adapting to the dynamic scene characteristics of substation near-electricity operations.

[0173] 3. Completeness and timeliness of security management

[0174] A smart closed-loop system of "perception-analysis-early warning-control" is constructed. From real-time data collection by the integrated radar and vision device, to data analysis of target association, state estimation and trajectory prediction, to risk early warning through multiple channels (such as smart bracelets and audible and visual alarms), and finally to the mandatory intervention of the vehicle lock controller, a complete safety control link is formed, avoiding control loopholes caused by relying solely on passive alarms. In the risk response stage, early warning and intervention actions can be triggered quickly to support the timeliness requirements of safety control for near-electric work and reduce the possibility of risk spread.

[0175] 4. Targeted improvements to address the pain points of traditional technologies

[0176] Compared to traditional technologies such as two-dimensional laser fences, video surveillance, and point sensors, this solution addresses the issues of missed alarms in two-dimensional protection and incompatibility with static protection through three-dimensional modeling and dynamic protection. It enhances target differentiation capabilities through laser-visual fusion and semantic recognition, reducing false alarms caused by the inability to identify interfering objects. Furthermore, the design eliminates the need for additional vehicle-mounted sensors, avoiding complex deployment and susceptibility to electromagnetic interference. This approach better meets the actual technical requirements of near-electrical operations in substations, improving the reliability and practicality of safety management.

[0177] Based on the above-mentioned technical points in this embodiment, the overall process for implementing the solution of this invention can be found in [reference needed]. Figure 3 Specifically, it includes the following steps:

[0178] S100: System Deployment and Initialization. Hardware deployment is completed at the substation's near-electricity work site (such as a 500kV substation renovation / expansion area or equipment maintenance area). The 3D LiDAR and high-definition camera are rigidly fixed using tripods (deployment height 1.5 meters, tripods placed on the extended diagonal of the work area rectangle, 5-10 meters from the vertex). Simultaneously, intelligent alarm wristbands (worn by the work supervisor, crane operator, and aerial work personnel) and vehicle locking controllers (installed on the crane / aerial work vehicle) are installed, and the main control unit's power and communication lines are connected. After powering on the equipment, a system self-test (checking sensor status, communication link, and power status) and sensor calibration (LiDAR-visual fusion calibration, using OpenCV functions to solve for camera intrinsic / extrinsic parameters, aligning the coordinate system to a dynamic world coordinate system, with the origin set as the radar's geometric center) are performed to ensure equipment and data consistency.

[0179] S200: 3D Scene Modeling and Electronic Fence Setting. Initial point cloud data is collected by scanning the work area with LiDAR. After denoising, filtering, and segmentation preprocessing, the LiDAR-visual data is fused with synchronized camera images (data synchronization triggering, point cloud coordinate transformation, image semantic segmentation, and point cloud semantic annotation) to generate a 3D voxel semantic map with color and semantic labels (background / crane / personnel / electrical equipment). Multi-level defense zones (early warning zone, alarm zone, danger zone) are set in the map. The boundaries of the defense zones are discretized into point clouds S_fence with a density of ≥50 points / ㎡ and a KD tree is constructed to provide spatial reference for subsequent distance calculations.

[0180] S300: Real-time data acquisition and processing. The lidar continuously acquires real-time point cloud data of the protected area. After motion distortion removal, filtering and segmentation, and target clustering, it combines camera images to complete target association (optional nearest neighbor method / probabilistic data association method with semantic constraints), state estimation (Kalman filtering), and multi-target tracking. For complex moving parts such as booms, a pure external kinematics chain model (multi-rigid-body abstraction → component-level point cloud segmentation → joint hinge point calculation → model-driven smoothing) and semantic back-projection contour enhancement (Mask R-CNN / U-Net semantic segmentation → edge back-projection → bidirectional optimization of theoretical and semantic contours) are used to generate high-precision 3D contours. At the same time, dynamic occlusion is handled through Kalman filtering prediction, target re-identification, and trajectory interpolation strategies to ensure tracking continuity.

[0181] S400: Risk Assessment and Alarm Decision-Making. Based on KD-tree fast nearest neighbor search, a hierarchical ranging strategy is adopted according to the target's distance from the base (long-range pure lidar processing, mid-range point cloud downsampling, and near-range point cloud refinement + Canny edge extraction + back projection) to calculate the shortest distance min(d_i) between the moving parts of the special vehicle and the boundary of the defense zone; the risk level is determined according to the distance (e.g., 7m intrusion warning zone, 6.5m intrusion alarm zone, 6m intrusion danger zone), and corresponding alarm or locking decisions are generated.

[0182] S500: Multi-source alarm and forced intervention. The main control unit sends commands to all smart alarm wristbands via LoRa broadcast mode (star / mesh topology, communication distance ≥1km), and ensures synchronized response of the wristbands (slight vibration / strong vibration + voice / LCD display) by combining the IEEE1588-2008 time synchronization protocol; if the target intrudes into the danger zone, a 3-second countdown is started, and if the operation does not stop within the time limit, the vehicle locking controller is triggered to lock the emergency operation circuit of the special vehicle; at the same time, the system records the entire process data (point cloud, image, positioning data) to provide a basis for subsequent analysis.

[0183] As a more preferred implementation method, the specific implementation of step S300 is further demonstrated below with reference to a specific implementation example:

[0184] S301: Flowchart of Joint Angle Inverse Decomposition Algorithm Based on External Perception Point Cloud

[0185] like Figure 4 As shown, taking a crane used in a typical substation hoisting operation as an example, its motion can be simplified to a slewing joint centered on the turntable and a pitching joint connecting the boom. This step aims to inversely solve the real-time angles of these two key joints using only external lidar point clouds.

[0186] Definition of noun:

[0187] 1. Slewing Joint: A mechanical connection device that allows the crane's superstructure to rotate in the horizontal plane. Its function is to enable the boom to rotate left and right, covering working areas in different horizontal directions. The physical counterpart is the slewing support device (large bearing or gear mechanism) between the crane's turntable and chassis.

[0188] 2. Pitch Joint: A mechanical connection device that allows the boom to swing up and down in a vertical plane. Its function is to control the boom's elevation and lowering, changing the working height and radius. In practice, it corresponds to the hinge pin at the connection between the boom and the turntable, and is usually driven by a hydraulic cylinder.

[0189] 3. Rotary joint hinge point J_swing: The theoretical center of rotation of the rotary joint, i.e., the axis of rotation of the crane in the horizontal plane. Its physical location is on the vertical line of the geometric center of the turntable, and all horizontal rotational movements occur around this point.

[0190] 4. Pitch Joint Hinge Point J_luff: The actual physical connection point of the pitch joint, i.e., the fulcrum for the boom's up-and-down swing. Located at the connection between the turntable and the boom, it is usually the center of a visible pin or hinge device. All pitch movements of the boom rotate around this point.

[0191] 5. Rotation angle θ: The angle through which the crane's superstructure (mainly the turntable and boom) rotates in the horizontal plane relative to its initial direction. The range is typically 0° to 360° (or a finite angle range).

[0192] 6. Pitch angle φ: The angle of inclination of the boom relative to the horizontal plane. The range is usually 0° to +90°.

[0193] Scenario example: Suppose a crane is lifting equipment inside a substation:

[0194] 1. The operator rotates the boom 45° to the right: rotation angle θ = 45°

[0195] 2. The operator raises the boom upwards by 45°: pitch angle φ = 45°

[0196] 3. Center of horizontal rotation: J_swing, the hinge point of the rotary joint;

[0197] 4. The fulcrum for the boom's up-and-down swing: the pitch joint hinge point J_luff.

[0198] like Figure 4 As shown, based on the multi-rigid-body system characteristics of special vehicles, the hinge points of each moving joint are virtually determined. Taking a crane-type special vehicle used in this embodiment as an example, the hinge points of its core moving joints and their corresponding angle parameters can be specifically defined as follows:

[0199] 1. Rotary joint hinge point (Jswing): The location is the vertical projection of the turntable center onto the ground, with coordinates (2, 1, 0)m in the world coordinate system. It is the theoretical center of the crane's horizontal rotation.

[0200] 2. Pitch joint hinge point: The location is the physical connection point between the boom and the turntable. Its coordinates in the world coordinate system are (2, 1, 0.6)m. It is the fulcrum for the boom to swing up and down.

[0201] 3. Rotation angle (θ): Defined as the rotation angle of the boom in the horizontal plane, with the slewing joint hinge point Jswing as the center and the X-axis of the world coordinate system as the 0° reference. The example value is 45° (meaning the boom rotates in the 45° direction), and the range is 0° to 360°.

[0202] 4. Pitch angle (φ): Defined as the tilt angle of the boom relative to the horizontal plane, with the pitch joint hinge point Juff as the center and the horizontal line as the 0° reference. The example value is 45° (meaning the boom tilts upward by 45°), and the range is 0° to 90°.

[0203] Based on the spatial position and angle parameter definitions of the hinge points mentioned above, the point cloud data collected by the external lidar can be combined with the inverse kinematics of the pose changes of the component point cloud clusters (such as the center distance and attitude angle difference of the turntable and boom point cloud clusters) to obtain the real-time joint parameters such as rotation angle and pitch angle, thereby driving the parametric kinematic model to generate the theoretical three-dimensional contour of the crane operating components.

[0204] Step S301-1: Component-level point cloud segmentation and feature extraction

[0205] For each frame of real-time point cloud P_t, the point cloud cluster P_crane belonging to the "crane" category is first separated through preprocessing and semantic segmentation. Further, the DBSCAN clustering algorithm based on Euclidean distance is used to segment P_crane into finer-grained component point cloud clusters, including: chassis C_chassis, turntable C_turntable, and boom C_boom. For each component point cloud cluster, principal component analysis (PCA) is performed to obtain its first principal component vector, which represents the component's main extension direction in space.

[0206] The first principal component of the turntable point cloud cluster C_turntable is defined as the direction vector v_turntable.

[0207] The first principal component of the boom point cloud cluster C_boom is defined as the direction vector v_boom.

[0208] Step S301-2: Virtual positioning of joint hinge points

[0209] Based on the prior kinematic model of the crane (which can be obtained through a vehicle model database or initial scan calibration), the joint hinge points are virtually located on the component point cloud cluster.

[0210] The pivot point J_swing of the rotary joint is usually located at the geometric center of the turntable. It can be obtained by calculating the center of the 3D bounding box (OBB) of C_turntable and translating it upwards along the vertical direction (the Z-axis of the world coordinate system) by a fixed model parameter h_swing (half the height of the turntable).

[0211] The pitch joint hinge point J_luff is located at the connection between the turntable and the boom. It can be obtained by extracting a subset of the point cloud near the turntable end of C_boom and calculating its centroid.

[0212] Step S301-3: Joint Angle Calculation

[0213] The following solution model is established:

[0214] 1. Calculation of the rotation angle θ: Calculate the projection vector v_turntable_xy onto the XOY plane (horizontal plane) of the turntable direction vector v_turntable. The rotation angle θ is the angle between v_turntable_xy and the positive X-axis direction X_W of the world coordinate system, which can be solved using the cross product and dot product of the vectors.

[0215] θ=arctan2((X_W×v_turntable_xy)_z,X_W·v_turntable_xy)

[0216] Here, arctan2(y,x) is the arctangent function in the four quadrants, and (·)_z represents the component of the vector along the Z-axis.

[0217] 2. Pitch angle φ calculation: Calculate the angle between the boom direction vector v_boom and the horizontal plane (XOY plane). First, calculate the vector v_boom_J formed by the hinge point J_luff and a point on the boom end direction (taking the v_boom direction). The pitch angle φ is:

[0218] φ=arcsin((v_boom_J·Z_W) / ‖v_boom_J‖)

[0219] Where Z_W is the vertical upward unit vector in the world coordinate system, and ||·|| represents the magnitude of the vector.

[0220] Step S301-4: State prediction under dynamic occlusion processing

[0221] When components such as the boom are briefly occluded, causing the point cloud cluster C_boom to be incomplete or missing, dynamic occlusion processing is activated. Specifically, an Extended Kalman Filter (EKF) is used, with the state variables x = [θ, φ, θ_dot, φ_dot]^T, representing the joint angles and their angular velocities. Based on the motion state before occlusion, the current θ_hat_t and φ_hat_t are predicted using a uniform velocity model as estimates of the joint angles, maintaining tracking continuity.

[0222] S302: Optimization steps for fusing theoretical and semantic profiles

[0223] This step aims to fuse and optimize the theoretical 3D contour generated based on the kinematic model with the 3D semantic contour generated based on visual back projection, so as to obtain the final high-precision dynamic 3D contour M_fused.

[0224] Step S302-1: Theoretical Profile Generation

[0225] The joint angles θ and φ calculated in step S301 are input into the parametric kinematic model of the crane (such as the Denavit-Hartenberg model), which drives the simplified 3D CAD model or geometric primitives (such as combinations of cylinders and cuboids) to generate the theoretical 3D mesh model M_kinematic of the boom at the current moment. This model describes the precise spatial position of each part of the boom under ideal geometry.

[0226] Step S302-2: Semantic Contour Generation

[0227] Simultaneously, pixel-level semantic segmentation is performed on the optical camera image to obtain a precise mask for the "crane" region. Subpixel-precision Canny edge detection is applied to this mask to obtain a set of two-dimensional edge points, E_2d. Using the calibrated camera intrinsic matrix K, distortion coefficients D, and the sparse depth map provided by the LiDAR (obtained by projecting the point cloud P_t onto the image and interpolating), each edge point in E_2d is back-projected onto the world coordinate system to form a three-dimensional semantic contour point cloud, P_semantic.

[0228] Step S302-3: Contour Matching and Optimization

[0229] The theoretical mesh model M_kinematic and the semantic point cloud P_semantic may have slight deviations due to model simplification, sensor noise, etc. We define the difference between them as a distance metric from a point to the mesh. We use a variant of the directed Hausdorff distance as the optimization objective.

[0230] 1. Define residual: For each point p_i in the semantic point cloud P_semantic, calculate the distance d_i from it to the nearest point in the theoretical grid M_kinematic(Θ), where Θ=[θ,φ,t,s] are the parameters to be optimized, including joint angles, and optional local model translation t and scale fine-tuning s.

[0231] r_i(Θ)=d_i(p_i,M_kinematic(Θ))

[0232] 2. Construct the optimization problem: Find the optimal parameter Θ and minimize the sum of squared distance residuals for all semantic contour points:

[0233] Θ=argmin_ΘΣ_{i=1}^{N}[r_i(Θ)]^2

[0234] 3. The Levenberg-Marquardt (LM) algorithm is used to solve this problem: The LM algorithm combines the Gauss-Newton method with the steepest descent method, and can effectively solve this type of nonlinear least squares problem. The iterative update formula is as follows:

[0235] (J^TJ+λdiag(J^TJ))δ=-J^Tr

[0236] Where J is the Jacobian matrix of the residual vector r with respect to the parameter Θ, and λ is the damping factor. In each iteration, the increment δ is solved and the parameters are updated: Θ←Θ+δ, until convergence.

[0237] 4. Output Fusion Contour: Substitute the optimized parameters Θ into the kinematic model to obtain the optimized theoretical contour M_kinematic. Simultaneously, points in the semantic point cloud P_semantic that are very close to the optimized mesh (e.g., d_i < 5cm) are used as surface details to supplement the model, ultimately forming a high-precision dynamic 3D contour M_fused. This contour maintains both the overall structural smoothness and physical plausibility of the kinematic model and possesses high-resolution edge details for visual perception. The overall process is as follows: Figure 5 As shown.

[0238] Through the above steps, this embodiment of the invention achieves real-time reconstruction of the contours of complex moving parts with centimeter-level accuracy without the need for onboard sensors, laying a reliable foundation for subsequent accurate distance calculation and safety warning.

[0239] Based on the design of the above embodiments of the present invention, the following introduction of a more specific application example further demonstrates the specific application and implementation process of the solution:

[0240] A 500kV substation is undergoing a main transformer expansion project, requiring the hoisting of a new main transformer within the operating equipment area. The working radius is 15m, and the nearest working point is only 6.5m from the energized busbar (the safety distance requirement is 6m), posing an extremely high risk. This example uses a substation near-energized work safety management method based on the aforementioned system, including:

[0241] 1 three-dimensional lidar,

[0242] One high-definition camera, working in conjunction with LiDAR;

[0243] The lidar and high-definition camera are supported by a tripod and deployed at a height of 1.5 meters (from the ground), which can monitor and cover the entire work area;

[0244] One main control unit connects to the power supply and communication lines to power on the equipment;

[0245] Three smart alarm wristbands are worn by the work supervisor, crane operator, and high-altitude workers.

[0246] Two sets of vehicle locking controllers are installed on the crane and the aerial work platform.

[0247] System deployment includes the following:

[0248] System self-test: checks sensor status, communication link, and power supply status;

[0249] Sensor calibration: Joint fusion calibration of lidar and camera, coordinate system one.

[0250] World Coordinate System Definition and Registration:

[0251] To adapt to the dynamic changes in special vehicle operation sites and ensure data consistency for each deployment, the system adopts a dynamically defined world coordinate system.

[0252] Origin determination principle: The geometric center of the main lidar sensor in this deployment is taken as the coordinate origin.

[0253] The optimal location for the device deployment is within the working area of ​​the special vehicle (a rectangular area during power outages within a substation), on the extended diagonal of the rectangle, at a distance of 5-10 meters from the vertex. Note that there should be no significant obstruction between the main lidar sensor and the special vehicle.

[0254] Coordinate axis definition: The Z-axis (Z_W) is perpendicular to the horizontal plane and pointing upwards. The X-axis (X_W) and Y-axis (Y_W) are parallel to the ground plane. The X-axis is usually set to point in the main direction of operation (such as the main direction of crane travel) or to be oriented according to the layout of the main equipment on site (such as parallel to the busbar). The Y-axis is determined by the right-hand rule.

[0255] Coordinate system registration: During system initialization, the radar's own position is directly set to (0,0,0), completing the establishment of the world coordinate system. All sensor data, the spatial position of virtual fences, and the trajectory of moving targets are uniformly converted and calculated in this world coordinate system.

[0256] The radar and camera are rigidly fixed to the same base to ensure that their relative positions and orientations do not change during the calibration process. This is the physical basis of the entire calibration work.

[0257] Base selection principles: The base location should ensure that the combined field of view of the LiDAR and camera covers the entire operating area, including the movement range of special vehicles, the boundaries of energized equipment, and key risk points (such as crane boom trajectory). There should be no large obstructions (such as structures or other equipment) between the base and the special vehicles to avoid blocking sensor signals. Prioritize locations with higher elevations or open views. The base should be deployed in a safe area, away from high-voltage energized equipment, traffic lanes, and the movement paths of operating vehicles to prevent collisions or interference. The base should be built on solid, level ground (such as a concrete foundation), avoiding soft soil or areas prone to vibration, ensuring that the sensors are rigidly fixed and will not shift.

[0258] Specific selection method: Based on the geometry of the work area: The work area is usually rectangular (such as a power outage interval). The base should be located on the extension of the diagonal of the rectangular area, 5-10 meters away from the vertex of the rectangle. For example, for a rectangular interval with a length and width of 30m × 20m, the base can be set at the outer edge of one corner of the rectangle to ensure that the sensor's field of view can cover the diagonal area.

[0259] Ensure that the field of view (FOV) of the radar and camera has a sufficiently large overlap area so that common targets can be observed.

[0260] Hardware synchronization (such as GPS PPS pulses or a dedicated synchronizer) is used to ensure that each frame of radar point cloud and camera image is acquired at the same time, avoiding errors caused by sensor or object movement.

[0261] During operation, when the crane boom approaches the live busbar while rotating, the system provides an accurate warning: when the distance reaches 7m, all personnel's wristbands vibrate slightly, and the crane operator slows down the operation.

[0262] If, due to a command error, the crane continues to operate and approaches the 6.5m warning line, the system will trigger an audible and visual alarm and cause the wristband to vibrate strongly.

[0263] If the crane continues to move forward due to inertia after being immediately stopped by the on-site commander and enters the 6m danger zone, the system will start a 3-second countdown. If the crane does not stop moving after the countdown ends, the main control unit will immediately send a locking signal to the crane locking controller. The crane locking controller will then activate and automatically lock the crane's emergency stop circuit.

[0264] The entire process, from warning to locking, took 5.8 seconds, with the closest distance being 6.2 meters, thus avoiding potential discharge accidents. The system fully recorded the data of the entire event and recorded the boom's movement trajectory and distance changes as point cloud data.

[0265] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0266] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0267] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0268] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

[0269] This invention is not limited to the above-described preferred embodiments. Anyone inspired by this invention can derive other methods for intelligent control of special vehicle operation processes based on virtual space. All equivalent variations and modifications made within the scope of the claims of this invention shall fall within the scope of this invention.

Claims

1. A method for intelligent control of special vehicle operation processes based on virtual space, characterized in that: The overlapping detection fields of lidar and optical cameras deployed at special vehicle operation sites serve as external sensing devices to synchronously collect operational environment data, without relying on any additional sensors or tags installed on the special vehicles. Based on the aforementioned work environment data, a three-dimensional virtual space containing semantic information of electrical equipment is constructed, and a virtual electronic fence is set up in the three-dimensional virtual space to form a safety zone; Based on real-time point cloud data from LiDAR, point cloud clusters of different rigid components of special vehicles are identified through segmentation and clustering. The rotation angles of the joints connecting each component are calculated and input into a parametric kinematic model to generate the theoretical three-dimensional contours of the moving parts of the special vehicle. Simultaneously, based on optical camera image data, two-dimensional edges of vehicle components are extracted through semantic segmentation. Combined with LiDAR depth information, the two-dimensional edges are back-projected into three-dimensional space to generate three-dimensional semantic contours. The theoretical three-dimensional contours and the three-dimensional semantic contours are fused to obtain high-precision dynamic three-dimensional contours of the moving parts of the special vehicle. The shortest distance between the high-precision dynamic three-dimensional contour and the boundary of the virtual electronic fence is calculated as a benchmark for performing graded early warning or operational intervention on special vehicles; The specific method for calculating the rotation angle of each component's connecting joint is as follows: based on the characteristics of the multi-rigid-body system of the special vehicle, the hinge point position of each moving joint is virtually determined, and then the rotation angle of each joint is calculated in reverse by analyzing the geometric relationship of the point cloud clusters of adjacent components. The geometric relationship includes the center distance and attitude angle difference of the point cloud clusters of adjacent components. The semantic segmentation to extract the two-dimensional edges of vehicle components uses a trained deep learning pixel-level semantic segmentation model, and the edge extraction uses an edge extraction operator that can achieve sub-pixel accuracy. During the generation of high-precision dynamic 3D contours, a dynamic occlusion processing strategy is simultaneously employed to maintain the continuity of target tracking, specifically including: Motion state prediction: Based on the filtering algorithm, a motion model is established for the moving parts of the tracked special vehicle. When the part is briefly occluded, the position of the part in the subsequent acquisition frame is predicted based on the motion parameters before occlusion. Target re-identification: When the occluded part reappears in the detection field of view, the point cloud shape features, motion continuity and semantically segmented visual features of the part are compared with the predicted trajectory to restore continuous tracking; Trajectory interpolation: If the occlusion time of a component meets a preset short time threshold, an interpolation algorithm is used to complete the trajectory data during the occlusion period to avoid abrupt changes in subsequent distance calculations; The calculation of the rotation angle of the joints connecting each component specifically includes: segmenting the point cloud clusters of special vehicle components using the DBSCAN clustering algorithm, extracting the main direction vector of the component using principal component analysis, solving the rotation angle by vector cross product and dot product based on the virtual positioning results of the joint hinge points, and solving the pitch angle by the angle between the vector and the vertical direction; when the component is briefly occluded, the extended Kalman filter is used to predict the joint angle and its angular velocity. The fusion of the theoretical 3D contour and the 3D semantic contour specifically includes: using the directed Hausdorff distance variant as the optimization objective, constructing optimization parameters that include joint angles, local translations, and scale fine-tuning; using the Levenberg-Marquardt algorithm to minimize the sum of squared residual distances from the semantic contour points to the theoretical contour; fusing the optimized theoretical contour and the semantic contour point cloud; and outputting a high-precision dynamic 3D contour.

2. The method for intelligent control of special vehicle operation processes based on virtual space according to claim 1, characterized in that: The lidar and optical camera are deployed together through a rigid support structure. The position of the rigid support structure is determined according to the geometry of the special vehicle's operating area to ensure that the combined field of view of the lidar and optical camera covers the entire operating area and the boundary of the electrical equipment, and that the overlapping area of ​​their detection fields meets the requirements for jointly acquiring calibration objects. The lidar is a three-dimensional lidar, and the optical camera is a high-definition camera. The deployment height of the rigid support structure is adapted to the field of view requirements of the operating area to avoid obstruction.

3. The method for intelligent control of special vehicle operation processes based on virtual space according to claim 1, characterized in that: When the lidar and optical camera synchronously collect operational environment data, hardware synchronization methods are used to ensure that the data collection time is consistent. The hardware synchronization methods include GPS PPS pulses or dedicated synchronizers. The main control unit maintains time consistency with the lidar and optical camera through a time synchronization protocol. The synchronization accuracy of the time synchronization protocol meets the requirement of no misalignment in data fusion.

4. The method for intelligent control of special vehicle operation processes based on virtual space according to claim 1, characterized in that: The construction of the three-dimensional virtual space containing semantic information of the electrical equipment specifically includes: Preprocessing of the initial point cloud data acquired by lidar includes denoising, filtering, and segmentation; The lidar and optical camera are jointly calibrated. The camera intrinsic parameter matrix, distortion coefficient and the transformation matrix from lidar to optical camera are obtained sequentially through the corner extraction function, camera calibration function and pose solving function of the image processing software. The coordinate systems of the two are unified into a dynamically defined world coordinate system. The origin of the world coordinate system is set as the geometric center of lidar, and the coordinate axis direction is oriented according to the operation scene. The calibrated lidar point cloud is transformed to the world coordinate system, and semantic labels are assigned to the point cloud by combining the pixel-level semantic segmentation results of the optical camera image. The semantic labels include at least the background, special vehicles and electrical equipment categories. Then, the data volume is reduced by downsampling while preserving the spatial structure, generating a three-dimensional virtual space with semantic information.

5. The method for intelligent control of special vehicle operation processes based on virtual space according to claim 1, characterized in that: Calculating the shortest distance between the high-precision dynamic 3D contour and the boundary of the virtual electronic fence specifically includes: The surface of the virtual electronic fence is discretized into a dense point cloud of preset density to form a defense zone boundary point cloud. A spatial index structure is constructed based on the defense zone boundary point cloud, and the preset density meets the distance calculation accuracy requirements. Based on the distance range between the special vehicle and the lidar, different point cloud processing methods are used to calculate the distance. The closer the distance range, the more refined the point cloud processing method is, in order to improve the ranging accuracy. A fast nearest neighbor search is performed using the spatial index structure to traverse all points of the high-precision dynamic 3D contour, find the minimum Euclidean distance to the point cloud of the defense zone boundary, and use this minimum Euclidean distance as the shortest distance.

6. The method for intelligent control of special vehicle operation processes based on virtual space according to claim 1, characterized in that: The tiered early warning is achieved through wireless communication between the main control unit and the smart alarm bracelet. The wireless communication is a low-power wide-area communication method, and the smart alarm bracelet is worn by key personnel at the work site. When the shortest distance reaches the preset warning threshold, the smart alarm bracelet triggers the first-level alarm response, including vibration reminder and information display; When the shortest distance reaches the preset alarm threshold, the smart alarm bracelet triggers a second-level alarm response. The intensity of the second-level alarm response is higher than that of the first-level alarm response, and an on-site audible and visual alarm is triggered simultaneously. The wireless communication uses a broadcast mode to ensure that all smart alarm bracelets respond synchronously, and supports an alarm command retransmission mechanism to improve reliability. Higher-level alarm commands cover the execution tasks of lower-level alarm commands.

7. The method for intelligent control of special vehicle operation processes based on virtual space according to claim 1, characterized in that: The specific intervention for special vehicles is as follows: when the shortest distance reaches a preset danger threshold, the main control unit sends a locking command to the locking controller installed on the special vehicle, and the locking command triggers a countdown of a preset duration; if the special vehicle still does not stop the dangerous action after the countdown ends, the locking controller automatically locks the special vehicle's operating circuit to prevent the dangerous action from continuing; at the same time, the collected data and decision data during the control process are recorded for subsequent traceability analysis.

8. A system for intelligent control of special vehicle operation processes based on virtual space, used to implement the method as described in claim 1, characterized in that, include: External perception module, main control module, 3D modeling module, contour tracking module, and risk management module; The external sensing module includes a lidar and an optical camera, both rigidly fixed to the same base and with overlapping detection fields. The external sensing module achieves synchronous data collection of the working environment only by being deployed at the special vehicle operation site, without relying on any additional sensors or tags installed on the special vehicle. The 3D modeling module is communicatively connected to the external sensing module and is used to construct a 3D virtual space containing semantic information of electrical equipment based on the working environment data, and to set up a virtual electronic fence in the 3D virtual space to form a safety zone. The contour tracking module is communicatively connected to the external perception module and the 3D modeling module. It is used to identify point cloud clusters of different rigid components of special vehicles based on real-time point cloud data from LiDAR, calculate the rotation angle of the joints connecting each component and input the parameterized kinematic model to generate a theoretical 3D contour. At the same time, it extracts the 2D edges of vehicle components based on optical camera image data and back-projects them to generate a 3D semantic contour. The theoretical 3D contour and the 3D semantic contour are fused to obtain a high-precision dynamic 3D contour of the moving parts of the special vehicle. The risk management module is communicatively connected to the contour tracking module and the 3D modeling module, and is used to calculate the shortest distance between the high-precision dynamic 3D contour and the boundary of the virtual electronic fence, and to perform graded early warning or operational intervention on special vehicles based on the shortest distance; The main control module is communicatively connected to and powered by the external perception module, 3D modeling module, contour tracking module, and risk management module, and is used to coordinate and control the collaborative work of each module.

Images