Hierarchical modeling and planning method and system for working environment of engineering machinery

Abstract

The invention provides an engineering machinery working environment layered modeling and planning method and system, and the method comprises the steps: carrying out the modeling based on the scanning point cloud data of an engineering machinery working region, the geometric point cloud of an object in the working region, and the aerial photographing image and aerial photographing point cloud data of the working region; a bottom geometric perception layer, a middle semantic planning layer and an upper topological navigation layer are obtained, and a corresponding model is reconstructed when an object moves, so that the contradiction between high-precision calculation power waste and low-precision modeling insufficiency caused by a current single-precision map is solved, the modeling calculation efficiency is improved, and the method can be expanded to different complex scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field related to engineering machinery, and in particular relates to a hierarchical modeling and planning method and system for an engineering machinery operating environment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The current modeling of engineering machinery operating environments generally uses single-precision maps such as the dynamic grid method, which has significant defects: (1) Waste of computing power: High-precision maps such as millimeter-level collision avoidance scenarios require global calculations, resulting in redundant computing power for low-precision scenarios such as vehicle navigation; (2) Insufficient modeling: Low-precision maps cannot meet local fine-grained operation requirements such as robotic arm grasping posture planning; (3) Semantic loss: Existing technologies lack structured semantic information and are difficult to support multi-level task collaborative optimization.

[0004] In ship hold cleaning operations, a single high-precision map needs to continuously update the geometric details of the bulkhead, but the vehicle's global path planning only requires topological relationships, resulting in a serious waste of computing resources; in yard stacking scenarios, a single low-precision map cannot provide a parametric model of the material contour, affecting the efficiency of grasping path generation; port cleaning robots are prone to robotic arm collision accidents due to excessive response delays caused by the global update of high-precision maps.

[0005] Therefore, how to solve the problem of waste of high-precision computing power caused by single-precision maps and insufficient low-precision modeling, and improve the collaborative optimization of multiple tasks in complex scenarios, is a problem that needs to be solved at present. Summary of the Invention

[0006] In order to overcome the shortcomings of the above-mentioned existing technologies, the present invention provides a hierarchical modeling and planning method and system for the working environment of engineering machinery. The hierarchical modeling solution for grasping and path planning solves the contradiction between the waste of high-precision computing power caused by the current single-precision map and the insufficient low-precision modeling, improves the computational efficiency of modeling, and can be expanded to different complex scenarios.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a hierarchical modeling and planning method for an engineering machinery operating environment, comprising: Modeling is performed based on the scanning point cloud data of the construction machinery operation area, the geometric point cloud of objects in the operation area, and the aerial images and aerial point cloud data of the operation area, to obtain the bottom geometric perception layer, the middle semantic planning layer, and the upper topological navigation layer; When an object moves in the working environment of the engineering machinery, the key point cloud of the object surface is obtained and clustered to reconstruct the middle-level semantic planning layer. The grasping path of the engineering machinery manipulator is generated based on the reconstructed middle-level semantic planning layer. Affine transformation is performed on the key point cloud of the object surface after clustering processing, and the upper topological navigation layer is reconstructed. The path of the construction machinery is planned based on the reconstructed upper topological navigation layer.

[0008] In a second aspect, the present invention provides a hierarchical modeling and planning system for an engineering machinery operating environment, comprising: A hierarchical modeling module is configured to: perform modeling based on the scanned point cloud data of the construction machinery operation area, the geometric point cloud of objects in the operation area, and the aerial imagery and aerial point cloud data of the operation area, to obtain a bottom-level geometric perception layer, a middle-level semantic planning layer, and an upper-level topological navigation layer; A grasping module is configured to: when an object moves in the working environment of the engineering machinery, obtain a key point cloud of the object surface, perform clustering processing on it, reconstruct a middle-level semantic planning layer, and generate a grasping path for the engineering machinery manipulator based on the reconstructed middle-level semantic planning layer; The path planning module is configured to: perform affine transformation on the key point cloud of the object surface after clustering processing, reconstruct the upper topological navigation layer, and plan the path of the construction machinery according to the reconstructed upper topological navigation layer.

[0009] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0010] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0011] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0012] One or more of the above technical solutions have the following beneficial effects: In the present invention, modeling is performed based on the scanning point cloud data of the engineering machinery operation area, the geometric point cloud of objects in the operation area, and the aerial images and aerial point cloud data of the operation area, respectively, to obtain the bottom geometric perception layer, the middle semantic planning layer and the upper topological navigation layer. When the object moves, the corresponding model is reconstructed, which solves the contradiction between the waste of high-precision computing power and the insufficient low-precision modeling caused by the current single-precision map, improves the computational efficiency of modeling, and can be extended to different complex scenarios.

[0013] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0015] Figure 1 This is a schematic diagram of the hierarchical modeling structure in the first embodiment of the present invention; Figure 2 Schematic diagram of the update and reconstruction process of each module layer in the first embodiment of the present invention; Figure 3 This is a schematic diagram of modeling the middle semantic planning layer in the first embodiment of the present invention; Figure 4 This is a schematic diagram of modeling the upper topology navigation layer in the first embodiment of the present invention. DETAILED DESCRIPTION

[0016] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0017] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0018] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0019] Example 1 This embodiment discloses a hierarchical modeling and planning method for an engineering machinery operating environment, including: Modeling is performed based on the scanning point cloud data of the construction machinery operation area, the geometric point cloud of objects in the operation area, and the aerial images and aerial point cloud data of the operation area, to obtain the bottom geometric perception layer, the middle semantic planning layer, and the upper topological navigation layer; When an object moves in the working environment of the engineering machinery, the key point cloud of the object surface is obtained and clustered to reconstruct the middle-level semantic planning layer. The grasping path of the engineering machinery manipulator is generated based on the reconstructed middle-level semantic planning layer. Affine transformation is performed on the key point cloud of the object surface after clustering processing, and the upper topological navigation layer is reconstructed. The path of the construction machinery is planned based on the reconstructed upper topological navigation layer.

[0020] This embodiment performs modeling based on the scanning point cloud data of the engineering machinery operation area, the geometric point cloud of objects in the operation area, and the aerial images and aerial point cloud data of the operation area, respectively, to obtain the bottom-level geometric perception layer, the middle-level semantic planning layer, and the upper-level topological navigation layer. When the object moves, the corresponding model is reconstructed, which solves the contradiction between the waste of high-precision computing power and the insufficient low-precision modeling caused by the current single-precision map, improves the computational efficiency of modeling, and can be extended to different complex scenarios.

[0021] The following combination Figures 1-4 The following describes in detail the hierarchical modeling and planning method for the construction machinery operation environment proposed in this embodiment: In this embodiment, the underlying geometric perception layer uses LiDAR and binocular vision to model the detailed environment of the robotic arm's operating area. After LiDAR scans the surface of the cabin wall, ore, etc. to obtain point cloud data, the scanned point cloud preprocessing includes three stages: Statistical filtering is used to remove noise points with a standard deviation exceeding 3σ; Based on the RANSAC algorithm, the ground plane is fitted and non-ground point clouds are extracted for ground segmentation. The residual ground point error is less than 2cm. Using Poisson reconstruction optimization, we set CGAL library parameters such as reconstruction depth = 9 and point weight = 4.0 to generate an 8cm resolution triangular mesh model. The SGBM algorithm using binocular vision then completes occluded areas, compressing the depth error to within 1cm. Using the Poisson surface reconstruction algorithm to generate an 8cm resolution triangular mesh model, binocular vision simultaneously completes the depth information of occluded areas, forming a high-precision geometric model that includes surface normals.

[0022] When the robot arm detects that the end is less than a set distance from an obstacle, such as 30 cm, a 1m×1m local area is updated at a high frequency of 50 Hz, and a real-time collision avoidance vector is calculated based on the surface normal vector. This drives the robot arm joint controller to achieve ±0.8cm precision collision avoidance control.

[0023] The update range of the underlying geometric perception layer is strictly limited to the set radius of the end of the robotic arm, such as a radius of 5 meters. When the distance between the robotic arm and the obstacle is less than 30 centimeters, the update frequency is automatically increased from the basic 10 Hz to 50 Hz, ensuring that the real-time collision avoidance control accuracy error is less than 0.8 centimeters.

[0024] The following is a detailed description of the construction of the underlying geometric perception layer: The LiDAR scans the robotic arm's operating radius of 5 meters at 10Hz, acquiring raw data with a point cloud density of 2000 points / m³. An 8cm precision mesh model is generated using the following steps: Step 11: Point cloud data collection and fusion.

[0025] The three-dimensional point cloud data is obtained by periodically scanning the robotic arm's operating area with a lidar, and a binocular vision system is used to simultaneously collect RGB-D images to build a multi-source sensor data fusion framework.

[0026] Step 12: Point cloud preprocessing and reconstruction.

[0027] (1) Outlier filtering: Remove noise points based on statistical filtering algorithm.

[0028] (2) Ground segmentation: Separate ground and non-ground point clouds through a random sampling consensus algorithm.

[0029] (3) Surface reconstruction: The Poisson surface reconstruction algorithm is used to generate a triangular mesh model with normal vectors.

[0030] Specifically, point cloud surface reconstruction is implemented based on the CGAL library, the reconstruction depth is set to 9, the point weight parameter is set to 4.0, and a triangular mesh model with normal vectors is generated.

[0031] (4) Occlusion completion: By integrating visual depth data, the lidar occlusion area is completed through stereo matching algorithm to improve geometric integrity.

[0032] Specifically, the binocular camera collects RGB-D images and uses OpenCV's SGBM algorithm to complete the lidar occlusion area, with a depth error of <1cm.

[0033] Step 13: Dynamic update mechanism.

[0034] Dynamically adjust the model update frequency based on the real-time distance between the end of the robotic arm and the obstacle: when the distance is greater than the safety threshold, the basic update frequency is used; when the distance approaches the safety threshold, the local area update frequency is automatically increased to generate high-frequency collision avoidance commands; The collision avoidance instructions are calculated based on the normal vector of the triangular mesh surface and output to the robot arm joint controller in real time.

[0035] The middle semantic planning layer extracts geometric features of stacked materials through principal component analysis. For regular objects such as coal piles in ship holds and cargo containers in yards, the minimum bounding box of the point cloud data is calculated and the length, width, and height parameters are output with a tolerance of ±3%. The spatial coordinates of the grasping points are identified. For irregular ore piles in mines, a convex hull algorithm is used to generate a simplified contour model.

[0036] The following is a detailed description of the construction of the middle semantic planning layer: Step 21: Modeling regular objects.

[0037] For regular targets such as coal piles in ship holds, the principal component analysis algorithm is used to calculate the minimum bounding box, extract the eigenvectors of the point cloud covariance matrix, and construct the minimum bounding box coordinate system of the object. The projection coordinates of the extreme points of each axial direction of the bounding box are calculated to obtain the length, width, and height dimensions with a tolerance of ±3%. Combined with the center of mass projection position and the surface curvature distribution, a set of candidate points is constructed on the bounding box surface, and the optimal grasping point is determined with curvature minimization and force closure as the joint optimization goals.

[0038] The grasping point pose is converted into the spatial motion constraint equation of the robot arm joint, and the feasible joint angle range is solved through inverse kinematics.

[0039] For example, taking a 2.2m×1.1m×1.8m cargo box as an example, the coordinates of the grasping point are (1.1, 0.55, 1.8), and the pose constraints are converted into the range of motion of the robot arm joint angle, such as the joint J5 limit of ±120°.

[0040] Step 22: Modeling irregular objects.

[0041] Specifically, the convex hull modeling process includes three stages of optimization: The first step is point cloud preprocessing and feature enhancement. For the original LiDAR point cloud, such as ore pile scanning data, the data volume is first compressed by voxel downsampling (Leaf Size = 5cm), and then normal-based bilateral filtering (parameters: σ_color = 0.1, σ_space = 0.2) is used to retain edge features.

[0042] Aiming at the measured point clouds of irregular objects such as the particle size distribution characteristics of ore (D50=25cm), an improved DBSCAN clustering algorithm with physical constraints is introduced; the improved DBSCAN clustering algorithm performs preliminary segmentation on the ore point cloud; among them, during the preliminary segmentation of the ore point cloud by the improved DBSCAN clustering algorithm, the neighborhood search radius of the clustering algorithm is dynamically adjusted according to the typical particle size distribution characteristics of the material, such as expanding the neighborhood range in large particle size areas to avoid over-segmentation; and narrowing the neighborhood range in small particle size dense areas to improve detail recognition accuracy; in addition, bilateral filtering is used to retain the edge features of the point cloud while suppressing non-structured noise (such as dust drift points).

[0043] Secondly, geometric modeling and physical correction are performed based on the initially segmented point cloud. Specifically, a convex hull model is generated based on the AlphaShape algorithm. The critical collapse angle θ = arctan(μ) = 23° is calculated using the material friction coefficient μ = 0.42. This critical collapse angle is used as the slope safety threshold. An incremental convex hull algorithm is used to introduce physical constraints when constructing the convex hull: when the local slope θ exceeds the slope safety threshold, virtual support points are forcibly added. The spacing between support points is dynamically adjusted based on the average material particle size. The virtual support points are incorporated into the convex hull calculation, expanding the model boundary to form a safe operating buffer zone. For low-rigidity rock masses, such as fracture zones, the point cloud is segmented into sub-regions using the stiffness threshold. Independent sub-clusters are generated and marked as high-risk units.

[0044] Finally, the generated convex hull model is converted into a lightweight engineering interface, and the main skeleton direction of the convex hull, namely the first eigenvector of the PCA, is extracted as the reference axis of the robotic arm grasping, and the safe working distance is calculated in combination with the critical angle of collapse; the convex hull is decomposed into a hierarchy of oriented bounding boxes to improve the efficiency of real-time collision detection; each OBB dynamically adjusts its direction based on the local geometric features of the convex hull, which can tightly wrap the object and reduce the invalid detection area, and the detection efficiency is improved by 5 times compared with the traditional AABB; when the convex hull volume changes by more than 8% for three consecutive frames, the model is updated within 40ms and a new S-shaped trajectory is output, with a curvature radius ≥1.5 times the gripper width.

[0045] By monitoring the distance between the bucket and the material in real time, contact surface slope detection is initiated when entering the operating warning range; if the slope is lower than θ, a straight excavation path is generated, i.e. the energy-optimal mode; if the slope exceeds θ, it automatically switches to the safe operation mode: the measures of the safe operation mode include: increasing the path curvature radius, the increase in the path curvature radius is adaptively scaled based on the material size; reducing the feed speed and injecting vibration suppression instructions.

[0046] For example, when the point cloud registration error is greater than 5%, the ICP algorithm is triggered to rebuild the model within 40ms and output an S-shaped grasping path, and the curvature radius of the S-shaped grasping path is ≥1.2 times the material size; the critical angle drives safety decisions: when the bucket approaches the ore pile, if the distance is less than 1m, the system calculates the contact surface slope θ in real time: if θ<23°: a straight excavation path is generated with optimal energy consumption; if θ>23°: the safety mode is activated, the path curvature radius is increased from the default 1.2m to 2.5m, and the feed speed is reduced to 30% to prevent landslides.

[0047] Step 23: Trigger the rebuild condition.

[0048] When the registration error of three consecutive point clouds is greater than 5% or the size mutation is greater than 8%, the model reconstruction is completed within 40ms.

[0049] For example, the middle semantic planning layer constructs a parametric model at a resolution of 32 cm or 64 cm, and reconstruction is triggered only when the point cloud registration error of the target object exceeds 5% for three consecutive frames or the size mutation exceeds 8%. After reconstruction, an S-shaped robotic arm grasping trajectory with pose constraints is generated.

[0050] The upper topology navigation layer constructs a 128cm-precision environmental topology map based on drone aerial photography or vehicle-mounted wide-angle cameras. Using image recognition, it marks hazardous areas such as leaks, potholes, and slopes, assigning risk weights (e.g., 0.9 for leaks and 0.7 for slopes) and generating safe corridors at least 1.5 meters wide. This layer is updated only when topology changes, such as when new obstacles cover more than 15% of the safe corridor or when the terrain slope changes by more than 5 degrees.

[0051] The following is a detailed description of the construction method of the upper topology navigation layer: Step 31: Use YOLOv5 to identify drain holes in the drone aerial image. If the confidence level is greater than 0.9, a risk weight of 0.9 is assigned. Combined with the point cloud data, a 128cm resolution risk grid map is generated. Step 32: Use the Voronoi diagram algorithm to extract safe paths, convert the risk weight into a distance cost function, and dynamically adjust the corridor width: the initial width is 1.5m, and when the distance between the path and the high-risk area (w>0.7) is less than 3m, it is widened to 2.2m to meet OSHA safety standards.

[0052] The specific implementation is to discretize the 128cm precision topological map into grid cells, such as cell size = 16cm, and linearly interpolate the risk cost of each grid according to the weight; calculate the Euclidean distance from each grid to the nearest risk area, and generate a comprehensive cost based on the weight.

[0053] Traditional Voronoi diagrams only consider geometric distance. This embodiment introduces an improved algorithm with dual risk and distance constraints: a generalized Voronoi skeleton is generated based on grid costs, prioritizing low-risk channels with a risk level w < 0.2. The width is then dynamically adjusted, with an initial corridor width of 1.5 meters. The corridor is automatically widened when the following conditions are met: for example, the path section near the drain hole is widened to 2.2 meters.

[0054] The dynamic adjustment of the safety corridor driven by risk weights includes: 1. Construction of risk semantic map.

[0055] A convolutional neural network (CNN) is used to automatically detect hazardous areas such as leak holes and potholes in aerial images. The slope change rate is calculated based on point cloud data, and a slope weight (W ≥ 0.7) is activated when the slope is greater than 5°. A risk grid map is created, and the risk value of each grid cell is dynamically calculated according to the formula: Risk_value = α·W_object + β·W_slope + γ1 W_material Where: W_object represents the risk weight of the drainage hole and the landslide area; W_slope represents the terrain slope risk weight, and W ≥ 0.7 when the slope is greater than 5°; W_material represents the material intrusion risk weight, and W = 0.8 for scattered materials; α, β, and γ1 are dynamic confidence coefficients, α + β + γ1 ≡ 1, where α is the target recognition confidence of the YOLOv5 model; β is the terrain data reliability, such as when the point cloud slope calculation error is less than 2°, β = 0.3; γ1 is the timeliness of material status updates, such as when the middle semantic layer updates data within 10ms, γ1 = 0.1.

[0056] For example, if a landslide area (W_object=0.95) is superimposed on scattered materials (W_material=0.8), and if α=0.6 and γ1=0.4, the comprehensive risk value = 0.6×0.95 + 0.4×0.8 = 0.89.

[0057] 2. Generation of double-constraint safety corridors.

[0058] The traditional Voronoi diagram relies only on geometric distance. This embodiment introduces a risk-distance cost function: Cost = γ·(1 / Distance) + δ·Risk_value Among them, γ and δ are balance factors, the default values ​​are γ=0.6 and δ=0.4, and Distance is the Euclidean distance from the path point to the nearest hazard source.

[0059] Set the corridor width adaptation rule: the base width is 1.5 meters, which complies with OSHA standards. When a path segment meets the following conditions: Risk_value > 0.7 and Distance_to_risk < 3 meters, the width is adjusted: New_width = Base_width + K (Risk_value - 0.7) Risk_distance, where K is the safety margin factor, which is dynamically optimized through reinforcement learning.

[0060] 3. Real-time replanning mechanism.

[0061] When the new obstacles cover more than 15% of the safety corridor, the Dijkstra algorithm is used for replanning.

[0062] Path cost function injects risk weight: Path_cost = Σ (Segment_length × Max_risk_value) Among them, Segment_length_i represents the length of the i-th path segment. The Euclidean distance is calculated using the grid coordinates of the topological map. For example, if the center distance between two grids is 1.28m, the length = 1.28m. Max_risk_value represents the maximum risk value of the grids covered by the path segment. The Risk_value of all grids in the path segment is dynamically calculated and the maximum value is taken. Path_cost represents the comprehensive risk cost of the entire path, which is the cumulative sum of the costs of each segment.

[0063] The path cost function complements the mechanism of dynamic adjustment of safe corridors driven by risk weights, indicating that this layered map has a real-time replanning mechanism. It will calculate the path cost and determine a new detour route when new obstacles appear in the safe corridor.

[0064] The upper topology navigation layer shares material coordinates with the middle semantic planning layer. When scattered materials are detected invading the corridor, the middle semantic planning layer sends the coordinates of the material's bounding box. The upper topology navigation layer maps the bounding box as a temporary risk source, such as setting the weight W to 0.8. A detour path is output within 40ms, ensuring that the curvature radius is ≥ 2 times the vehicle width.

[0065] (3) The underlying geometric perception layer monitors the changes in the curvature of the bulkhead in real time. When the mutation is greater than 15%, the normal vector collision avoidance command is triggered and the local resource update is locked at the same time.

[0066] The bottom-level geometric perception layer pre-processes the key point cloud of the material surface and transmits it to the middle-level semantic planning layer. The middle-level semantic planning layer maps the center coordinates of the material bounding box to the upper-level topological navigation layer through affine transformation; the upper-level topological navigation layer downsamples the coordinates of the danger zone boundary to a 128 cm grid mask and transmits it to the bottom-level geometric perception layer, where it is merged with the grid of the bottom-level geometric perception layer and high-risk areas such as the area around the drain hole are marked. The bottom-level geometric perception layer receives the new danger zone mask and performs real-time collision avoidance.

[0067] During a ship hold clearing operation, if a robotic arm's grasping caused a coal pile to collapse, the newly reconstructed grasping path by the middle-level semantic planning layer simultaneously triggered the upper-level topological navigation layer to update the unloading area's topology, guiding transport vehicles to avoid the scattered coal pile in real time, forming a closed "perception-planning-execution" loop. The middle-level semantic planning layer is responsible for generating the robotic arm's grasping path, while the upper-level topological navigation layer is solely responsible for vehicle-level navigation path planning.

[0068] In this embodiment, the dynamic computing resource allocation mechanism between each layer is specifically as follows: (1) Edge node computing power scheduling.

[0069] Using Xavier NX edge computing nodes, CPU resource allocation is achieved through weighted core binding: For the robotic arm operation area, if the weight is 0.9, 6 CPU cores are bound to the underlying geometric perception layer for 50Hz updates. If the material change area has a weight of 0.5, it will bind 3 CPU cores to trigger the reconstruction of the middle semantic planning layer. If the weight of the idle area is 0.1, the update of the upper-layer topology navigation layer topology map is suspended.

[0070] The dynamic resource allocation mechanism leverages edge computing nodes for intelligent scheduling. When a cargo clearing robot approaches a bulkhead within 30 centimeters, the system immediately allocates, for example, 70% of CPU power to local updates in the underlying geometric perception layer, suspending non-urgent computations in the mid-layer semantic planning layer and the upper-layer topology navigation layer. When a forklift places a new container, causing a 9% change in the container's contour, 50% of the CPU power is allocated to trigger a parameter model rebuild in the mid-layer semantic planning layer, transmitting the material's dimensional parameters to the upper-layer topology navigation layer within 10 milliseconds. During vehicle navigation, only the upper-layer topology navigation map is loaded, compressing the data to 1 / 256 of the underlying grid. A resource scheduling strategy based on spatial location weights, such as a 0.9 weight for the robot's operating area and a 0.1 weight for idle areas, reduces the system's peak computing power consumption by 80%.

[0071] (2) Data update logic: Update the underlying geometric perception layer: The lidar data is partitioned using the KD-Tree space, and only the 90° fan-shaped area in front of the robotic arm with a radius of 5m is updated. Middle-layer semantic planning layer reconstruction: When the coal pile height change Δh>8%, the ICP algorithm of PCL is called for point cloud registration; Upper-layer topology navigation layer synchronization: Topology change messages are published through the ROS Topic mechanism, such as when the coverage rate of new obstacles is greater than 15%.

[0072] This embodiment builds collaborative work between multiple layers of models, specifically including: (1) Data transfer between layers Data communication between layers is achieved through lightweight protocols: Bottom-layer geometric perception layer → middle-layer semantic planning layer: Extract key point cloud clusters on the material surface through improved DBSCAN clustering (eps=0.2m, min_samples=5). The compressed transmission data volume is only 12% of the original point cloud. Middle semantic planning layer → upper topological navigation layer: The center coordinates of the bounding box are mapped to the topological map through affine transformation: Upper topological navigation layer → underlying geometric perception layer: The coordinates of the danger zone are downsampled to a 128cm grid mask and fused to the underlying grid through a bitwise AND operation.

[0073] For example, the closed-loop control process in the cabin cleaning operation is specifically manifested as follows: (1) The robotic arm grasping causes the coal pile to collapse → The middle-level semantic planning layer reconstructs the model based on the new point cloud and generates an S-shaped path with continuous curvature. The Bezier curve control point density is 5 points / m.

[0074] (2) After receiving the material coordinates, the upper topology navigation layer uses the Dijkstra algorithm to replan the vehicle path and introduces risk weights to dynamically adjust the safety corridor.

[0075] For example, in a ship cabin cleaning operation, after the robotic arm enters the cabin, its binocular vision system scans the bulkhead within 5 meters at 50Hz, generating a surface mesh model with 8cm accuracy. When the robotic arm is 28cm from a rusted pipe, the system detects a sudden change in the pipe's surface curvature exceeding 15%. It immediately generates a normal vector collision avoidance command, ensuring the arm's detour distance is precisely controlled within 0.8cm. Simultaneously, the vehicle's onboard LiDAR monitors the coal pile's contour. When a grasping operation causes the pile's height to drop by 12%, the mid-level semantic layer reconstructs a parametric model of 3.2m × 2.1m × 1.4m (±3%) in 40ms and plans the suction cup grasping trajectory. The transport vehicle, loaded with only a 128cm topological map marking the drain hole locations, navigates within a 2.2m-wide safety corridor, maintaining a CPU usage of less than 20% throughout the entire process.

[0076] For example, for a material stacking scenario in a yard, in the initial mapping phase: 1. Use drones to generate a 128cm topological map of the yard, marking the unloading area (risk 0.2) and the equipment area (risk 0.8); 2. A forklift-mounted LiDAR constructs a 64cm accurate cargo box model (size 2.2m×1.1m×1.8m±3%).

[0077] In the real-time operation phase, when a forklift places a new container: ① The middle layer detected contour changes by 9%, and the model was rebuilt within 40ms; ② The vertical error of the generated grasping path is less than 2cm (satisfying the unloading height constraint of 3.2m±1cm); ③ The upper layer dynamically adjusts the safety corridor to 2.5m (the original path overlaps with the cargo box by 35%).

[0078] The drone initially generates a 128cm topological map of the yard, identifying the unloading area (risk weight 0.2) and the equipment storage area (risk weight 0.8). When the unmanned forklift places a new container (2.2m × 1.1m × 1.8m), the middle semantic layer detects a 9% change in the contour and immediately reconstructs the parameter model and calculates the coordinates of the grasping points. The robotic arm then generates an S-shaped trajectory for precise grasping. The container's dimensional parameters are simultaneously uploaded to the upper-layer topological map. If the system detects >35% overlap between the container and the construction machinery path, it automatically widens the safety corridor to 2.5 meters.

[0079] For example, in a mining excavator operation scenario, when the excavator bucket approaches a rock wall, the bottom geometry layer updates a 5cm-precision rock fracture mesh at 30Hz, guiding the bucket to avoid fracture zones with strengths below 35MPa. The middle semantic layer models the ore pile as a convex hull to plan the optimal excavation trajectory. The upper topological layer marks landslide areas (risk weight 0.95) with 256cm accuracy, guiding vehicles to detour around a 3km dangerous section. Field tests have shown that this system has increased the excavator's effective operating time by 40% and reduced daily diesel consumption by 10L.

[0080] Example 2 The purpose of this embodiment is to provide a hierarchical modeling and planning system for an engineering machinery operating environment, including: A hierarchical modeling module is configured to: perform modeling based on the scanned point cloud data of the construction machinery operation area, the geometric point cloud of objects in the operation area, and the aerial imagery and aerial point cloud data of the operation area, to obtain a bottom-level geometric perception layer, a middle-level semantic planning layer, and an upper-level topological navigation layer; A grasping module is configured to: when an object moves in the working environment of the engineering machinery, obtain a key point cloud of the object surface, perform clustering processing on it, reconstruct a middle-level semantic planning layer, and generate a grasping path for the engineering machinery manipulator based on the reconstructed middle-level semantic planning layer; The path planning module is configured to: perform affine transformation on the key point cloud of the object surface after clustering processing, reconstruct the upper topological navigation layer, and plan the path of the construction machinery according to the reconstructed upper topological navigation layer.

[0081] In further embodiments, there is also provided: An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed by the processor. When the computer instructions are executed by the processor, the method described in Example 1 is performed. For the sake of brevity, no further details are given here.

[0082] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0083] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0084] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in embodiment 1 is performed.

[0085] The method in Example 1 can be directly implemented as being executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software module can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0086] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in embodiment 1 is implemented.

[0087] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions contained in program modules, which are executed in a device on a real or virtual processor of a target to perform the process / method described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided between program modules as needed. The machine-executable instructions for the program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0088] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the computer or other programmable data processing device, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0089] In the context of the present invention, computer program code or related data can be carried by any appropriate carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals include electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0090] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0091] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A hierarchical modeling and planning method for an engineering machinery operating environment, characterized in that: include: Modeling is performed based on the scanning point cloud data of the construction machinery operation area, the geometric point cloud of objects in the operation area, and the aerial images and aerial point cloud data of the operation area, to obtain the bottom geometric perception layer, the middle semantic planning layer, and the upper topological navigation layer; When an object moves in the working environment of the engineering machinery, the key point cloud of the object surface is obtained and clustered to reconstruct the middle-level semantic planning layer. The grasping path of the engineering machinery manipulator is generated based on the reconstructed middle-level semantic planning layer. Affine transformation is performed on the key point cloud of the object surface after clustering processing, and the upper topological navigation layer is reconstructed. The path of the construction machinery is planned based on the reconstructed upper topological navigation layer.

2. The method for hierarchical modeling and planning of an engineering machinery operating environment according to claim 1, wherein: The middle semantic planning layer is obtained by modeling the geometric point cloud of objects in the working area, specifically: Obtain point cloud data of a regular object, use the principal component analysis algorithm to extract the eigenvectors of the point cloud data covariance matrix, and construct the minimum bounding box of the object; calculate the projection coordinates of the extreme points of each axial direction of the bounding box to obtain the object size information and obtain the contour model of the regular object; Obtain point cloud data of irregular objects, use convex hull algorithm to generate convex hull model, dynamically calculate the critical angle threshold of collapse according to the material friction coefficient, and generate a straight path or safe path for the end of the robotic arm based on the distance between the end of the robotic arm and the material and the critical angle threshold of collapse.

3. The method for hierarchical modeling and planning of an engineering machinery operating environment according to claim 1, wherein: Modeling is performed based on the aerial images and aerial point cloud data of the operating area to obtain the upper topology navigation layer, specifically: Based on the acquired aerial images of the work area, the trained recognition model is used to obtain the hazard source identification results. Based on the hazard source identification results and the corresponding aerial point cloud, a risk grid map is generated; An improved Voronoi algorithm is used to extract a safe path for construction machinery on the risk grid map; wherein the improved Voronoi algorithm introduces a risk-distance cost function to dynamically adjust the width of the safe path.

4. The method for hierarchical modeling and planning of an engineering machinery operating environment according to claim 3, wherein: The risk value of each risk grid map is determined based on the risk weight of the hazard source and the corresponding identification confidence level; The risk-distance cost function is determined based on the risk value of each risk grid map and the distance from the path point to the nearest hazard source.

5. The method for hierarchical modeling and planning of an engineering machinery operating environment according to claim 2, wherein: When using the convex hull algorithm to generate the convex hull model, if the local slope is greater than the critical angle threshold of landslide, virtual support points are added to expand the convex hull boundary to form a safety margin.

6. The method for hierarchical modeling and planning of an engineering machinery operating environment according to claim 1, wherein: The grasping path of the engineering machinery manipulator is generated based on the reconstructed middle-level semantic planning layer, specifically: Combining the center of mass projection position and surface curvature distribution, a candidate point set is constructed on the bounding box surface, and the optimal grasping point is determined with curvature minimization and force closure as the joint optimization objectives; The grasping point pose is converted into the spatial motion constraint equation of the robot arm joint, and the feasible joint angle range is solved through inverse kinematics.

7. A hierarchical modeling and planning system for an engineering machinery operating environment, characterized in that: include: A hierarchical modeling module is configured to: perform modeling based on the scanned point cloud data of the construction machinery operation area, the geometric point cloud of objects in the operation area, and the aerial imagery and aerial point cloud data of the operation area, to obtain a bottom-level geometric perception layer, a middle-level semantic planning layer, and an upper-level topological navigation layer; A grasping module is configured to: when an object moves in the working environment of the engineering machinery, obtain a key point cloud of the object surface, perform clustering processing on it, reconstruct a middle-level semantic planning layer, and generate a grasping path for the engineering machinery manipulator based on the reconstructed middle-level semantic planning layer; The path planning module is configured to: perform affine transformation on the key point cloud of the object surface after clustering processing, reconstruct the upper topological navigation layer, and plan the path of the construction machinery according to the reconstructed upper topological navigation layer.

8. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which is used to implement the method according to any one of claims 1 to 6 when the computer program is executed by a processor.

Images