Ground separation method, device and loading operation system

Abstract

The application relates to a ground separation method and device and a loading operation system, and relates to the technical field of map processing.The ground separation method comprises the following steps: obtaining a global voxel point cloud map and a global grid point cloud map according to an initial map; obtaining pose information of each frame of scanning point cloud in a global coordinate system according to the global voxel point cloud map; outputting first information or second information according to the pose information of the current frame of scanning point cloud; if the first information is output, a current frame of local dynamic area grid point cloud is obtained by updating, and a dynamic area target ground point cloud and a dynamic area target non-ground point cloud are separated; and if the second information is output, a global target ground point cloud and a global target non-ground point cloud are separated.The ground separation method, the device and the loading operation system can separate the drivable area ground in a global working environment, and can also update and separate the material features stacked in a local dynamic area.

Description

Technical Field

[0001] This application relates to the field of map processing technology, specifically to a ground separation method, apparatus, and material feeding system. Background Technology

[0002] Currently, robots utilize LiDAR and various sensors to collect information about their working environment. This collected information provides accurate decision-making support for subsequent intelligent safety operations. During operation, robots typically encounter two scenarios: First, in a static environment, the robot needs to detect the boundaries of the drivable area, separate the drivable ground, and plan its overall path for safe movement. Second, the shape and size of materials piled in the working environment may change during operation, leading to localized dynamic changes in the material storage area. In other words, in situations with dynamic localized areas, the robot also needs to separate the ground from materials of varying shapes and sizes to find better digging points and improve work efficiency. For these two scenarios, current robot technologies cannot effectively handle them in a single pass, requiring repeated passes and impacting efficiency. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a ground separation method, apparatus, and material loading system, which can assist a robot in separating the drivable ground area in the overall working environment during a single trip, and also update and separate the drivable ground area and material stacking characteristics of the local dynamic area in local dynamic scenarios.

[0004] Firstly, a ground separation method is provided, including:

[0005] Obtain an initial map; wherein the initial map represents a preset working environment map;

[0006] Based on the initial map, a global voxel point cloud map and a global raster point cloud map are obtained; wherein, the global raster point cloud map represents the remaining raster point cloud map after deleting the initial local dynamic region raster point cloud.

[0007] Based on the global voxel point cloud map, the pose information of each frame of scanned point cloud in the global coordinate system is obtained; wherein, the scanned point cloud represents the point cloud obtained by the lidar in real time scanning in the working environment;

[0008] Based on the pose information of the scanned point cloud in the current frame, output first information indicating that the lidar is located within a local dynamic region or output second information indicating that the lidar is located outside the local dynamic region;

[0009] If the first information is output, based on the global grid point cloud map, the scanned point cloud of the current frame and the pose information of the scanned point cloud of the current frame, the local dynamic region grid point cloud of the current frame in the global grid point cloud map is updated, and the dynamic region target ground point cloud and the dynamic region target non-ground point cloud are separated from the local dynamic region grid point cloud of the current frame.

[0010] If the second information is output, the global target ground point cloud and the global target non-ground point cloud are separated based on the global voxel point cloud map and the scanned point cloud of the current frame.

[0011] According to a first aspect of this application, separating the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the current frame local dynamic region raster point cloud includes:

[0012] An initial ground point cloud is extracted from the global voxel point cloud map and / or the transition raster point cloud map; wherein, the transition raster point cloud map represents the global raster point cloud map before the initial local dynamic region raster point cloud is deleted.

[0013] Based on the initial ground point cloud, the equation of the target plane is obtained by fitting.

[0014] Based on the target plane equation and the preset horizontal plane equation, the local dynamic region raster point cloud of the current frame is horizontally rotated and calibrated.

[0015] From the calibrated local dynamic region raster point cloud of the current frame, the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud are separated.

[0016] Based on the target plane equation and the preset horizontal plane equation, the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud are horizontally rotated, calibrated, and restored to obtain the dynamic region target ground point cloud and the dynamic region target non-ground point cloud.

[0017] According to a first aspect of this application, separating the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud from the calibrated local dynamic region raster point cloud of the current frame includes:

[0018] Based on the target plane equation and the preset horizontal plane equation, a distance parameter is obtained; wherein, the distance parameter represents the distance between the plane represented by the target plane equation after horizontal calibration and the preset horizontal plane;

[0019] Based on the distance parameter and the height coordinates of each point in the local dynamic region raster point cloud of the current frame, the initial ground seed point set and the initial non-ground seed point set that meet the first filtering condition are selected.

[0020] The initial ground seed point set is subjected to secondary screening to obtain the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud.

[0021] According to a first aspect of this application, the secondary filtering of the initial ground seed point set to obtain the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud includes:

[0022] Based on the initial set of ground seed points, a transition plane is obtained by fitting.

[0023] From the initial ground seed point set, a target ground seed point set and a target non-ground seed point set that satisfy the second screening condition are obtained;

[0024] The point cloud represented by the target ground seed point set is output as the dynamic region transition ground point cloud;

[0025] The initial non-ground seed point set and the target non-ground seed point set are used to represent the point cloud as the output of the dynamic region transition non-ground point cloud.

[0026] According to a first aspect of this application, updating the local dynamic region grid point cloud in the global grid point cloud map based on the global grid point cloud map, the scanned point cloud of the current frame, and the pose information of the scanned point cloud of the current frame includes:

[0027] Rasterize the scanned point cloud of the current frame to obtain the global raster point cloud of the current frame;

[0028] Based on the global grid point cloud map, the pose information of the scanned point cloud in the current frame, and the global grid point cloud of the current frame, the local dynamic region grid point cloud of the current frame in the global grid point cloud map is updated.

[0029] According to a first aspect of this application, separating the global target ground point cloud and the global target non-ground point cloud includes:

[0030] Rasterize the scanned point cloud of the current frame to obtain the global raster point cloud of the current frame;

[0031] An initial ground point cloud is extracted from the global voxel point cloud map and / or the transition raster point cloud map; wherein, the transition raster point cloud map represents the global raster point cloud map before the initial local dynamic region raster point cloud is deleted.

[0032] Based on the initial ground point cloud, the equation of the target plane is obtained by fitting.

[0033] Based on the target plane equation and the preset horizontal plane equation, the current frame global grid point cloud is horizontally rotated and calibrated.

[0034] From the calibrated global raster point cloud of the current frame, the global transition ground point cloud and the global transition non-ground point cloud are separated.

[0035] Based on the target plane equation and the preset horizontal plane equation, the global transition ground point cloud and the global transition non-ground point cloud are horizontally rotated, calibrated, and restored to obtain the global target ground point cloud and the global target non-ground point cloud.

[0036] According to a first aspect of this application, separating the global transition ground point cloud and the global transition non-ground point cloud from the calibrated current frame global raster point cloud includes:

[0037] Based on the target plane equation and the preset horizontal plane equation, a distance parameter is obtained; wherein, the distance parameter represents the distance between the plane represented by the target plane equation after horizontal calibration and the preset horizontal plane;

[0038] Based on the distance parameter and the height coordinates of each point in the current frame global raster point cloud, the initial ground seed point set and the initial non-ground seed point set that meet the first filtering condition are selected.

[0039] The initial ground seed point set is subjected to secondary screening to obtain the global transition ground point cloud and the global transition non-ground point cloud.

[0040] According to a first aspect of this application, the secondary filtering of the initial ground seed point set to obtain the global transition ground point cloud and the global transition non-ground point cloud includes:

[0041] Based on the initial set of ground seed points, a transition plane is obtained by fitting.

[0042] From the initial ground seed point set, a target ground seed point set and a target non-ground seed point set that satisfy the second screening condition are obtained;

[0043] The point cloud represented by the target ground seed point set is output as the global transition ground point cloud;

[0044] The initial non-ground seed point set and the target non-ground seed point set are used to represent the point cloud as the global transition non-ground point cloud output.

[0045] According to a first aspect of this application, obtaining a global voxel point cloud map and a global raster point cloud map based on the initial map includes:

[0046] The point cloud in the initial map is downsampled and filtered to obtain the global voxel point cloud map;

[0047] The point cloud in the initial map is rasterized, filtered, and the initial local dynamic region raster point cloud is deleted to obtain the global raster point cloud map.

[0048] According to a first aspect of this application, obtaining the pose information of each frame of scanned point cloud in the global coordinate system based on the global voxel point cloud map includes:

[0049] The first frame of the scanned point cloud is matched with the global voxel point cloud map to relocate the current starting position of the lidar and obtain the initial pose of the lidar in the global coordinate system.

[0050] Based on the scanned point cloud obtained from each frame of the lidar scan and the initial pose of the lidar, the pose information of the scanned point cloud in the global coordinate system for each frame is obtained.

[0051] Secondly, a ground separation device is also provided, comprising:

[0052] The first acquisition module is configured to acquire an initial map; wherein the initial map represents a preset working environment map;

[0053] The first processing module is configured to obtain a global voxel point cloud map and a global raster point cloud map based on the initial map; wherein the global raster point cloud map represents the remaining raster point cloud map after deleting the initial local dynamic region raster point cloud.

[0054] The first conversion module is configured to obtain the pose information of each frame of scanned point cloud in the global coordinate system based on the global voxel point cloud map; wherein, the scanned point cloud represents the point cloud obtained by the lidar in real time in the working environment;

[0055] The first output module is configured to output first information indicating that the lidar is located within a local dynamic region or output second information indicating that the lidar is located outside the local dynamic region, based on the pose information of the scanned point cloud in the current frame.

[0056] The first separation module is configured to, if the first information is output, update the local dynamic region grid point cloud in the global grid point cloud map based on the global grid point cloud map, the scanned point cloud in the current frame and the pose information of the scanned point cloud in the current frame, and separate the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the local dynamic region grid point cloud in the current frame.

[0057] The second separation module is configured to, if the second information is output, separate the global target ground point cloud and the global target non-ground point cloud based on the global voxel point cloud map and the scanned point cloud of the current frame.

[0058] Thirdly, a material feeding system is also provided, including:

[0059] The robot is equipped with a lidar system.

[0060] As described in the previous embodiment, the ground separation device is communicatively connected to the lidar.

[0061] Fourthly, an electronic device is also provided, comprising:

[0062] processor;

[0063] And a memory for storing the processor's executable instructions;

[0064] The processor is used to execute the ground separation method described in the above embodiments.

[0065] Fifthly, a computer-readable storage medium is also provided, the storage medium storing a computer program for executing the ground separation method described in the above embodiments.

[0066] The ground separation method, apparatus, loading system, electronic device, and computer-readable storage medium provided in this application embodiment acquire an initial map, then obtain a global voxel point cloud map and a global grid point cloud map based on the initial map, then obtain the pose information of each frame of scanned point cloud in the global coordinate system based on the global voxel point cloud map, and then output first information indicating that the lidar is located within a local dynamic region or second information indicating that the lidar is located outside the local dynamic region based on the pose information of the current frame scanned point cloud; in the first aspect, when outputting the first information, the local dynamic region grid point cloud of the current frame in the global grid point cloud map can be updated based on the global grid point cloud map, the current frame scanned point cloud, and the pose information of the current frame scanned point cloud, and the dynamic region grid point cloud can be separated from the local dynamic region grid point cloud of the current frame. The system separates the target ground point cloud and the dynamic target non-ground point cloud. The dynamic target non-ground point cloud can be considered as the material piled up in a local dynamic area. By separating the dynamic target ground point cloud and the dynamic target non-ground point cloud, the robot can easily detect changes in material characteristics within the local dynamic area, thus facilitating the subsequent finding of better material digging points and improving material loading efficiency. Secondly, when outputting the second information, the global target ground point cloud and the global target non-ground point cloud can be separated based on the global voxel point cloud map and the current frame scan point cloud. The global target ground point cloud can be considered as the global drivable area ground in the working scene. This allows the robot to easily detect the drivable area ground, which serves as the basis for subsequent overall path planning and safe driving decisions, effectively improving driving safety. In summary, this ground separation method, device, loading operation system, electronic equipment, and computer-readable storage medium can assist the robot in separating the drivable area ground in the global working environment during a single descent, and also in updating and separating the drivable area ground and material pile characteristics in a local dynamic scene. Attached Figure Description

[0067] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0068] Figure 1 This is a schematic flowchart of a ground separation method provided for an exemplary embodiment of this application.

[0069] Figure 2This is a schematic diagram illustrating the process of separating the ground point cloud of the dynamic region target and the non-ground point cloud of the dynamic region target from the local dynamic region raster point cloud of the current frame, as provided in an exemplary embodiment of this application.

[0070] Figure 3 This is a schematic diagram illustrating the process of separating dynamic region transition ground point cloud and dynamic region transition non-ground point cloud from the calibrated local dynamic region raster point cloud of the current frame, as provided in an exemplary embodiment of this application.

[0071] Figure 4 This is a schematic diagram illustrating the process of performing secondary filtering on an initial ground seed point set to obtain dynamic region transition ground point clouds and dynamic region transition non-ground point clouds, provided as an exemplary embodiment of this application.

[0072] Figure 5 This is a schematic diagram illustrating the process of updating the local dynamic region grid point cloud in the current frame of the global grid point cloud map, provided for an exemplary embodiment of this application.

[0073] Figure 6 This is a schematic diagram illustrating the process of separating the global target ground point cloud and the global target non-ground point cloud, provided as an exemplary embodiment of this application.

[0074] Figure 7 This is a schematic diagram illustrating the process of separating a global transition ground point cloud and a global transition non-ground point cloud from a calibrated current frame global raster point cloud, provided as an exemplary embodiment of this application.

[0075] Figure 8 This is a schematic diagram illustrating the process of performing secondary filtering on an initial ground seed point set to obtain a global transition ground point cloud and a global transition non-ground point cloud, provided as an exemplary embodiment of this application.

[0076] Figure 9 This is a schematic diagram illustrating the process of obtaining a global voxel point cloud map and a global raster point cloud map based on an initial map, provided as an exemplary embodiment of this application.

[0077] Figure 10 The embodiment of this application provides that the pose information of each frame of scanned point cloud in the global coordinate system is obtained based on the global voxel point cloud map.

[0078] Figure 11 This is a structural block diagram of a ground separation device provided for an exemplary embodiment of this application.

[0079] Figure 12 A structural block diagram of a ground separation device provided for another exemplary embodiment of this application.

[0080] Figure 13 A structural block diagram of a material feeding system provided for an exemplary embodiment of this application.

[0081] Figure 14 A structural block diagram of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0082] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0083] Figure 1 This is a schematic flowchart illustrating a ground separation method provided as an exemplary embodiment of this application. Figure 1 As shown, the ground separation method provided in this application embodiment may include:

[0084] S310: Get the initial map.

[0085] Specifically, the initial map can be understood as a pre-defined working environment map. In practical applications, the pre-defined working environment map can be used to build a prior map using SLAM technology, and then the initial map can be obtained by adaptively processing the prior map.

[0086] SLAM (Simultaneous Localization and Mapping) technology refers to the process by which a robot starts from an unknown location in an unknown environment, locates its own position and orientation by repeatedly observing environmental features during its movement, and then builds an incremental map of the surrounding environment based on its own position, thereby achieving the goal of simultaneous localization and mapping.

[0087] Specifically, before starting work, the robot can be equipped with LiDAR and various other sensors. It starts from the initial position of the working environment (i.e., the robot's current parking area) and moves according to the predetermined main orientation (usually the head orientation when parked). Using LiDAR, IMU (Inertial Measurement Unit), and other sensor data as input, a prior map is built through SLAM technology. Then, the prior map is cropped (scenes outside the working environment are cropped out, and the required working scene is retained) to obtain the aforementioned initial map.

[0088] S320: Based on the initial map, obtain the global voxel point cloud map and the global raster point cloud map.

[0089] Specifically, by performing point cloud downsampling (e.g., using voxel filtering) and filtering operations on the initial map, a global voxel point cloud map can be obtained. It should be noted that the definition of a voxel point cloud map is documented in relevant technologies and will not be repeated here.

[0090] Specifically, by performing operations such as 3D rasterization of the point cloud, filtering (i.e. removing noise points, useless points, etc., for example, statistical filtering, radius filtering, etc.) and deleting local dynamic area raster point clouds on the initial map, a global raster point cloud map can be obtained.

[0091] It's important to note that the global raster point cloud map can be understood as the raster point cloud map remaining after deleting the initial local dynamic region raster point cloud. That is, before obtaining the global raster point cloud map, the boundaries of the local dynamic region are first defined based on the specific working environment. Then, the initial local dynamic region raster point cloud within these boundaries is deleted, and the global raster point cloud map is obtained. This allows for convenient real-time updates of the point cloud within the local dynamic region during subsequent operations, reflecting changes in the scene within that region.

[0092] S330: Based on the global voxel point cloud map, obtain the pose information of the scanned point cloud in the global coordinate system for each frame.

[0093] Specifically, a scanned point cloud can be understood as a point cloud obtained by a lidar in real time during operation. Taking the current frame scanned point cloud as an example, after obtaining the current frame scanned point cloud, it can be filtered, and then the filtered point cloud can be transformed to obtain the pose information of the current frame scanned point cloud in the global coordinate system.

[0094] It should be noted that during the calculation of pose information, LIO front-end odometry technology can be used to calculate the pose information of each point in the scanned point cloud in each frame in the global coordinate system.

[0095] Front-end odometry (LIO) is a point cloud matching-based technique used to estimate a robot's pose. It achieves this by accumulating poses through matching. Upon receiving a frame of point cloud data, it first performs a matching operation, comparing it with a map. Before matching, filtering is performed to sparsify the point cloud data, reducing the computational load of the matching process.

[0096] It should be noted that the global coordinate system is a coordinate system established based on the global area of ​​the working environment. The initial coordinate system of each frame of scanned point cloud obtained by the LiDAR is based on the coordinate system of the LiDAR itself. Therefore, by executing step S330, each frame of scanned point cloud can be transformed into the global coordinate system to obtain the pose information of each frame of scanned point cloud in the global coordinate system.

[0097] It should be noted that the pose information of each frame of the scanned point cloud in the global coordinate system can include the pose parameters such as the coordinates and orientation angles of each point in the scanned point cloud in the global coordinate system.

[0098] S340: Based on the pose information of the scanned point cloud in the current frame, output first information indicating that the lidar is located within the local dynamic region or output second information indicating that the lidar is located outside the local dynamic region.

[0099] Specifically, based on the pose information of the point cloud scanned in the current frame, the specific location of the LiDAR in the working environment can be confirmed, thereby determining whether the LiDAR has moved with the robot into the local dynamic area. In practical applications, if the LiDAR is located within the local dynamic area, then the first piece of information can be output; if the LiDAR is located outside the local dynamic area, then the second piece of information can be output.

[0100] S350: If the first information is output, based on the global grid point cloud map, the current frame scan point cloud, and the pose information of the current frame scan point cloud, update the current frame local dynamic region grid point cloud in the global grid point cloud map, and separate the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the current frame local dynamic region grid point cloud.

[0101] Specifically, since the initial local dynamic region grid point cloud is deleted from the global grid point cloud map, when the lidar is located within the local dynamic region, the grid point cloud in the missing local dynamic region in the global grid point cloud map can be updated based on the current frame scan point cloud and its pose information obtained by the lidar scan, thus obtaining the current frame local dynamic region grid point cloud.

[0102] It should be understood that the local dynamic region grid point cloud of the current frame can reflect the current working conditions within the local dynamic region, that is, it can reflect the characteristics of the currently stacked materials within the local dynamic region. Then, the dynamic region target ground point cloud and the dynamic region target non-ground point cloud can be separated from the local dynamic region grid point cloud of the current frame.

[0103] It should be noted that the ground point cloud representing the target in the dynamic region can be understood as the ground of the drivable area within the local dynamic region; the non-ground point cloud representing the target in the dynamic region can be understood as the material piled up within the local dynamic region. As the shape, size and other characteristics of the material change, the non-ground point cloud representing the target in the dynamic region will constantly change, which requires updating the local dynamic region grid point cloud of the current frame in the aforementioned global grid point cloud map.

[0104] It should be understood that executing step S350, separating the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the local dynamic region grid point cloud of the current frame, can facilitate the robot to detect changes in material characteristics within the local dynamic region, thereby facilitating the subsequent finding of better material digging points and improving material loading efficiency.

[0105] S360: If the second information is output, the global target ground point cloud and the global target non-ground point cloud are separated based on the global voxel point cloud map and the current frame scan point cloud.

[0106] Specifically, the global voxel point cloud map includes all the required parts of the work scene. The current frame scan point cloud is matched with the global voxel point cloud to separate the global target ground point cloud and the global target non-ground point cloud (the specific process will be described in detail later). The global target ground point cloud can be considered as the globally drivable ground area in the work scene; the global target non-ground point cloud can be considered as the globally non-drivable area in the work scene.

[0107] It should be understood that executing step S360 to obtain the global target ground point cloud and the global target non-ground point cloud can facilitate the robot's detection of the drivable ground area, thus serving as the basis for subsequent overall path planning and safe driving decisions, which can effectively improve driving safety.

[0108] The ground separation method provided in this application involves acquiring an initial map, then obtaining a global voxel point cloud map and a global grid point cloud map based on the initial map. Next, based on the global voxel point cloud map, the pose information of the scanned point cloud in each frame in the global coordinate system is obtained. Then, based on the pose information of the scanned point cloud in the current frame, first information indicating that the lidar is located within a local dynamic region or second information indicating that the lidar is located outside the local dynamic region is output. In the first aspect, when the first information is output, the local dynamic region grid point cloud in the current frame can be updated in the global grid point cloud map based on the global grid point cloud map, the scanned point cloud in the current frame, and the pose information of the scanned point cloud in the current frame. The target ground point cloud and the dynamic region are then separated from the local dynamic region grid point cloud in the current frame. The target non-ground point cloud and the dynamic region target non-ground point cloud can be considered as materials piled up in a local dynamic area. By separating the dynamic region target ground point cloud and the dynamic region target non-ground point cloud, the robot can easily detect changes in material characteristics within the local dynamic area, thus facilitating the subsequent finding of better material scooping points and improving material loading efficiency. Secondly, when outputting the second information, the global target ground point cloud and the global target non-ground point cloud can be separated based on the global voxel point cloud map and the current frame scan point cloud. The global target ground point cloud can be considered as the global drivable area ground in the working scene. This allows the robot to easily detect the drivable area ground, which serves as the basis for subsequent overall path planning and safe driving decisions, effectively improving driving safety. In summary, this ground separation method can assist the robot in separating the drivable area ground in the global working environment during a single descent, and also in updating and separating the drivable area ground and material pile characteristics in a local dynamic scene.

[0109] Figure 2 This is a schematic diagram illustrating a process for separating the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the local dynamic region raster point cloud of the current frame, as provided in an exemplary embodiment of this application. Figure 2 As shown, S350 includes:

[0110] S351: Extract the initial ground point cloud from the global voxel point cloud map and / or the transition raster point cloud map.

[0111] It should be noted that the transition raster point cloud map can be understood as the global raster point cloud map before the initial local dynamic region raster point cloud is deleted. In other words, both the global voxel point cloud map and the transition raster point cloud map include all the required parts of the working scene. Therefore, the initial ground point cloud in the global working scene can be extracted through the global voxel point cloud map and / or the transition raster point cloud map.

[0112] Specifically, the initial ground point cloud can be extracted by filtering the point cloud in the global voxel point cloud map and / or the transition raster point cloud map. The specific filtering process is described in relevant technologies and will not be repeated here.

[0113] S352: Based on the initial ground point cloud, the equation of the target plane is obtained by fitting.

[0114] Specifically, the initial ground point cloud includes multiple points. By fitting and calculating these multiple points, the equation of the target plane can be obtained. The specific fitting and calculation methods are documented in relevant technologies and will not be elaborated here.

[0115] S353: Perform horizontal rotation calibration on the local dynamic region raster point cloud of the current frame according to the target plane equation and the preset horizontal plane equation.

[0116] It should be noted that the preset horizontal plane equation can be considered as the equation of the preset horizontal plane in the current working environment.

[0117] Specifically, by calculating the target plane equation and the preset horizontal plane equation, the rotation matrix between the target plane equation and the preset horizontal plane equation can be obtained, which can confirm the tilt attitude of the ground relative to the preset horizontal plane represented by the target plane equation.

[0118] It should be noted that since the tilt attitude of the ground represented in the local dynamic region grid point cloud of the current frame relative to the preset horizontal plane is consistent with the tilt attitude of the ground represented by the aforementioned target plane equation relative to the preset horizontal plane, the local dynamic region grid point cloud of the current frame can be horizontally rotated and calibrated first according to the rotation matrix calculated above. This makes the ground represented in the local dynamic region grid point cloud of the current frame parallel to the preset horizontal plane. In the subsequent process of separating the ground point cloud from the local dynamic region grid point cloud of the current frame, the error influence caused by the uneven height of the local dynamic region grid point cloud of the current frame on the ground point cloud separation process can be eliminated, which is beneficial to improving the accuracy and stability of the separation results.

[0119] S354: Separate the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud from the calibrated local dynamic region raster point cloud of the current frame.

[0120] Specifically, in step S354, the GPF segmentation algorithm can be used to separate the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud. The specific separation process will be described in detail later.

[0121] The GPF segmentation algorithm is a point cloud data segmentation algorithm that segments ground and non-ground point clouds by estimating the normal vector of each point. The algorithm selects initial seed points using a set height coordinate threshold (z-coordinate threshold) and then uses these seed points to solve a planar model.

[0122] S355: Based on the target plane equation and the preset horizontal plane equation, perform horizontal rotation calibration and restoration on the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud to obtain the dynamic region target ground point cloud and the dynamic region target non-ground point cloud.

[0123] Specifically, by separating the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud, and using the inverse matrix of the rotation matrix calculated by the target plane equation and the preset horizontal plane equation, the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud can be horizontally rotated and calibrated to restore them. In this way, the restored dynamic region target ground point cloud and dynamic region target non-ground point cloud can meet the true posture in the global coordinate system, and obtain the updated true working condition in the local dynamic region, which helps the robot to find the appropriate digging point for the material more accurately.

[0124] It should be noted that by executing steps S351, S352, S353, S354, and S355, the point cloud can be horizontally calibrated before ground segmentation, and then horizontal calibration can be restored after segmentation. Thus, the ground separation method provided in this embodiment, compared to related technologies that directly separate ground point clouds using point cloud height information, can reduce the impact of uneven point cloud height, improve the generalization of segmentation parameters and the stability of segmentation results for drivable ground within local dynamic regions, and achieve effective and stable segmentation of drivable ground within local dynamic regions.

[0125] Furthermore, since the ground separation method provided in this application embodiment performs horizontal plane calibration on the point cloud before ground segmentation and then performs horizontal calibration restoration after segmentation, it can reduce the impact of uneven point cloud height. Therefore, the ground separation method provided in this application embodiment can also be applied to situations where there is a certain angle difference between the ground and the preset horizontal plane in the work scene, or where the ground has a certain slope and tilt angle. In these cases, using the ground separation method provided in this application embodiment to segment the drivable area ground can mitigate the impact of uneven point cloud height, improve the generalization of ground segmentation parameter tuning, and enhance the stability of the segmentation results.

[0126] Figure 3 This is a schematic diagram illustrating the process of separating a dynamic region transition ground point cloud and a dynamic region transition non-ground point cloud from a calibrated local dynamic region raster point cloud of the current frame, as provided in an exemplary embodiment of this application. Figure 3 As shown, step S354 includes:

[0127] S3541: Obtain the distance parameters based on the target plane equation and the preset horizontal plane equation.

[0128] Specifically, the distance parameter can be understood as the distance between the ground represented by the target plane equation after horizontal calibration and the preset horizontal plane. In other words, when calculating the distance parameter, the rotation matrix can be calculated first using the target plane equation and the preset horizontal plane equation. Then, the target plane equation is multiplied by the rotation matrix to make the ground represented by the target plane equation parallel to the preset horizontal plane. Finally, the distance between the two parallel planes is calculated, which yields the distance parameter.

[0129] It should be noted that the aforementioned horizontal calibration of the ground represented by the target plane equation is only a calculation process, and there is no need to perform an actual rotation operation on the ground represented by the target plane equation.

[0130] S3542: Based on the distance parameter and the height coordinates of each point in the local dynamic region raster point cloud of the current frame, filter out the initial ground seed point set and the initial non-ground seed point set that meet the first filtering condition.

[0131] It should be noted that since the pose of the part representing the ground in the local dynamic region grid point cloud of the current frame after horizontal calibration is consistent with the pose of the ground represented by the aforementioned target plane equation after horizontal calibration calculation, the aforementioned distance parameter can be used as the input for filtering to select the initial ground seed point set and the initial non-ground seed point set that meet the first filtering condition from the horizontally calibrated local dynamic region grid point cloud of the current frame.

[0132] It should be noted that the first screening condition indicates that the height coordinate is greater than a first height threshold and less than a second height threshold. The first height threshold is equal to the difference between the distance parameter and a first preset parameter, and the second height threshold is equal to the sum of the distance parameter and a second preset parameter. In other words, if the height coordinate of a point in the local dynamic region raster point cloud of the current frame is greater than the first height threshold and less than the second height threshold, then the point can be temporarily considered to belong to the ground point cloud and included in the initial ground seed point set; otherwise, the point can be considered to belong to the non-ground point cloud and included in the initial non-ground seed point set.

[0133] It should be understood that the first and second preset parameters can be set according to the actual situation, and this application does not make specific limitations on the preset parameters.

[0134] In one embodiment, the first preset parameter and the second preset parameter may be equal or unequal.

[0135] S3543: Perform secondary filtering on the initial ground seed point set to obtain dynamic region transition ground point cloud and dynamic region transition non-ground point cloud.

[0136] Specifically, to improve the accuracy and reliability of the ground point cloud separated from the local dynamic region raster point cloud of the current frame, the initial ground seed point set can be screened a second time to obtain the aforementioned dynamic region transition ground point cloud and dynamic region transition non-ground point cloud. Then, step S355 is executed to perform horizontal rotation calibration and restoration on the dynamic region transition ground point cloud and dynamic region transition non-ground point cloud to obtain the dynamic region target ground point cloud and dynamic region target non-ground point cloud.

[0137] Figure 4 This is a schematic diagram illustrating a process for performing secondary filtering on an initial ground seed point set to obtain a dynamic region transition ground point cloud and a dynamic region transition non-ground point cloud, provided as an exemplary embodiment of this application. Figure 4 As shown, step S3543 includes:

[0138] S35431: Based on the initial ground seed point set, the transition plane is obtained by fitting.

[0139] S35432: Select the target ground seed point set and the target non-ground seed point set that satisfy the second selection condition from the initial ground seed point set.

[0140] S35433: Output the point cloud represented by the target ground seed point set as the dynamic region transition ground point cloud;

[0141] S35434: The initial non-ground seed point set and the target non-ground seed point set are used to represent the point cloud as the dynamic region transition non-ground point cloud output.

[0142] Specifically, multiple points in the initial ground seed point set are fitted again to obtain the transition plane.

[0143] It should be noted that the second screening criterion can be understood as the distance between a point in the initial ground seed point set and the transition plane being greater than the first distance threshold and less than the second distance threshold. In other words, if the distance between a point in the initial ground seed point set and the transition plane is greater than the first distance threshold and less than the second distance threshold, then that point can be considered a point in the ground point cloud and included in the target ground seed point set; otherwise, that point will be considered a point in the non-ground point cloud and included in the target non-ground seed point set.

[0144] It should be noted that points that meet both the first and second screening conditions are ultimately assigned to the target ground seed point set. Therefore, the point cloud represented by the target ground seed point set can be output as the dynamic region transition ground point cloud. Points that do not simultaneously meet both screening conditions are assigned to either the initial non-ground seed point set or the target non-ground seed point set. Therefore, the point clouds represented by the initial and target non-ground seed point sets can be output as the dynamic region transition non-ground point cloud.

[0145] It should be understood that the first distance threshold and the second distance threshold can be set according to the actual situation, and this application does not specifically limit the first distance threshold and the second distance threshold.

[0146] It should be noted that in practical applications, steps S35432, S35433, and S35434 can be repeated. This allows for multiple iterative filtering of the points in the initial ground seed point set, further improving the accuracy and reliability of the separated ground point cloud. It should be understood that the number of iterations can be set according to actual conditions; this embodiment does not specifically limit the number of iterations.

[0147] Figure 5 This is a schematic diagram illustrating the process of updating the local dynamic region raster point cloud in the current frame of the global raster point cloud map, provided as an exemplary embodiment of this application. (See diagram for details.) Figure 5 As shown, step S350 includes:

[0148] S356: Rasterize the current frame scan point cloud to obtain the current frame global raster point cloud.

[0149] S357: Based on the pose information of the global grid point cloud map, the current frame scan point cloud, and the current frame global grid point cloud, update the current frame local dynamic region grid point cloud in the global grid point cloud map.

[0150] Specifically, in order to unify the format of the current frame scanned point cloud with the format of the raster point cloud in the missing local dynamic region in the global raster point cloud map, the current frame scanned point cloud needs to be rasterized to obtain the current frame global raster point cloud. Then, based on the local dynamic region boundary in the global raster point cloud map and the pose information of the current frame scanned point cloud, the current frame local dynamic region raster point cloud is updated in the local dynamic region.

[0151] Figure 6 This is a schematic diagram illustrating the process of separating the global target ground point cloud and the global target non-ground point cloud, provided as an exemplary embodiment of this application. Figure 6 As shown, step S360 includes:

[0152] S361: Rasterize the current frame's scanned point cloud to obtain the current frame's global raster point cloud.

[0153] It should be noted that the specific rasterization algorithms discussed in this article are documented in relevant technologies and will not be elaborated upon here.

[0154] S362: Extract the initial ground point cloud from the global voxel point cloud map and / or the transition raster point cloud map.

[0155] S363: Based on the initial ground point cloud, the equation of the target plane is obtained by fitting.

[0156] S364: Perform horizontal rotation calibration on the global raster point cloud of the current frame based on the target plane equation and the preset horizontal plane equation.

[0157] S365: Separate the global transition ground point cloud and the global transition non-ground point cloud from the calibrated current frame global raster point cloud.

[0158] S366: Based on the target plane equation and the preset horizontal plane equation, perform horizontal rotation calibration and restoration on the global transition ground point cloud and the global transition non-ground point cloud to obtain the global target ground point cloud and the global target non-ground point cloud.

[0159] It should be noted that the execution process of steps S362, S363, S364, S365 and S366 is similar to the execution process of steps S351, S352, S353, S354 and S355 mentioned above, and can be referred to the content introduced above.

[0160] Figure 7 This is a schematic diagram illustrating the process of separating a global transition ground point cloud and a global transition non-ground point cloud from the calibrated current frame global raster point cloud, provided as an exemplary embodiment of this application. Figure 7 As shown, step S365 includes:

[0161] S3651: Obtain the distance parameters based on the target plane equation and the preset horizontal plane equation.

[0162] S3652: Based on the distance parameter and the height coordinates of each point in the current frame's global raster point cloud, filter out the initial ground seed point set and the initial non-ground seed point set that meet the first filtering condition.

[0163] S3653: Perform a secondary screening on the initial ground seed point set to obtain the global transition ground point cloud and the global transition non-ground point cloud.

[0164] It should be noted that the execution process of steps S3651, S3652 and S3653 is similar to that of steps S3541, S3542 and S3543 mentioned above, and can be referred to the content introduced above.

[0165] Figure 8 This is a schematic diagram illustrating the process of performing secondary filtering on an initial ground seed point set to obtain a global transition ground point cloud and a global transition non-ground point cloud, provided as an exemplary embodiment of this application. Figure 8 As shown, step S3653 includes:

[0166] S36531: Based on the initial ground seed point set, the transition plane is obtained by fitting.

[0167] S36532: Select the target ground seed point set and the target non-ground seed point set that satisfy the second selection condition from the initial ground seed point set.

[0168] S36533: Output the point cloud represented by the target ground seed point set as the global transition ground point cloud.

[0169] S36534: The initial non-ground seed point set and the target non-ground seed point set are used to represent the point cloud as the global transition non-ground point cloud output.

[0170] It should be noted that the execution process of steps S36531, S36532, S36533 and S36534 is similar to that of the aforementioned steps S35431, S35432, S35433 and S35434, and can be referred to the content introduced above.

[0171] Figure 9 This is a schematic diagram illustrating the process of obtaining a global voxel point cloud map and a global raster point cloud map based on an initial map, provided as an exemplary embodiment of this application. Figure 9 As shown, S320 includes:

[0172] S321: Downsample and filter the point cloud in the initial map to obtain a global voxel point cloud map.

[0173] Specifically, downsampling the point cloud in the initial map refers to reducing the number of points in the point cloud data to decrease its complexity and storage space, while preserving its main features and shape information. Filtering the point cloud in the initial map can include denoising, smoothing, and other processing. It should be noted that the specific algorithms for downsampling and filtering are documented in relevant technical literature and will not be elaborated upon here.

[0174] S322: Rasterize, filter, and delete the initial local dynamic region raster point cloud in the point cloud of the initial map to obtain a global raster point cloud map.

[0175] Specifically, before obtaining the global raster point cloud map, the boundaries of local dynamic regions are first delineated according to the specific conditions of the working environment. Then, the initial map is subjected to 3D rasterization of the point cloud, filtering, and deletion of local dynamic region raster point clouds based on the boundaries of the local dynamic regions to obtain the global raster point cloud map.

[0176] Figure 10 This application provides an exemplary embodiment of obtaining the pose information of each frame of scanned point cloud in the global coordinate system based on a global voxel point cloud map. For example... Figure 10 As shown, step S330 includes:

[0177] S331: Match the first frame of scanned point cloud with the global voxel point cloud map, relocate the current starting position of the lidar, and obtain the initial pose of the lidar in the global coordinate system.

[0178] S332: Based on the scanned point cloud of each frame obtained by the lidar and the initial pose of the lidar, obtain the pose information of the scanned point cloud in the global coordinate system.

[0179] Specifically, as mentioned earlier, when constructing the global voxel map, the robot is equipped with a LiDAR and various corresponding sensors. It starts from the initial position of the working environment (i.e., the robot's current parking area), and then the LiDAR performs a global scan during its movement. During this construction process, the robot's parking position is the origin of the global coordinate system. However, when executing the ground separation method of this application embodiment, there will be an error between the robot's starting position and the origin of the global coordinate system. Therefore, step S331 needs to be executed to match the first frame of scanned point cloud with the previously constructed global voxel point cloud map. Based on the matching result, the starting position of the robot is repositioned, thereby obtaining a more accurate initial pose of the LiDAR in the global coordinate system. Then, based on the initial pose of the LiDAR and the scanned point cloud obtained during the LiDAR's movement, a more accurate pose information of each frame of scanned point cloud in the global coordinate system can be calculated. This is also beneficial for subsequently separating a more accurate and reliable ground point cloud.

[0180] Figure 11 This is a structural block diagram of a ground separation device provided for an exemplary embodiment of this application. Figure 11 As shown, the ground separation device 500 provided in this application embodiment includes: a first acquisition module 510, configured to acquire an initial map; wherein the initial map represents a preset working environment map; a first processing module 520, configured to obtain a global voxel point cloud map and a global grid point cloud map based on the initial map; wherein the global grid point cloud map represents the remaining grid point cloud map after deleting the initial local dynamic region grid point cloud; a first conversion module 530, configured to obtain the pose information of each frame of scanned point cloud in the global coordinate system based on the global voxel point cloud map; wherein the scanned point cloud represents the point cloud obtained by the lidar in real time scanning in the working environment; and a first output module 540, configured to output the current frame of scanned point cloud. The first separation module 550 is configured to, if the first information is output, update the local dynamic region grid point cloud in the global grid point cloud map based on the global grid point cloud map, the current frame scan point cloud, and the pose information of the current frame scan point cloud, and separate the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the current frame local dynamic region grid point cloud; the second separation module 560 is configured to, if the second information is output, separate the global target ground point cloud and the global target non-ground point cloud based on the global voxel point cloud map and the current frame scan point cloud.

[0181] The ground separation device provided in this application embodiment acquires an initial map, then obtains a global voxel point cloud map and a global grid point cloud map based on the initial map, then obtains the pose information of the scanned point cloud in each frame in the global coordinate system based on the global voxel point cloud map, and then outputs first information indicating that the lidar is located within a local dynamic region or second information indicating that the lidar is located outside the local dynamic region based on the pose information of the scanned point cloud in the current frame. In the first aspect, when the first information is output, the local dynamic region grid point cloud in the current frame in the global grid point cloud map can be updated based on the global grid point cloud map, the scanned point cloud in the current frame, and the pose information of the scanned point cloud in the current frame, and the target ground point cloud and the dynamic region can be separated from the local dynamic region grid point cloud in the current frame. The target non-ground point cloud and the dynamic region target non-ground point cloud can be considered as materials piled up in a local dynamic area. By separating the dynamic region target ground point cloud and the dynamic region target non-ground point cloud, the robot can easily detect changes in material characteristics within the local dynamic area, thus facilitating the subsequent finding of better material scooping points and improving material loading efficiency. Secondly, when outputting the second information, the global target ground point cloud and the global target non-ground point cloud can be separated based on the global voxel point cloud map and the current frame scan point cloud. The global target ground point cloud can be considered as the global drivable area ground in the working scene. This allows the robot to easily detect the drivable area ground, which serves as the basis for subsequent overall path planning and safe driving decisions, effectively improving driving safety. In summary, this ground separation device can assist the robot in separating the drivable area ground in the global working environment during a single descent, and also in updating and separating the drivable area ground and material pile characteristics in a local dynamic scene.

[0182] Figure 12 A structural block diagram of a ground separation device provided for another exemplary embodiment of this application. (See diagram below.) Figure 12As shown, in one embodiment, the first separation module 550 includes a first extraction module 551, configured to extract an initial ground point cloud from a global voxel point cloud map and / or a transition raster point cloud map; wherein, the transition raster point cloud map represents the raster point cloud map of the global raster point cloud map before the initial local dynamic region raster point cloud is deleted; a first fitting module 552, configured to fit a target plane equation based on the initial ground point cloud; a first calibration module 553, configured to perform horizontal rotation calibration on the current frame's local dynamic region raster point cloud based on the target plane equation and a preset horizontal plane equation; a third separation module 554, configured to separate a dynamic region transition ground point cloud and a dynamic region transition non-ground point cloud from the calibrated current frame's local dynamic region raster point cloud; and a first restoration module 555, configured to perform horizontal rotation calibration restoration on the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud based on the target plane equation and the preset horizontal plane equation, to obtain a dynamic region target ground point cloud and a dynamic region target non-ground point cloud.

[0183] like Figure 12 As shown, in one embodiment, the third separation module 554 includes a first calculation module 5541, configured to obtain a distance parameter based on the target plane equation and a preset horizontal plane equation; wherein the distance parameter represents the distance between the plane represented by the target plane equation after horizontal calibration and the preset horizontal plane; a first filtering module 5542, configured to filter out an initial ground seed point set and an initial non-ground seed point set that meet the first filtering conditions based on the distance parameter and the height coordinates of each point in the local dynamic region raster point cloud of the current frame; wherein the first filtering condition represents that the height coordinates are greater than a first height threshold and less than a second height threshold; wherein the first height threshold is equal to the difference between the distance parameter and the preset parameter, and the second height threshold is equal to the sum of the distance parameter and the preset parameter; a second filtering module 5543, configured to perform a second filtering on the initial ground seed point set to obtain a dynamic region transition ground point cloud and a dynamic region transition non-ground point cloud.

[0184] like Figure 12 As shown, in one embodiment, the second filtering module 5543 includes a second fitting module 55431, configured to fit a transition plane based on an initial ground seed point set; a third filtering module 55432, configured to filter a target ground seed point set and a target non-ground seed point set that satisfy the second filtering condition from the initial ground seed point set; a second output module 55433, configured to output the point cloud represented by the target ground seed point set as a dynamic region transition ground point cloud; and a third output module 55434, configured to output the point cloud represented by the initial non-ground seed point set and the target non-ground seed point set as a dynamic region transition non-ground point cloud.

[0185] like Figure 12As shown, in one embodiment, the first separation module 550 includes a first raster processing module 556, configured to perform rasterization processing on the current frame scan point cloud to obtain the current frame global raster point cloud; and an update module 557, configured to update the current frame local dynamic region raster point cloud in the global raster point cloud map according to the pose information of the current frame scan point cloud and the current frame global raster point cloud.

[0186] like Figure 12 As shown, in one embodiment, the second separation module 560 includes a second raster processing module 561, configured to rasterize the current frame scan point cloud to obtain the current frame global raster point cloud; a second extraction module 562, configured to extract the initial ground point cloud from the global voxel point cloud map and / or the transition raster point cloud map; wherein, the transition raster point cloud map represents the global raster point cloud map before the initial local dynamic region raster point cloud is deleted; and a third fitting module 563, configured to fit the target plane according to the initial ground point cloud. The process includes: a second calibration module 564 configured to perform horizontal rotation calibration on the current frame global grid point cloud according to the target plane equation and the preset horizontal plane equation; a fourth separation module 565 configured to separate the global transition ground point cloud and the global transition non-ground point cloud from the calibrated current frame global grid point cloud; and a second restoration module 566 configured to perform horizontal rotation calibration restoration on the global transition ground point cloud and the global transition non-ground point cloud according to the target plane equation and the preset horizontal plane equation to obtain the global target ground point cloud and the global target non-ground point cloud.

[0187] like Figure 12 As shown, in one embodiment, the fourth separation module 565 includes a second calculation module 5651, configured to obtain a distance parameter based on the target plane equation and a preset horizontal plane equation; wherein, the distance parameter represents the distance between the plane represented by the target plane equation after horizontal calibration and the preset horizontal plane; the fourth filtering module 5652 is configured to filter out an initial ground seed point set and an initial non-ground seed point set that meet the first filtering condition based on the distance parameter and the height coordinate of each point in the current frame's global raster point cloud; wherein, the first filtering condition represents that the height coordinate is greater than a first height threshold and less than a second height threshold; wherein, the first height threshold is equal to the difference between the distance parameter and the preset parameter, and the second height threshold is equal to the sum of the distance parameter and the preset parameter; the fifth filtering module 5653 is configured to perform a second filtering on the initial ground seed point set to obtain a global transition ground point cloud and a global transition non-ground point cloud.

[0188] like Figure 12As shown, in one embodiment, the fourth filtering module 5653 includes a fourth fitting module 56531, configured to fit a transition plane based on an initial ground seed point set; a sixth filtering module 56532, configured to filter a target ground seed point set and a target non-ground seed point set that satisfy the second filtering condition from the initial ground seed point set; a fourth output module 56533, configured to output the point cloud represented by the target ground seed point set as a global transition ground point cloud; and a fifth output module 56534, configured to output the point cloud represented by the initial non-ground seed point set and the target non-ground seed point set as a global transition non-ground point cloud.

[0189] like Figure 12 As shown, in one embodiment, the first processing module 520 includes a second processing module 521, configured to downsample and filter the point cloud in the initial map to obtain a global voxel point cloud map; and a third processing module 522, configured to rasterize, filter, and delete the initial local dynamic region raster point cloud in the point cloud of the initial map to obtain a global raster point cloud map.

[0190] like Figure 12 As shown, in one embodiment, the first conversion module 530 includes a positioning module 531, configured to match the first frame scanned point cloud with the global voxel point cloud map, reposition the current starting position of the lidar, and obtain the initial pose of the lidar in the global coordinate system; the second conversion module 532 is configured to obtain the pose information of each frame scanned point cloud in the global coordinate system based on each frame scanned point cloud obtained by the lidar and the initial pose of the lidar.

[0191] Figure 13 This is a structural block diagram of a material loading system provided for an exemplary embodiment of this application. Figure 13 As shown, the material loading system 700 provided in this application embodiment includes: a robot 710 equipped with a lidar; and a ground separation device 500 as described above, wherein the bottom separation device 500 is communicatively connected to the lidar.

[0192] The material feeding system 700 provided in this application includes the ground separation device 500 in the aforementioned embodiments and has all the functions of the ground separation device 500. Its beneficial effects can be referred to the beneficial effects of the aforementioned ground separation device 500.

[0193] In one embodiment, the material feeding system 700 can be applied to scenarios such as mixing plants and yard bridges.

[0194] Figure 14 This is a structural block diagram of an electronic device provided as an exemplary embodiment of this application. (See diagram below.) Figure 14As shown, the electronic device 900 provided in this application embodiment can be any one or both of the first device and the second device, or a standalone device independent of them. The standalone device can communicate with the first device and the second device to receive the collected input signals from them.

[0195] like Figure 14 As shown, the electronic device 900 includes one or more processors 910 and a memory 920. The memory 920 is used to store executable instructions of the processor 910, which is used to perform the ground separation method as described above.

[0196] The processor 910 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 900 to perform desired functions.

[0197] The memory 920 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 910 may execute the program instructions to implement the control methods and / or other desired functions of the various embodiments of this application described above. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0198] In one example, the electronic device 900 may also include an input device 930 and an output device 940, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0199] When the controller is a standalone device, the input device 930 can be a communication network connector for receiving the acquired input signals from the first device and the second device.

[0200] In addition, the input device 930 may also include, for example, a keyboard, a mouse, etc.

[0201] The output device 940 can output various information to the outside, including determined distance information, direction information, etc. The output device 940 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0202] Of course, for the sake of simplicity, Figure 14Only some of the components of the electronic device 900 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 900 may include any other suitable components depending on the specific application.

[0203] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0204] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0205] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0206] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0207] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0208] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0209] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A ground separation method, characterized in that, include: Obtain an initial map; wherein the initial map represents a preset working environment map; Based on the initial map, a global voxel point cloud map and a global raster point cloud map are obtained; wherein, the global raster point cloud map represents the remaining raster point cloud map after deleting the initial local dynamic region raster point cloud. Based on the global voxel point cloud map, the pose information of each frame of scanned point cloud in the global coordinate system is obtained; wherein, the scanned point cloud represents the point cloud obtained by the lidar in real time scanning in the working environment; Based on the pose information of the scanned point cloud in the current frame, output first information indicating that the lidar is located within a local dynamic region or output second information indicating that the lidar is located outside the local dynamic region; If the first information is output, based on the global grid point cloud map, the scanned point cloud of the current frame and the pose information of the scanned point cloud of the current frame, the local dynamic region grid point cloud of the current frame in the global grid point cloud map is updated, and the dynamic region target ground point cloud and the dynamic region target non-ground point cloud are separated from the local dynamic region grid point cloud of the current frame. If the second information is output, the global target ground point cloud and the global target non-ground point cloud are separated based on the global voxel point cloud map and the scanned point cloud of the current frame.

2. The ground separation method according to claim 1, characterized in that, The step of separating the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the local dynamic region raster point cloud of the current frame includes: An initial ground point cloud is extracted from the global voxel point cloud map and / or the transition raster point cloud map; wherein, the transition raster point cloud map represents the global raster point cloud map before the initial local dynamic region raster point cloud is deleted. Based on the initial ground point cloud, the equation of the target plane is obtained by fitting. Based on the target plane equation and the preset horizontal plane equation, the local dynamic region raster point cloud of the current frame is horizontally rotated and calibrated. From the calibrated local dynamic region raster point cloud of the current frame, the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud are separated. Based on the target plane equation and the preset horizontal plane equation, the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud are horizontally rotated, calibrated, and restored to obtain the dynamic region target ground point cloud and the dynamic region target non-ground point cloud.

3. The ground separation method according to claim 2, characterized in that, The step of separating the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud from the calibrated local dynamic region raster point cloud of the current frame includes: Based on the target plane equation and the preset horizontal plane equation, a distance parameter is obtained; wherein, the distance parameter represents the distance between the plane represented by the target plane equation after horizontal calibration and the preset horizontal plane; Based on the distance parameter and the height coordinates of each point in the local dynamic region raster point cloud of the current frame, the initial ground seed point set and the initial non-ground seed point set that meet the first filtering condition are selected. The initial ground seed point set is subjected to secondary screening to obtain the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud.

4. The ground separation method according to claim 3, characterized in that, The step of performing a secondary screening on the initial ground seed point set to obtain the dynamic region transition ground point cloud and the dynamic region transition non-ground point cloud includes: Based on the initial set of ground seed points, a transition plane is obtained by fitting. From the initial ground seed point set, a target ground seed point set and a target non-ground seed point set that satisfy the second screening condition are obtained; The point cloud represented by the target ground seed point set is output as the dynamic region transition ground point cloud; The initial non-ground seed point set and the target non-ground seed point set are used to represent the point cloud as the output of the dynamic region transition non-ground point cloud.

5. The ground separation method according to claim 1, characterized in that, The step of updating the local dynamic region grid point cloud in the global grid point cloud map based on the global grid point cloud map, the scanned point cloud in the current frame, and the pose information of the scanned point cloud in the current frame includes: Rasterize the scanned point cloud of the current frame to obtain the global raster point cloud of the current frame; Based on the global grid point cloud map, the pose information of the scanned point cloud in the current frame, and the global grid point cloud of the current frame, the local dynamic region grid point cloud of the current frame in the global grid point cloud map is updated.

6. The ground separation method according to claim 1, characterized in that, The separation of the global target ground point cloud and the global target non-ground point cloud includes: Rasterize the scanned point cloud of the current frame to obtain the global raster point cloud of the current frame; An initial ground point cloud is extracted from the global voxel point cloud map and / or the transition raster point cloud map; wherein, the transition raster point cloud map represents the global raster point cloud map before the initial local dynamic region raster point cloud is deleted. Based on the initial ground point cloud, the equation of the target plane is obtained by fitting. Based on the target plane equation and the preset horizontal plane equation, the current frame global grid point cloud is horizontally rotated and calibrated. From the calibrated global raster point cloud of the current frame, the global transition ground point cloud and the global transition non-ground point cloud are separated. Based on the target plane equation and the preset horizontal plane equation, the global transition ground point cloud and the global transition non-ground point cloud are horizontally rotated, calibrated, and restored to obtain the global target ground point cloud and the global target non-ground point cloud.

7. The ground separation method according to claim 6, characterized in that, The step of separating the global transition ground point cloud and the global transition non-ground point cloud from the calibrated current frame global raster point cloud includes: Based on the target plane equation and the preset horizontal plane equation, a distance parameter is obtained; wherein, the distance parameter represents the distance between the plane represented by the target plane equation after horizontal calibration and the preset horizontal plane; Based on the distance parameter and the height coordinates of each point in the current frame global raster point cloud, the initial ground seed point set and the initial non-ground seed point set that meet the first filtering condition are selected. The initial ground seed point set is subjected to secondary screening to obtain the global transition ground point cloud and the global transition non-ground point cloud.

8. The ground separation method according to claim 7, characterized in that, The step of performing a secondary screening on the initial ground seed point set to obtain the global transition ground point cloud and the global transition non-ground point cloud includes: Based on the initial set of ground seed points, a transition plane is obtained by fitting. From the initial ground seed point set, a target ground seed point set and a target non-ground seed point set that satisfy the second screening condition are obtained; The point cloud represented by the target ground seed point set is output as the global transition ground point cloud; The initial non-ground seed point set and the target non-ground seed point set are used to represent the point cloud as the global transition non-ground point cloud output.

9. The ground separation method according to claim 1, characterized in that, The process of obtaining the global voxel point cloud map and the global raster point cloud map based on the initial map includes: The point cloud in the initial map is downsampled and filtered to obtain the global voxel point cloud map; The point cloud in the initial map is rasterized, filtered, and the initial local dynamic region raster point cloud is deleted to obtain the global raster point cloud map.

10. The ground separation method according to claim 1, characterized in that, The step of obtaining the pose information of each frame of scanned point cloud in the global coordinate system based on the global voxel point cloud map includes: The first frame of the scanned point cloud is matched with the global voxel point cloud map to relocate the current starting position of the lidar and obtain the initial pose of the lidar in the global coordinate system. Based on the scanned point cloud obtained from each frame of the lidar scan and the initial pose of the lidar, the pose information of the scanned point cloud in the global coordinate system for each frame is obtained.

11. A ground separation device, characterized in that, include: The first acquisition module is configured to acquire an initial map; wherein the initial map represents a preset working environment map; The first processing module is configured to obtain a global voxel point cloud map and a global raster point cloud map based on the initial map; wherein the global raster point cloud map represents the remaining raster point cloud map after deleting the initial local dynamic region raster point cloud. The first conversion module is configured to obtain the pose information of each frame of scanned point cloud in the global coordinate system based on the global voxel point cloud map; wherein, the scanned point cloud represents the point cloud obtained by the lidar in real time in the working environment; The first output module is configured to output first information indicating that the lidar is located within a local dynamic region or output second information indicating that the lidar is located outside the local dynamic region, based on the pose information of the scanned point cloud in the current frame. The first separation module is configured to, if the first information is output, update the local dynamic region grid point cloud in the global grid point cloud map based on the global grid point cloud map, the scanned point cloud in the current frame and the pose information of the scanned point cloud in the current frame, and separate the dynamic region target ground point cloud and the dynamic region target non-ground point cloud from the local dynamic region grid point cloud in the current frame. The second separation module is configured to, if the second information is output, separate the global target ground point cloud and the global target non-ground point cloud based on the global voxel point cloud map and the scanned point cloud of the current frame.

12. A material feeding system, characterized in that, include: The robot is equipped with a lidar system. The ground separation device as described in claim 11 is communicatively connected to the lidar.

Images