Point cloud processing method and device, storage medium and electronic equipment
By distinguishing foreground and background targets in a mining scene and performing densification processing, the accuracy and quality issues of point cloud processing in mining scenes are solved, and efficient point cloud reconstruction and occupancy network model generation are achieved.
Patent Information
- Application Number
- CN202411545213.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-18
AI Technical Summary
In mining scenarios, conventional SLAM algorithms and point cloud registration algorithms suffer severe degradation, and dust and non-rigid moving machinery prevent point cloud accumulation, affecting the point cloud processing effect.
By acquiring multi-frame point cloud data, foreground and background targets are distinguished based on semantic information, rigid body judgment is performed and densification is carried out separately to generate an occupancy network model, avoiding ghosting and layering problems.
It improves the accuracy of point cloud reconstruction and the quality of ground truth in mining scenarios, reduces errors, and is suitable for point cloud processing in mining environments.
Smart Images
Figure CN119478314A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical fields of autonomous driving and unmanned vehicles, and in particular to a point cloud processing method and device, a storage medium and an electronic device. Background Art
[0002] In mining scenes, due to the open terrain and low height of most backgrounds, conventional SLAM algorithms and point cloud registration algorithms will suffer from serious degradation in mining scenes. In addition, there is a lot of dust and non-rigid moving machinery in the mining environment. The point clouds of these objects cannot be accumulated, and special measures need to be taken. Therefore, a point cloud processing method for mining scenes is urgently needed. Summary of the invention
[0003] In view of this, the present disclosure provides a point cloud processing method and device, a storage medium and an electronic device.
[0004] In a first aspect, a point cloud processing method is provided, including: acquiring multi-frame point cloud data corresponding to a target space region, the multi-frame point cloud data including first point cloud data corresponding to each of the multi-frame key frames, the first point cloud data containing corresponding semantic information; determining a foreground target and a background target in the key frame based on the semantic information; performing rigid body judgment on the foreground target and the background target respectively, dividing the first point cloud data corresponding to the foreground target into a foreground rigid point cloud and a foreground non-rigid point cloud, and dividing the first point cloud data corresponding to the background target into a background rigid point cloud and a background non-rigid point cloud; densifying the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud respectively; generating an occupancy network model corresponding to the target space region based on the densified foreground rigid point cloud, the densified foreground non-rigid point cloud, the densified background rigid point cloud, and the background non-rigid point cloud.
[0005] In some embodiments, the background rigid body point cloud is densified, including: performing a first registration of the background rigid body point clouds of the multiple key frames according to the positioning information corresponding to each of the multiple key frames; reducing the coordinates of the background rigid body point cloud after the first registration to one s of the original, where s is a positive real number; performing a second registration of the background rigid body point cloud after the coordinate adjustment, and splicing the background rigid body point cloud after the second registration to obtain a spliced background rigid body point cloud; expanding the coordinates of the spliced background rigid body point cloud to s times the original; and / or, performing a densification of the foreground rigid body point cloud, including: aligning the foreground rigid body point clouds of the multiple key frames based on the center position of the foreground rigid body point clouds of the multiple key frames; and splicing the aligned foreground rigid body point clouds.
[0006] In some embodiments, the background rigid body point cloud after coordinate adjustment is re-aligned, and the background rigid body point cloud after the re-alignment is spliced to obtain a spliced background rigid body point cloud, including: grouping multiple key frames, each key frame group includes multiple consecutive key frames; for each key frame group, aligning the background rigid body point clouds of the multiple key frames included in the key frame group, and splicing the aligned background rigid body point clouds to obtain intermediate point cloud data corresponding to the key frame group; aligning the intermediate point cloud data corresponding to the multiple key frame groups, and splicing the aligned intermediate point cloud data to obtain a spliced background rigid body point cloud.
[0007] In some embodiments, the multi-frame point cloud data also includes second point cloud data corresponding to each of the multi-frame non-key frames. Densification processing is performed on the foreground non-rigid point cloud, including: respectively determining the second point cloud data corresponding to each of the preset number of non-key frames before and / or after the target key frame as the adjacent frame point cloud; based on the coordinate information of the foreground non-rigid point cloud corresponding to the target key frame, searching the adjacent frame point cloud to obtain the adjacent frame target point cloud; based on the adjacent frame target point cloud and the foreground non-rigid point cloud, obtaining a densified foreground non-rigid point cloud.
[0008] In some embodiments, based on adjacent frame target point clouds and foreground non-rigid point clouds, a dense foreground non-rigid point cloud is obtained, including: calculating the angle between the vector line between the foreground non-rigid point cloud and the adjacent frame target point cloud and the height direction; when the angle is less than a preset angle, the foreground non-rigid point cloud and the adjacent frame target point cloud are spliced to obtain a dense foreground non-rigid point cloud.
[0009] In some embodiments, after generating an occupancy network model corresponding to the target spatial area, the point cloud processing method further includes: traversing each horizontal coordinate of the occupancy network model to determine the number of grid pillars containing ground voxels; determining the number of ground voxels in the pillar containing ground voxels; determining the average ground thickness of the occupancy network model based on the number of pillars containing ground voxels and the number of ground voxels; and determining the ground quality of the occupancy network model based on the average ground thickness.
[0010] In some embodiments, after generating an occupancy network model corresponding to the target spatial area, the point cloud processing method also includes: traversing each height coordinate of the occupancy network model to determine the number of voxel layers containing retaining wall voxels; determining the number of retaining wall voxels in the voxel layers containing retaining wall voxels; determining the average retaining wall area of the occupancy network model based on the number of voxel layers containing retaining wall voxels and the number of retaining wall voxels; and determining the retaining wall quality of the occupancy network model based on the average retaining wall area.
[0011] In a second aspect, a point cloud processing device is provided, including: an acquisition module, configured to acquire multi-frame point cloud data corresponding to a target space area, the multi-frame point cloud data including first point cloud data corresponding to each of the multi-frame key frames, and the first point cloud data contains corresponding semantic information; a determination module, configured to determine a foreground target and a background target in the key frame based on the semantic information; a division module, configured to perform rigid body judgment on the foreground target and the background target respectively, divide the first point cloud data corresponding to the foreground target into a foreground rigid point cloud and a foreground non-rigid point cloud, and divide the first point cloud data corresponding to the background target into a background rigid point cloud and a background non-rigid point cloud; a processing module, configured to perform densification processing on the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud respectively; a generation module, configured to generate an occupancy network model corresponding to the target space area based on the densified foreground rigid point cloud, the densified foreground non-rigid point cloud, the densified background rigid point cloud, and the background non-rigid point cloud.
[0012] According to a third aspect, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method provided in the first aspect by executing the executable instructions.
[0013] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method provided in the first aspect is implemented.
[0014] The point cloud processing method provided by the present disclosure classifies the first point cloud data into foreground rigid point cloud, foreground non-rigid point cloud, background rigid point cloud, and background non-rigid point cloud based on the target detection and semantic segmentation results of the first point cloud data, processes different types of point clouds based on different densification methods, and then merges them into dense point cloud data. This avoids ghosting or stratification problems that may occur during the densification process, and improves the quality of the occupancy network truth value. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 Shown is a schematic diagram of a scenario applicable to an embodiment of the present disclosure.
[0016] Figure 2 The figure is a flow chart of a point cloud processing method provided by an embodiment of the present disclosure.
[0017] Figure 3 The figure is a schematic diagram of a process for densifying a background rigid point cloud provided by an embodiment of the present disclosure.
[0018] Figure 4 The figure is a schematic diagram of a process of densifying a foreground rigid body point cloud provided by an embodiment of the present disclosure.
[0019] Figure 5 The figure shows a flow chart of the steps of performing secondary registration on the background rigid body point cloud after coordinate adjustment and splicing the secondary registered background rigid body point cloud to obtain the spliced background rigid body point cloud provided by an embodiment of the present disclosure.
[0020] Figure 6 The figure is a schematic diagram of a process of densifying a foreground non-rigid point cloud provided by an embodiment of the present disclosure.
[0021] Figure 7 The figure is a flow chart of the steps of obtaining a densified foreground non-rigid point cloud based on target point clouds and foreground non-rigid point clouds of adjacent frames provided by an embodiment of the present disclosure.
[0022] Figure 8 Shown is a flow chart of a point cloud processing method provided by another embodiment of the present disclosure.
[0023] Fig. 9 Shown is a flow chart of a point cloud processing method provided by yet another embodiment of the present disclosure.
[0024] Fig.10 Shown is a schematic structural diagram of a point cloud processing device provided in one embodiment of the present disclosure.
[0025] Fig.11 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0027] Occupancy Networks is an advanced environmental perception technology that enables autonomous driving systems to generate high-resolution, continuous 3D spatial representations, which is essential for accurate navigation and obstacle avoidance.
[0028] The occupancy network truth generation method is usually applied in passenger car scenes. Directly applying this technology to mining scenes will face many challenges. First of all, in mining scenes, the overall terrain is relatively open, and most of the background heights are low, which makes the slam algorithm and point cloud registration algorithm used in conventional passenger car scenes seriously degraded in mining scenes. Secondly, the terrain of mining scenes is more complex and changeable than urban roads, with more dramatic ups and downs, and the radar device is non-rigidly connected to the vehicle body. During the collection of point cloud data, the vibration of the radar device and the vehicle body will cause the point cloud to fluctuate greatly in continuous frames. In addition, there is a lot of dust and non-rigid moving machinery in the mining environment, and the point clouds of these objects cannot be accumulated.
[0029] In view of the above technical problems, the present disclosure provides a point cloud processing method, including: obtaining multi-frame point cloud data corresponding to the target space area, the multi-frame point cloud data includes first point cloud data corresponding to each of the multi-frame key frames, and the first point cloud data has corresponding semantic information; based on the semantic information, determining the foreground target and the background target in the key frame; performing rigid body judgment on the foreground target and the background target respectively, dividing the first point cloud data corresponding to the foreground target into foreground rigid point cloud and foreground non-rigid point cloud, and dividing the first point cloud data corresponding to the background target into background rigid point cloud and background non-rigid point cloud; performing densification processing on the foreground rigid point cloud, foreground non-rigid point cloud, and background rigid point cloud respectively; based on the densified foreground rigid point cloud, the densified foreground non-rigid point cloud, the densified background rigid point cloud, and the background non-rigid point cloud, generating an occupancy network model corresponding to the target space area. The present method provides a point cloud processing method suitable for mining scenes, effectively reducing the error of point cloud reconstruction and improving the quality of occupancy network truth generation.
[0030] Figure 1 FIG. 1 is a schematic diagram of a scenario applicable to an embodiment of the present disclosure. Figure 1 As shown, the scene is a mining scene, which includes a vehicle. The vehicle is equipped with a radar device 110 and a computing platform 120, and the radar device 110 is communicatively connected to the computing platform 120. The radar device 110 is used to collect three-dimensional point cloud data of the target space area around the vehicle. The computing platform 120 annotates the key frame point cloud data in the three-dimensional point cloud data, and processes the three-dimensional point cloud data based on the method provided in the embodiment of the present disclosure to generate an occupancy network model corresponding to the target space area. After generating the occupancy network model, the computing platform 120 can perceive the drivable area or obstacle distribution in the target space area based on the occupancy network model.
[0031] The computing platform 120 may also evaluate the truth quality of the occupancy network model according to the method provided in the embodiment of the present disclosure, adjust relevant parameters of the point cloud processing method based on the truth quality, or determine the reliability of the occupancy network model.
[0032] In addition, the method provided in the embodiment of the present disclosure can also be executed by an independent server, and the server is connected to the vehicle in communication. The server can receive the three-dimensional point cloud data collected by the radar device 10, and execute the method provided in the embodiment of the present disclosure to align and densify the three-dimensional point cloud data. The densified point cloud data can be used to generate an occupancy network model, or it can also be used for other tasks.
[0033] Combine the following Figures 2 to 9 The point cloud processing method provided by the embodiment of the present disclosure is illustrated by way of example.
[0034] Figure 2 FIG. 1 is a flow chart of a point cloud processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the point cloud processing method provided in the embodiment of the present disclosure includes the following steps.
[0035] S210, obtaining multi-frame point cloud data corresponding to the target spatial area.
[0036] The target space region represents the space region of the collected point cloud. The multi-frame point cloud data includes first point cloud data corresponding to each of the multiple key frames, and the first point cloud data has corresponding semantic information.
[0037] Specifically, the multi-frame point cloud data includes point cloud data corresponding to each of the multiple continuous frames. The multiple continuous frames are further divided into multiple key frames and multiple non-key frames. A key frame refers to a specific frame with important information or features in the multiple continuous frames, and the point cloud data corresponding to each key frame (i.e., the first point cloud data) is pre-annotated with corresponding semantic information.
[0038] The semantic information includes the semantic segmentation result of the first point cloud data, that is, the semantic information includes the category label of each point in the first point cloud data. In addition, the semantic information also includes the target detection result of the first point cloud data, that is, the semantic information includes the category, position and other information of the specific target in the space. In the mining scene, the specific target can be a vehicle, excavator and other targets in the target space area.
[0039] S220, determining a foreground target and a background target in the key frame based on the semantic information.
[0040] Based on the first point cloud data corresponding to each key frame and the semantic information corresponding to the first point cloud data, the first point cloud data is separated into foreground and background to determine the foreground object and the background object in each key frame.
[0041] Foreground targets refer to objects of interest in the target space area. In the field of autonomous driving, they usually include moving targets such as vehicles, pedestrians, or the excavators mentioned in the above steps. Background targets usually include static environmental features such as roads, buildings, retaining walls, etc., as well as dynamic targets such as rain, snow, and dust.
[0042] S230, performing rigid body judgment on the foreground target and the background target respectively, dividing the first point cloud data corresponding to the foreground target into a foreground rigid body point cloud and a foreground non-rigid body point cloud, and dividing the first point cloud data corresponding to the background target into a background rigid body point cloud and a background non-rigid body point cloud.
[0043] According to the category information of the target included in the semantic information, the foreground target and the background target are judged as rigid bodies. A rigid target refers to a target whose shape and size remain unchanged and the relative position of each point inside it remains unchanged in multiple consecutive frames. A non-rigid target refers to a target whose relative position relationship between each point inside it changes in multiple consecutive frames.
[0044] Specifically, the foreground targets are divided into foreground rigid targets and foreground non-rigid targets according to the category information corresponding to each foreground target. Based on the division result, the first point cloud data corresponding to the foreground target is divided into foreground rigid point cloud and foreground non-rigid point cloud, wherein the foreground rigid point cloud is a part of the point cloud corresponding to the foreground rigid target in the first point cloud data, and the foreground non-rigid point cloud is a part of the point cloud corresponding to the foreground non-rigid target in the first point cloud data.
[0045] Similarly, the background targets are divided into background rigid targets and background non-rigid targets according to the category information corresponding to each background target. Based on the division result, the first point cloud data corresponding to the background target is divided into background rigid point cloud and background non-rigid point cloud, wherein the background rigid point cloud is a part of the point cloud corresponding to the background rigid target in the first point cloud data, and the background non-rigid point cloud is a part of the point cloud corresponding to the background non-rigid target in the first point cloud data.
[0046] S240, densification processing is performed on the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud respectively.
[0047] In this embodiment, based on the densification methods corresponding to the foreground rigid point cloud, foreground non-rigid point cloud, and background rigid point cloud, the foreground rigid point cloud, foreground non-rigid point cloud, and background rigid point cloud are densified respectively, and the densification methods of the three are different.
[0048] Specifically, foreground rigid body targets usually include vehicles, etc. For such targets, the foreground rigid body point clouds in multiple key frames can be directly spliced.
[0049] Foreground non-rigid targets usually include excavators, etc. For targets such as excavators that can perform non-rigid movements, directly stitching the foreground non-rigid point clouds in multiple key frames based on the target detection results and position information will produce serious ghosting phenomena.
[0050] Background rigid targets are usually static backgrounds in the target space. Due to the undulating terrain of the mine scene and the non-rigid connection between the radar device and the vehicle body, during the acquisition of point cloud data, the vibration of the radar device and the vehicle body will cause the point cloud to fluctuate greatly in continuous frames. Therefore, directly splicing the background rigid point cloud in multiple key frames will result in layering problems.
[0051] It can be seen that the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud have different characteristics, so different densification methods need to be adopted according to the characteristics of these three.
[0052] S250, generating an occupancy network model corresponding to the target space area based on the densified foreground rigid point cloud, the densified foreground non-rigid point cloud, the densified background rigid point cloud, and the background non-rigid point cloud.
[0053] In the above steps, based on the densification methods corresponding to the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud, the densified foreground rigid point cloud, the densified foreground non-rigid point cloud, and the densified background rigid point cloud are obtained respectively.
[0054] The background non-rigid point cloud is usually a part of the point cloud corresponding to the target such as rain, snow, dust, etc. in the environment. For such background non-rigid point clouds, they can be directly merged into the densified point cloud of the frame to which they belong without additional processing. Finally, the dense point cloud data corresponding to the target space area is obtained.
[0055] The points in the dense point cloud data are voxelized to determine the point cloud included in each voxel grid, and the occupancy network truth of the voxel grid is determined according to the semantics of the point cloud contained in the voxel grid to obtain the occupancy network model corresponding to the target space area.
[0056] In the disclosed embodiment, based on the target detection and semantic segmentation results of the first point cloud data, the first point cloud data is classified into foreground rigid point cloud, foreground non-rigid point cloud, background rigid point cloud, and background non-rigid point cloud, and different types of point clouds are processed based on different densification methods and then merged into dense point cloud data. This avoids ghosting or stratification problems that may occur during the densification process and improves the quality of the occupancy network truth.
[0057] The following introduces the densification methods of foreground rigid point cloud, foreground non-rigid point cloud, and background rigid point cloud respectively.
[0058] Figure 3 FIG. 1 is a schematic diagram of a process for densifying a background rigid point cloud provided by an embodiment of the present disclosure. Figure 3 As shown, the densification processing of the background rigid body point cloud provided in the embodiment of the present disclosure includes the following steps.
[0059] S310, performing a first registration on the background rigid body point clouds of the multiple key frames according to the positioning information corresponding to the multiple key frames.
[0060] Each frame of point cloud data in the multi-frame point cloud data may be collected by multiple radar devices. In this case, the background rigid point cloud of the multi-radar devices can be converted to the vehicle coordinate system according to the calibration external parameters of the multi-sensor. Then, according to the positioning information corresponding to each of the multi-frame key frames, the background rigid point cloud of each of the multi-frame key frames is moved to the coordinate system of the first frame point cloud. After achieving the spatiotemporal alignment of the background rigid point cloud in the multi-frame key frames, a preliminary and approximate point cloud registration result can be obtained, and the first registration (or coarse registration) can be achieved.
[0061] The positioning information may be RTK (Real-Time Kinematic) positioning information. It is also understandable that the multi-frame point cloud data may be moved to the coordinate system of any other frame of point cloud, and the present disclosure does not impose any specific limitation on this.
[0062] S320, reducing the coordinates of the background rigid body point cloud after the first registration to one s of the original coordinates.
[0063] The terrain of mining scenes is mostly flat, and the nearest neighbor distance between the background rigid point clouds in adjacent key frames is large. This will cause traditional point cloud registration algorithms such as GICP to be unable to accurately fit the plane, and the registration effect based on adjacent points is not good.
[0064] In order to adapt to the flat terrain of the mine scene, in this embodiment, the coordinates of the background rigid body point cloud after the first registration are uniformly reduced to one s of the original, to obtain the background rigid body point cloud after the coordinate adjustment, where s is a positive real number.
[0065] S330, performing secondary registration on the background rigid body point cloud after the coordinate adjustment, and splicing the background rigid body point cloud after the secondary registration to obtain a spliced background rigid body point cloud.
[0066] By adjusting the coordinates, the distance between points in the background rigid body point cloud can be shortened. At this time, the correspondence between the background rigid body point clouds in adjacent key frames can be determined by nearest neighbor search to achieve secondary registration of the background rigid body point cloud. The secondary registered background rigid body point clouds are spliced to obtain a spliced background rigid body point cloud.
[0067] S340, expand the coordinates of the stitched background rigid body point cloud to s times the original size.
[0068] The coordinates of the spliced background rigid body point cloud are expanded to s times of the original ones, that is, the distance between points in the background rigid body point cloud is restored, and the densified background rigid body point cloud is obtained.
[0069] In the disclosed embodiment, by shortening the distance between points in the background rigid body point cloud, the problem of degradation of the traditional point cloud registration algorithm caused by a flat weak feature environment can be avoided, thereby effectively improving the registration accuracy.
[0070] Figure 4 FIG. 1 is a flow chart of a densification process for a foreground rigid point cloud provided by an embodiment of the present disclosure. Figure 4 As shown, the densification processing of the foreground rigid body point cloud provided in the embodiment of the present disclosure includes the following steps.
[0071] S410, aligning the foreground rigid body point clouds of the multiple key frames based on the center positions of the foreground rigid body point clouds of the multiple key frames.
[0072] First, according to the target detection result of the first point cloud data, the center position of the foreground rigid body point cloud of each of the multiple key frames is determined. For example, the center point of the prediction box of each foreground rigid body target can be directly used as the center position of the foreground rigid body point cloud corresponding to the foreground rigid body target.
[0073] The foreground rigid body point clouds of the multiple key frames are aligned by aligning the center positions of the foreground rigid body point clouds of the multiple key frames. For example, each group of foreground rigid body point clouds can be moved so that the center position is moved to a specific coordinate (such as the origin coordinate of the coordinate system) to achieve alignment between the center positions.
[0074] In some scenarios, each key frame may include multiple foreground rigid body targets. In this case, it is necessary to determine the center position of the foreground rigid body point cloud of each foreground rigid body target in different frames, and align the center position of the foreground rigid body point cloud of each foreground rigid body target in different frames.
[0075] S420, stitching the aligned foreground rigid body point clouds.
[0076] The aligned foreground rigid body point clouds are spliced to obtain the densified foreground rigid body point clouds.
[0077] In the disclosed embodiment, the foreground rigid body point cloud is densified by aligning and splicing the center positions of the foreground rigid body point clouds in multiple key frames.
[0078] Figure 5FIG. 1 is a flow chart of the steps of performing secondary registration on the background rigid body point cloud after coordinate adjustment and splicing the secondary registered background rigid body point cloud to obtain the spliced background rigid body point cloud provided by an embodiment of the present disclosure. Figure 5 As shown, the steps of performing secondary registration on the background rigid body point cloud after coordinate adjustment and splicing the background rigid body point cloud after secondary registration to obtain the spliced background rigid body point cloud include the following steps.
[0079] S331, group multiple key frames.
[0080] According to the acquisition time of each of the multiple key frames, a preset number of key frames adjacent to each other in the time sequence are grouped into a group to form a key frame group, and each key frame group includes multiple continuous key frames. The number of key frames in the key frame group can be determined according to the number of key frames in the multi-frame point cloud data. For example, each key frame group can include 3 key frames.
[0081] S332, for each key frame group, registering the background rigid body point clouds of the multiple key frames included in the key frame group, and splicing the registered background rigid body point clouds to obtain intermediate point cloud data corresponding to the key frame group.
[0082] For each key frame group, the background rigid body point clouds of the multiple key frames in the key frame group are aligned. Exemplarily, the background rigid body point clouds of the multiple key frames can be iteratively aligned using the GICP algorithm, and the background rigid body point cloud of each frame is spliced with the background rigid body point cloud of the previous frame after the alignment is completed. In this way, the background rigid body point clouds of the multiple key frames in the same key frame group are iteratively aligned and spliced in turn to obtain the intermediate point cloud data corresponding to the key frame group. The intermediate point cloud data is the densified point cloud data corresponding to the background rigid body target in the key frame group.
[0083] Optionally, before iterative registration, the source point cloud and the target point cloud can be filtered by voxel filtering to prevent the problem of uneven algorithm convergence caused by uneven distribution of point clouds. The source point cloud and the target point cloud are the background rigid point cloud of the current frame to be registered and the background rigid point cloud of the previous frame that has completed the registration. Optionally, other point cloud registration algorithms can also be used to achieve registration between the background rigid point clouds of multiple key frames in the key frame group.
[0084] S333, registering the intermediate point cloud data corresponding to each of the multiple key frame groups, and splicing the registered intermediate point cloud data to obtain a spliced background rigid body point cloud.
[0085] Through the above steps, relatively dense intermediate point cloud data corresponding to each of the multiple key frame groups are obtained. Then, the multiple intermediate point cloud data are registered again, and the registered intermediate point cloud data are spliced to obtain the spliced background rigid body point cloud.
[0086] Optionally, the GICP algorithm may be used to iteratively register multiple intermediate point cloud data.
[0087] As mentioned above, during the acquisition of point cloud data, the vibration of the radar device and the vehicle body will cause the point cloud to fluctuate greatly in continuous frames. Directly splicing the background rigid point cloud in multiple key frames will cause stratification problems. In the disclosed embodiment, the background rigid point cloud of each group of key frames is registered, and the point cloud fragments of multiple key frame groups are registered in turn. The group registration method can effectively improve the registration accuracy and avoid stratification problems.
[0088] Figure 6 FIG. 1 is a flow chart of a densification process for a foreground non-rigid point cloud provided by an embodiment of the present disclosure. Figure 6 As shown, the densification processing of the foreground non-rigid point cloud provided in the embodiment of the present disclosure includes the following steps.
[0089] S610, respectively determine second point cloud data corresponding to a preset number of non-key frames before and / or after the target key frame as adjacent frame point clouds.
[0090] The target key frame is one of the multiple key frames, and the target key frame includes a foreground non-rigid target. The first point cloud data corresponding to the target key frame includes a foreground non-rigid point cloud corresponding to the foreground non-rigid target.
[0091] The multi-frame point cloud data also includes the second point cloud data corresponding to each of the multi-frame non-key frames. The multi-frame key frames and the multi-frame non-key frames are arranged in chronological order, a preset number of non-key frames before and / or after the target key frame are determined, and the second point cloud data corresponding to each of the determined non-key frames are used as adjacent frame point clouds.
[0092] For example, two non-key frames before the target key frame and two non-key frames after the target key frame may be taken. Alternatively, one non-key frame before the target key frame and three non-key frames after the target key frame may be taken. The specific value of the preset number may be set according to actual conditions, and the present disclosure does not impose any specific restrictions on this.
[0093] S620, based on the coordinate information of the foreground non-rigid point cloud corresponding to the target key frame, search the adjacent frame point cloud to obtain the adjacent frame target point cloud.
[0094] Since the second point cloud data of non-key frames does not have corresponding semantic information, it is impossible to directly register the multi-frame foreground non-rigid point clouds through semantic information. Therefore, it is necessary to use the nearest neighbor search method to perform a radius search on the adjacent frame point cloud based on the coordinate information of the foreground non-rigid point cloud in the target key frame, and obtain the points near the coordinate information position in the adjacent frame point cloud as the adjacent frame target point cloud.
[0095] The target point clouds of adjacent frames can be regarded as the point cloud data corresponding to the foreground non-rigid target in the non-key frame.
[0096] S630, obtaining a densified foreground non-rigid point cloud based on the target point cloud and the foreground non-rigid point cloud of adjacent frames.
[0097] The foreground non-rigid targets in mining scenes are usually non-rigid targets with slow movement speeds, such as excavators. Therefore, although the deviation of the foreground non-rigid point cloud between key frames is large, the deviation between adjacent frames is small. That is, the deviation between the adjacent frame target point cloud determined by the above steps and the foreground non-rigid point cloud of the target key frame is small. Therefore, the foreground non-rigid point cloud can be densified by aggregating the adjacent frame target point cloud and the foreground non-rigid point cloud to obtain the densified foreground non-rigid point cloud.
[0098] The target point clouds of adjacent frames are obtained by searching. During this process, some background point clouds may be searched and mistakenly regarded as the point clouds corresponding to the foreground non-rigid targets in non-key frames. Therefore, some background point clouds may be mistakenly aggregated with foreground non-rigid point clouds in this step. To reduce such problems, only one aggregation operation is performed for each foreground non-rigid target in the target key frame.
[0099] In the disclosed embodiment, a method for densifying foreground non-rigid point cloud is introduced. According to the motion characteristics of foreground non-rigid target, part of point cloud data in adjacent frames is selected to realize densification of foreground non-rigid point cloud.
[0100] Figure 7 FIG. 1 is a flow chart of the steps of obtaining a densified foreground non-rigid point cloud based on adjacent frame target point clouds and foreground non-rigid point clouds provided by an embodiment of the present disclosure. Figure 7 As shown, the step of obtaining a densified foreground non-rigid point cloud based on adjacent frame target point clouds and foreground non-rigid point clouds provided in the embodiment of the present disclosure includes the following steps.
[0101] S631, calculating the angle between the vector line between the foreground non-rigid point cloud and the adjacent frame target point cloud and the height direction.
[0102] The vector line between the foreground non-rigid point cloud and the adjacent frame target point cloud refers to the line between the point in the foreground non-rigid point cloud and the corresponding point in the adjacent frame target point cloud. The height direction is the direction of the z-axis.
[0103] Calculate the angle between the vector line and the height direction between the foreground non-rigid point cloud and the target point cloud of the adjacent frame.
[0104] S632, when the included angle is less than a preset angle, the foreground non-rigid point cloud and the adjacent frame target point cloud are spliced to obtain a densified foreground non-rigid point cloud.
[0105] As mentioned above, in the process of searching adjacent frame point clouds, some background point clouds may be searched. In mining scenes, background point clouds are usually ground point clouds or retaining wall point clouds. To prevent the background point cloud from being aggregated with the foreground non-rigid point cloud, the angle between the vector line and the height direction can be used to determine whether the searched point is a background point cloud.
[0106] Specifically, when the angle between the vector line and the height direction is less than a preset angle, the point is considered not to be a background point cloud, and the point can be added to the foreground non-rigid point cloud. The preset angle can be set according to actual conditions, and for example, the preset angle can be set to 90 degrees.
[0107] In the embodiment of the present disclosure, part of the point cloud that may be the background point cloud is filtered out from the adjacent frame target point cloud by using the angle between the vector line and the height direction between the foreground non-rigid point cloud and the adjacent frame target point cloud, so as to avoid adding the ground point cloud or the retaining wall point cloud to the foreground non-rigid point cloud during the aggregation process.
[0108] The above introduces the densification methods corresponding to different types of point clouds. Based on the densified point cloud data, the occupancy network model corresponding to the target space area can be generated. The following further introduces the evaluation method of the occupancy network model.
[0109] The ground and retaining wall are the two most common categories in the open-pit mine scene, and the plane normal angles of these two categories are large, which can provide rich registration quality information. The following introduces the evaluation methods of the ground-based and retaining wall-based occupancy network models respectively.
[0110] Figure 8 FIG. 2 is a flow chart of a point cloud processing method provided by another embodiment of the present disclosure. Figure 8 As shown, after the step of generating an occupancy network model corresponding to the target space area, the point cloud processing method provided in the embodiment of the present disclosure also includes the following steps.
[0111] S810, traverse each horizontal coordinate of the occupancy network model to determine the number of grid pillars containing ground voxels.
[0112] In the occupancy network model, for each generated valid voxel truth value, it is divided into columnar structural units according to the horizontal coordinates (x, y), namely the grid pillars (hereinafter referred to as pillars). The horizontal coordinates of the valid voxel truth values included in each pillar are consistent, but the height coordinates are different.
[0113] Traverse each horizontal coordinate of the occupancy network model, determine whether each pillar contains ground voxels, and count the number of grid pillars containing ground voxels, recorded as N.
[0114] S820, determine the number of ground voxels in the pillar that includes the ground voxels.
[0115] Next, the number of ground voxels in the pillar that contains the ground voxel is determined, denoted as n.
[0116] Exemplarily, the number of pillars containing ground voxels is 2, one of which includes 3 ground voxels, and the other includes 1 ground voxel; in this case, the number of pillars containing ground voxels is determined to be 2, and the number of ground voxels in the pillar containing ground voxels is determined to be 4.
[0117] S830 , determining an average ground thickness of the occupancy network model based on the number of pillars including ground voxels and the number of ground voxels.
[0118] The ratio n / N of the number of ground voxels to the number of pillars containing ground voxels is taken as the average ground thickness of the occupancy network model.
[0119] Exemplarily, continuing to refer to the example in the above steps, the number of ground voxels is 4, the number of pillars containing ground voxels is 2, and the average ground thickness can be determined to be 4 / 2=2.
[0120] S840, determine the ground quality of the occupancy network model based on the average ground thickness.
[0121] For the occupancy network model, we expect each pillar to include only one ground voxel. Therefore, the closer the average ground thickness is to 1, the higher the ground quality of the occupancy network model; conversely, the farther the average ground thickness is from 1, the lower the ground quality of the occupancy network model.
[0122] Therefore, after obtaining the average ground thickness, the ground quality of the occupancy network model can be evaluated based on the average ground thickness.
[0123] In the embodiment of the present disclosure, a method for evaluating an occupancy network model based on ground voxels is provided. The method is designed according to the characteristics of the mine scene and can more objectively and quantitatively evaluate the quality of the occupancy network model.
[0124] Fig. 9 FIG. 2 is a flow chart of a point cloud processing method provided by another embodiment of the present disclosure. Fig. 9 As shown, after the step of generating an occupancy network model corresponding to the target space area, the point cloud processing method provided in the embodiment of the present disclosure also includes the following steps.
[0125] S910, traverse each height coordinate of the occupancy network model to determine the number of voxel layers containing retaining wall voxels.
[0126] In the occupancy network model, for each generated valid voxel truth value, it is divided into layered structural units, i.e., voxel layers, according to the height coordinate z. The height coordinates of the valid voxel truth values included in each voxel layer are consistent, but the horizontal coordinates are different.
[0127] Traverse each height coordinate of the occupancy network model, determine whether each voxel layer contains a retaining wall voxel, and count the number of voxel layers containing retaining wall voxels, recorded as M.
[0128] S920, determining the number of barrier wall voxels in the voxel layer containing the barrier wall voxels.
[0129] Next, the number of barrier voxels in the voxel layer containing barrier voxels is determined, which is denoted by m.
[0130] Exemplarily, the number of voxel layers containing barrier wall voxels is 2, one voxel layer includes 2 barrier wall voxels, and the other voxel layer includes 3 barrier wall voxels; at this time, the number of voxel layers containing barrier wall voxels is determined to be 2, and the number of barrier wall voxels in the voxel layer containing barrier wall voxels is determined to be 5.
[0131] S930, determining an average retaining wall area of the occupancy network model based on the number of voxel layers containing retaining wall voxels and the number of retaining wall voxels.
[0132] The ratio m / M of the number of retaining wall voxels to the number of voxel layers containing retaining wall voxels is taken as the average retaining wall area of the occupancy network model.
[0133] Exemplarily, continuing to refer to the example in the above steps, the number of retaining wall voxels is 5, and the number of voxel layers containing retaining wall voxels is 2. It can be determined that the average retaining wall thickness is 5 / 2=2.5.
[0134] S940, determine the retaining wall quality of the occupancy network model based on the average retaining wall area.
[0135] For the occupancy network model, the average retaining wall area is negatively correlated with the retaining wall quality. That is, the larger the average retaining wall area, the worse the retaining wall quality. Therefore, after obtaining the average retaining wall area, the retaining wall quality of the occupancy network model can be evaluated based on the average retaining wall area.
[0136] In the embodiment of the present disclosure, a method for evaluating an occupancy network model based on retaining wall voxels is provided. The method is designed according to the characteristics of the mine scene and can more objectively and quantitatively evaluate the quality of the occupancy network model.
[0137] Optionally, after determining the average retaining wall area and the average ground thickness, the quality of the occupancy network model can be comprehensively evaluated based on these two indicators. Specifically, if the ground quality is poor, the true value of the valid voxels is relatively scattered in the vertical direction, which leads to the problem of low retaining wall area. To avoid this problem, it is necessary to comprehensively evaluate the quality of the occupancy network model based on the ground thickness and the average retaining wall area.
[0138] Combination of the above Figures 1 to 9 The method embodiment of the present disclosure is described in detail. Fig.10 The device embodiment of the present disclosure is described in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.
[0139] Fig.10 FIG. 1 is a schematic diagram of the structure of a point cloud processing device provided by an embodiment of the present disclosure. Fig.10 As shown, the point cloud processing device 1000 of the embodiment of the present disclosure includes: an acquisition module 1010, a determination module 1020, a division module 1030, a processing module 1040 and a generation module 1050.
[0140] Specifically, the acquisition module 1010 is configured to acquire multi-frame point cloud data corresponding to the target space area, the multi-frame point cloud data includes first point cloud data corresponding to each of the multi-frame key frames, and the first point cloud data has corresponding semantic information. The determination module 1020 is configured to determine the foreground target and the background target in the key frame based on the semantic information. The division module 1030 is configured to perform rigid body judgment on the foreground target and the background target respectively, divide the first point cloud data corresponding to the foreground target into a foreground rigid point cloud and a foreground non-rigid point cloud, and divide the first point cloud data corresponding to the background target into a background rigid point cloud and a background non-rigid point cloud. The processing module 1040 is configured to perform densification processing on the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud respectively. The generation module 1050 is configured to generate an occupancy network model corresponding to the target space area based on the densified foreground rigid point cloud, the densified foreground non-rigid point cloud, the densified background rigid point cloud, and the background non-rigid point cloud.
[0141] In some embodiments, the point cloud processing device 1000 further includes a first evaluation module. The first evaluation module is configured to traverse each horizontal coordinate of the occupancy network model, determine the number of grid pillars containing ground voxels; determine the number of ground voxels in the pillar containing ground voxels; determine the average ground thickness of the occupancy network model based on the number of pillars containing ground voxels and the number of ground voxels; and determine the ground quality of the occupancy network model based on the average ground thickness.
[0142] In some embodiments, the point cloud processing device 1000 further includes a second evaluation module. The second evaluation module is configured to traverse each height coordinate of the occupancy network model, determine the number of voxel layers containing retaining wall voxels; determine the number of retaining wall voxels in the voxel layers containing retaining wall voxels; determine the average retaining wall area of the occupancy network model based on the number of voxel layers containing retaining wall voxels and the number of retaining wall voxels; and determine the retaining wall quality of the occupancy network model based on the average retaining wall area.
[0143] Below, reference Fig.11 An electronic device according to an embodiment of the present disclosure is described. Fig.11 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Fig.11 As shown, electronic device 1100 includes one or more processors 1110 and memory 1120 .
[0144] The processor 1110 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1100 to perform desired functions.
[0145] The memory 1120 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1110 may execute the program instructions to implement the point cloud processing method of each embodiment of the present disclosure described above and / or other desired functions.
[0146] In some embodiments, the electronic device 1100 may further include: an input device 1130 and an output device 1140 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0147] The input device 1130 may include, for example, a touch screen, a microphone, a keyboard, a mouse, etc. The output device 1140 may include, for example, a display, a speaker, a communication network and a remote output device connected thereto.
[0148] Of course, to simplify, Fig.11 Only some of the components related to the present disclosure in the electronic device 1100 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application scenarios, the electronic device 1100 may also include any other appropriate components.
[0149] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the point cloud processing method according to various embodiments of the present disclosure described above in this specification.
[0150] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0151] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the point cloud processing method according to various embodiments of the present disclosure described above in this specification.
[0152] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0153] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0154] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.
[0155] It should also be noted that in the system, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0156] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0157] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A point cloud processing method, characterized in that: include: Acquire multi-frame point cloud data corresponding to the target spatial area, wherein the multi-frame point cloud data includes first point cloud data corresponding to each of the multiple key frames, and the first point cloud data contains corresponding semantic information; Based on the semantic information, determining a foreground object and a background object in the key frame; Performing rigid body judgment on the foreground target and the background target respectively, dividing the first point cloud data corresponding to the foreground target into a foreground rigid body point cloud and a foreground non-rigid body point cloud, and dividing the first point cloud data corresponding to the background target into a background rigid body point cloud and a background non-rigid body point cloud; Densifying the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud respectively; Based on the densified foreground rigid point cloud, the densified foreground non-rigid point cloud, the densified background rigid point cloud, and the background non-rigid point cloud, an occupancy network model corresponding to the target space area is generated.
2. The point cloud processing method according to claim 1, characterized in that: The background rigid body point cloud is densified, including: According to the positioning information corresponding to each of the multiple key frames, a first registration is performed on the background rigid body point clouds of each of the multiple key frames; Reduce the coordinates of the background rigid body point cloud after the first registration to one-s of the original, where s is a positive real number; Perform secondary registration on the background rigid body point cloud after coordinate adjustment, and splice the background rigid body point cloud after secondary registration to obtain a spliced background rigid body point cloud; Expanding the coordinates of the spliced background rigid body point cloud to s times the original coordinates; and / or, The foreground rigid body point cloud is densified, including: Based on the center positions of the foreground rigid body point clouds of the multiple key frames, the foreground rigid body point clouds of the multiple key frames are aligned; The aligned foreground rigid body point clouds are stitched together.
3. The point cloud processing method according to claim 2, characterized in that: The method of performing secondary registration on the background rigid body point cloud after the coordinate adjustment, and splicing the background rigid body point cloud after the secondary registration to obtain the spliced background rigid body point cloud includes: Grouping the multiple key frames, each key frame group includes a plurality of consecutive key frames; For each key frame group, the background rigid body point clouds of the multiple key frames included in the key frame group are registered, and the registered background rigid body point clouds are spliced to obtain the intermediate point cloud data corresponding to the key frame group; The intermediate point cloud data corresponding to each of the multiple key frame groups are registered, and the registered intermediate point cloud data are spliced to obtain the spliced background rigid body point cloud.
4. The point cloud processing method according to claim 1, characterized in that: The multi-frame point cloud data also includes second point cloud data corresponding to each of the multi-frame non-key frames. The foreground non-rigid point cloud is densified, including: Respectively determine second point cloud data corresponding to a preset number of non-key frames before and / or after the target key frame as adjacent frame point clouds; Based on the coordinate information of the foreground non-rigid point cloud corresponding to the target key frame, searching the adjacent frame point cloud to obtain the adjacent frame target point cloud; Based on the adjacent frame target point clouds and the foreground non-rigid point clouds, the densified foreground non-rigid point clouds are obtained.
5. The point cloud processing method according to claim 4, characterized in that: The step of obtaining the densified foreground non-rigid point cloud based on the adjacent frame target point cloud and the foreground non-rigid point cloud comprises: Calculating the angle between the vector line between the foreground non-rigid point cloud and the adjacent frame target point cloud and the height direction; When the angle is smaller than a preset angle, the foreground non-rigid point cloud and the adjacent frame target point cloud are spliced to obtain the densified foreground non-rigid point cloud.
6. The point cloud processing method according to claim 1, characterized in that: After generating the occupancy network model corresponding to the target space area, the method further includes: Traversing each horizontal coordinate of the occupancy network model, determining the number of grid pillars containing ground voxels; Determining the number of ground voxels in the pillar containing the ground voxels; Determining an average ground thickness of the occupancy network model based on the number of the pillars containing ground voxels and the number of the ground voxels; Based on the average ground thickness, a ground quality of the occupancy network model is determined.
7. The point cloud processing method according to claim 1, characterized in that: After generating the occupancy network model corresponding to the target space area, the method further includes: traversing each height coordinate of the occupancy network model to determine the number of voxel layers containing retaining wall voxels; determining the number of barrier wall voxels in the voxel layer containing barrier wall voxels; determining an average retaining wall area of the occupancy network model based on the number of voxel layers including retaining wall voxels and the number of retaining wall voxels; Based on the average retaining wall area, a retaining wall quality of the occupancy network model is determined.
8. A point cloud processing device, characterized in that: include: An acquisition module is configured to acquire multiple frames of point cloud data corresponding to the target space area, wherein the multiple frames of point cloud data include first point cloud data corresponding to each of the multiple key frames, and the first point cloud data contains corresponding semantic information; A determination module, configured to determine a foreground object and a background object in the key frame based on the semantic information; A division module is configured to perform rigid body judgment on the foreground target and the background target respectively, divide the first point cloud data corresponding to the foreground target into a foreground rigid body point cloud and a foreground non-rigid body point cloud, and divide the first point cloud data corresponding to the background target into a background rigid body point cloud and a background non-rigid body point cloud; A processing module is configured to perform densification processing on the foreground rigid point cloud, the foreground non-rigid point cloud, and the background rigid point cloud respectively; The generation module is configured to generate an occupancy network model corresponding to the target space area based on the foreground rigid point cloud after densification, the foreground non-rigid point cloud after densification, the background rigid point cloud after densification, and the background non-rigid point cloud.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the point cloud processing method described in any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the point cloud processing method according to any one of claims 1 to 7 is implemented.