Method and device for reconstructing multi-frame point cloud static scene, computer device and medium

Abstract

The disclosure provides a multi-frame point cloud static scene reconstruction method, device, computer equipment and medium, comprising: acquiring time sequence point cloud data when a vehicle is in an automatic driving state; frame-by-frame detection is performed on the time sequence point cloud data; target tracking is performed on each single-frame scene target to obtain corresponding trajectory attitude data; pose estimation is performed on each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target; based on the trajectory attitude data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, point cloud separation is performed on single-frame scene targets belonging to dynamic targets to obtain a plurality of single-frame static point cloud data; and the plurality of single-frame static point cloud data is spliced based on the single-frame estimated pose corresponding to each single-frame scene target to reconstruct a static point cloud scene not containing dynamic obstacles. Thus, accurate static three-dimensional scene reconstruction is effectively realized.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of three-dimensional scene reconstruction technology, and more specifically, to a method, apparatus, computer device, and medium suitable for reconstructing a multi-frame point cloud static scene. Background Technology

[0002] 4D annotation for autonomous driving scenarios involves labeling 3D data with temporal information. This requires labeling not only static targets in 3D space but also dynamic targets containing temporal information. The purpose is to record the geometric and attribute information of static driving scenes, as well as the appearance, attributes, motion trajectories, and attitude changes of dynamic targets, providing ground truth for autonomous driving perception.

[0003] Since a single frame of laser point cloud can only display information about a local area at a certain moment and cannot capture information about the complete scene, separating dynamic and static targets from a single frame of point cloud can effectively improve the accuracy of dynamic target modeling in autonomous driving scenarios and increase the efficiency of 4D annotation. Therefore, before using laser point cloud for 4D annotation, it is necessary to reconstruct the static 3D scene based on multiple frames of data and separate dynamic and static targets for each frame of laser point cloud data. However, traditional laser point cloud static scene detection can result in false detections, reducing the accuracy of single-frame dynamic and static point cloud separation and thus affecting the static point cloud reconstruction effect. Summary of the Invention

[0004] The embodiments described herein provide a method, apparatus, computer device, and medium for reconstructing a multi-frame point cloud static scene, overcoming the aforementioned problems.

[0005] Firstly, based on the content of this disclosure, a method for reconstructing a multi-frame point cloud static scene is provided, including:

[0006] Acquire temporal point cloud data of a vehicle in autonomous driving mode, wherein the temporal point cloud data includes: multiple frames of continuous laser point cloud data;

[0007] The temporal point cloud data is detected frame by frame to determine the corresponding single-frame scene target from each frame of the laser point cloud data;

[0008] For each single-frame scene target, target tracking is performed to obtain corresponding trajectory and pose data, which is used to describe the motion trajectory and motion pose of the single-frame scene target.

[0009] Perform pose estimation on each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target;

[0010] Based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, point cloud separation is performed on the single-frame scene targets that belong to dynamic targets to obtain multiple single-frame static point cloud data.

[0011] Multiple static point cloud data of each single frame are stitched together based on the single frame estimated pose corresponding to each single frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0012] Secondly, according to the present disclosure, a device for reconstructing a multi-frame point cloud static scene is provided, comprising:

[0013] The acquisition module is used to acquire time-series point cloud data of the vehicle in autonomous driving mode, the time-series point cloud data including: multiple frames of continuous laser point cloud data;

[0014] The detection module is used to perform frame-by-frame detection on the time-series point cloud data to determine the corresponding single-frame scene target from each frame of the laser point cloud data.

[0015] The tracking module is used to track each target in the single-frame scene and obtain corresponding trajectory and pose data. The trajectory and pose data is used to describe the motion trajectory and motion pose of the target in the single-frame scene.

[0016] The estimation module is used to perform pose estimation for each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target;

[0017] The determination module is used to perform point cloud separation on the single-frame scene targets that belong to dynamic targets based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, so as to obtain multiple single-frame static point cloud data.

[0018] The stitching module is used to stitch together multiple static point cloud data of each single frame based on the single frame estimated pose corresponding to each single frame scene target, so as to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0019] Thirdly, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method for reconstructing a multi-frame point cloud static scene as described in any of the above embodiments.

[0020] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the method for reconstructing a multi-frame point cloud static scene as described in any of the above embodiments.

[0021] The method for reconstructing a multi-frame static point cloud scene provided in this application embodiment acquires temporal point cloud data of a vehicle in autonomous driving mode. The temporal point cloud data includes: multiple consecutive frames of laser point cloud data; performs frame-by-frame detection on the temporal point cloud data to determine the corresponding single-frame scene target from each frame of laser point cloud data; performs target tracking on each single-frame scene target to obtain corresponding trajectory and pose data, which is used to describe the motion trajectory and motion pose of the single-frame scene target; performs pose estimation on each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target; based on the trajectory and pose data and the single-frame estimated pose corresponding to each single-frame scene target, performs point cloud separation on the single-frame scene targets that belong to dynamic targets to obtain multiple single-frame static point cloud data; and stitches together the multiple single-frame static point cloud data based on the single-frame estimated pose corresponding to each single-frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles. In this way, the dynamic and static point clouds of a single frame can be accurately separated by time-series target tracking information, and the accuracy of laser point cloud pose estimation results can be improved by using target detection, thus effectively achieving accurate static 3D scene reconstruction.

[0022] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure, wherein:

[0024] Figure 1 This is a flowchart illustrating a method for reconstructing a static scene from a multi-frame point cloud, as disclosed in this publication.

[0025] Figure 2 This is a schematic diagram of the structure of a multi-frame point cloud static scene reconstruction device provided in this disclosure.

[0026] Figure 3 This is a schematic diagram of the structure of a computer device provided in this disclosure.

[0027] It should be noted that the elements in the attached diagram are schematic and not drawn to scale. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0029] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, the statement of “connecting” or “coupling” two or more parts together shall mean that these parts are directly joined together or joined through one or more intermediate components.

[0030] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of the phrase "embodiment" in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0031] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists, A and B exist simultaneously, or B exists. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Terms such as "first" and "second" are only used to distinguish one component (or part of a component) from another component (or another part of a component).

[0032] In the description of this application, unless otherwise stated, "multiple" means two or more (including two), and similarly, "multiple groups" means two or more (including two groups).

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0034] Figure 1 This is a flowchart illustrating a method for reconstructing a multi-frame point cloud static scene according to an embodiment of this disclosure, as shown below. Figure 1As shown, the specific process of the multi-frame point cloud static scene reconstruction method includes:

[0035] S110. Acquire temporal point cloud data of the vehicle in autonomous driving mode. The temporal point cloud data includes: multiple frames of continuous laser point cloud data.

[0036] Among them, the multi-frame continuous laser point cloud data are laser point cloud data collected at different times, that is, different laser point cloud data correspond to different collection times.

[0037] In some embodiments, before performing frame-by-frame detection on the temporal point cloud data, the method of this embodiment further includes:

[0038] Based on the motion pose and acquisition timestamp of each laser point cloud data at the acquisition time, the estimated pose of each laser point cloud data at the acquisition timestamp is obtained; based on the estimated pose of each laser point cloud data at the acquisition timestamp, distortion correction processing is performed on each laser point cloud data to update the temporal point cloud data.

[0039] In the process of distorting laser point cloud data, a uniform velocity model is used for point cloud distortion processing. It is assumed that the carrier moves at a constant speed and a constant angular velocity. The pose at the start and end of the acquisition time of a single frame of point cloud is used as the input of the uniform velocity model. Combined with the timestamp of each point, the pose of the carrier at that timestamp can be calculated. Then, coordinate transformation is performed on the point to complete the point cloud distortion correction.

[0040] Based on the motion pose and acquisition timestamp of each laser point cloud data at the acquisition time, the estimated pose of each laser point cloud data at the acquisition timestamp can be obtained. This can include: inputting the motion pose and acquisition timestamp of the laser point cloud data at the acquisition time into a uniform velocity model, and determining the estimated pose of the laser point cloud data at the acquisition timestamp based on the output of the uniform velocity model.

[0041] Based on the estimated pose of each laser point cloud data at the acquisition timestamp, distortion correction processing is performed on each laser point cloud data. This may include: transforming the estimated pose of the laser point cloud data at the acquisition timestamp into the world coordinate system, and comparing the error between the transformed estimated pose and the actual pose of the laser point cloud data at the acquisition timestamp. If the error is greater than a preset difference, the laser point cloud data is considered distorted data and is removed; if the error is less than or equal to the preset difference, the laser point cloud data is considered not distorted data and is retained.

[0042] Therefore, by performing distortion correction on the laser point cloud data before performing frame-by-frame detection on the temporal point cloud data, the accuracy of the laser point cloud data for frame-by-frame detection is effectively guaranteed, and the impact on the detection of targets in a single frame scene is avoided.

[0043] S120. Perform frame-by-frame detection on the temporal point cloud data to determine the corresponding single-frame scene target from each frame of laser point cloud data.

[0044] In the frame-by-frame detection of temporal point cloud data, a multi-stage 3D object detection framework based on Cascade Attention Network (CasA) can be implemented. CasA consists of a Region Proposal Network (RPN) and a Cascaded Refinement Network (CRN). The CRN includes a novel cascaded attention module that uses multiple sub-networks and attention modules to aggregate object features from different stages and progressively refine the region generation. This detection method can detect targets such as vehicles, pedestrians, and cars / humans from a single frame of point cloud data.

[0045] S130. Perform target tracking on each single-frame scene target to obtain the corresponding trajectory and attitude data.

[0046] Among them, trajectory and pose data are used to describe the motion trajectory and motion pose of the target in a single frame scene.

[0047] The target tracking tracker is a data association tracker guided by prediction confidence. It uses a constant acceleration (CA) motion model to estimate future positions and outputs prediction confidence for data association. Furthermore, it leverages the geometric, appearance, and motion features of point cloud targets to achieve faster and more accurate data association. Target tracking can determine whether detected targets in multiple frames of point clouds are the same target, and the motion state of each detected target.

[0048] In some embodiments, target tracking is performed on the target in each single-frame scene to obtain the corresponding trajectory and pose data, including:

[0049] Motion modeling is performed on each single-frame scene target based on a preset motion model to obtain the motion pose of each single-frame scene target at different times; two single-frame scene targets with a preset single-frame relationship are obtained; data association is performed on the two single-frame scene targets with a preset single-frame relationship to obtain the motion trajectory corresponding to the same single-frame scene target.

[0050] The preset motion model is the CA motion model, which can model the motion of the detected target and estimate its position and pose at each moment. For example, the constant acceleration model assumes that the vehicle's speed is variable, but its acceleration is constant. Compared with the uniform motion model, it can estimate the vehicle's pose more accurately. The input of the CA motion model is the single-frame scene target of the vehicle and its single-frame motion pose, and the output is the moment-to-moment motion pose of the single-frame scene target at each moment.

[0051] The preset single-frame relationship is used to describe two single-frame scene targets in adjacent frames as the same scene target.

[0052] For example, if single-frame scene target A1 and single-frame scene target A2 are adjacent frames, and single-frame scene target A1 is the target in the previous frame of single-frame scene target A2, and single-frame scene target A1 and single-frame scene target A2 are the same target (such as the same pedestrian), then single-frame scene target A1 and single-frame scene target A2 have a preset single-frame relationship.

[0053] The purpose of data association is to determine whether a target in the point cloud of the previous frame and a target in the point cloud of the next frame are the same target. If they are, the data of the two are associated and identified, and the trajectory and attitude of the target in the time series can be obtained based on the association results of multiple frames of data.

[0054] Data association can be performed on two single-frame scene targets with a preset single-frame relationship to obtain the motion trajectory corresponding to the same single-frame scene target. This can include: data association and identification of the motion poses of two single-frame scene targets with a preset single-frame relationship corresponding to different times (such as sorting them according to time sequence), so as to effectively obtain the motion trajectory and motion posture of the single-frame scene target in the time series.

[0055] In some embodiments, obtaining two single-frame scene targets with a preset single-frame relationship includes:

[0056] Obtain two single-frame scene targets that are in adjacent frames; based on the appearance attribute data of the two single-frame scene targets that are in adjacent frames, determine whether the two single-frame scene targets that are in adjacent frames are the same scene target, so as to obtain two single-frame scene targets with a preset single-frame relationship.

[0057] The appearance attribute data may include, but is not limited to: geometric shape and appearance characteristics (such as color, size, etc.).

[0058] Determining whether two single-frame scene targets in adjacent frames belong to the same scene target based on their appearance attribute data can include: 1) Based on the appearance attribute data of the target in the preceding frame, inferring the appearance attribute data of the target in the next frame corresponding to the target in the preceding frame based on its movement speed. If the similarity between the appearance attribute data of the target in the next frame corresponding to the target in the preceding frame calculated based on its movement speed and the appearance attribute data of the target in the following frame of the two adjacent single-frame scene targets meets a preset threshold, then the two single-frame scene targets in the adjacent frames are determined to be the same scene target. 2) If the similarity between the appearance attribute data of the target in the next frame corresponding to the target in the preceding frame calculated based on its movement speed and the appearance attribute data of the target in the following frame of the two adjacent single-frame scene targets does not meet the preset threshold, then the two single-frame scene targets in the adjacent frames are determined to be different scene targets. Thus, based on the appearance attribute data between adjacent frames, identical scene targets can be effectively identified.

[0059] S140. Perform pose estimation for each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target.

[0060] Specifically, when estimating the pose of a scene target in each single frame, a coarse estimation is first performed based on inter-frame point cloud matching, and then optimized by point cloud sub-map matching to obtain an accurate pose estimation result.

[0061] In some embodiments, pose estimation is performed on the scene target in each single frame to obtain the estimated pose of the scene target in each single frame, including:

[0062] From the laser point cloud data corresponding to the scene target in the previous frame associated with each single-frame scene target, the corresponding target point cloud data is removed to obtain the scene point cloud data corresponding to the scene target in the previous frame associated with each single-frame scene target. Inter-frame point cloud matching is then performed between the scene point cloud data corresponding to the scene target in the previous frame associated with each single-frame scene target and the laser point cloud data corresponding to the single-frame scene target to obtain the initial estimated pose for each single-frame scene target. Point cloud matching is then performed between the laser point cloud data corresponding to each single-frame scene target and a preset point cloud sub-map to obtain the current estimated pose for each single-frame scene target. Based on the initial estimated pose for each single-frame scene target and the current estimated pose for each single-frame scene target, the single-frame estimated pose for each single-frame scene target is determined.

[0063] Specifically, by removing the target point cloud from the laser point cloud data of the target in the previous frame, and retaining only the point cloud corresponding to the background environment, the laser point cloud data of the target in the current frame is used for point cloud registration. In this way, the influence of dynamic obstacles on the point cloud registration accuracy can be eliminated, so that the initial estimated pose obtained by registration is not affected by the point cloud of dynamic obstacles.

[0064] The preset point cloud sub-map is a local point cloud map obtained by aggregating laser point cloud data of the target in the corresponding single frame scene at different times.

[0065] The data coordinate system corresponding to the current estimated pose of the scene target in each single frame is the same as the data coordinate system corresponding to the preset point cloud sub-map. That is, before matching with the preset point cloud sub-map, it is necessary to determine the data coordinate system of the current estimated pose of the scene target in a single frame. If it is not the world coordinate system, it needs to be converted to the world coordinate system to ensure the effectiveness of point cloud matching.

[0066] Based on the initial estimated pose corresponding to each scene target in a single frame, and the current estimated pose corresponding to each scene target in a single frame, the single-frame estimated pose corresponding to each scene target in a single frame is determined. This can include: adjusting the initial estimated pose corresponding to each scene target in a single frame based on the current estimated pose (e.g., fusing the current estimated pose and the initial estimated pose corresponding to the scene target in a single frame according to different weight ratios, the weight ratios can be adaptively adjusted according to the scene), to obtain the single-frame estimated pose corresponding to each scene target in a single frame. This results in a more accurate position estimation result.

[0067] S150. Based on the trajectory and attitude data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, point cloud separation is performed on the single-frame scene targets that belong to dynamic targets to obtain multiple single-frame static point cloud data.

[0068] Each single-frame scene target may be a dynamic target or a static target, and it is necessary to separate the dynamic and static targets to effectively obtain the single-frame static point cloud data.

[0069] In some embodiments, based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, point cloud separation is performed on single-frame scene targets that are dynamic targets to obtain multiple single-frame static point cloud data, including:

[0070] Based on the trajectory and attitude data and the estimated pose of each single-frame scene target, the target type of each single-frame scene target is determined. The target type includes dynamic targets and static targets. Point cloud separation is performed on the laser point cloud data corresponding to the single-frame scene target whose target type is dynamic to obtain multiple single-frame static point cloud data.

[0071] Since the estimated pose of a single-frame scene target corresponds to the world coordinate system, the trajectory and pose data of the single-frame scene target can be pre-converted to the world coordinate system to facilitate subsequent operations.

[0072] Based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, the target type of each single-frame scene target is determined. This can include: if each single-frame estimated pose corresponding to a single-frame scene target is the same as the trajectory estimated pose in its corresponding trajectory and pose data, or the error is negligible (e.g., the error is below a preset threshold), then the single-frame scene target is determined to be at the same position coordinate at all times; if each single-frame estimated pose corresponding to a single-frame scene target is not entirely the same as the trajectory estimated pose in its corresponding trajectory and pose data, or the error is negligible (e.g., the error is below a preset threshold), then the coordinate position of the single-frame scene target changes at a specific time.

[0073] Therefore, by using the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, the target type of each single-frame scene target can be identified, and the laser point cloud data corresponding to the single-frame scene target whose target type is dynamic target can be separated into points cloud, effectively separating the dynamic point cloud and realizing the separation of dynamic and static point clouds.

[0074] In some embodiments, based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, the target type of each single-frame scene target is determined, including:

[0075] If, based on the trajectory and attitude data and the estimated pose of the target in a single frame, it is determined that the target in a single frame is at the same coordinate at all times, then the target type of the target in the single frame is determined to be a static target. If, based on the trajectory and attitude data and the estimated pose of the target in a single frame, it is determined that the target's coordinate position changes at a specific time, then the target type of the target in the single frame is determined to be a dynamic target.

[0076] The specific time can be one or more times during the point cloud data acquisition, which indicates that the target in a single frame scene is not always in a static state, thereby effectively determining the target type of the target in a single frame scene.

[0077] S160. Based on the single-frame estimated pose corresponding to the scene target in each single frame, multiple single-frame static point cloud data are stitched together to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0078] Among them, the estimated pose of a single-frame scene target includes the estimated pose of dynamic targets and the estimated pose of static targets. By using the estimated pose of static targets in each frame, the static point cloud data of each single frame can be sequentially stitched together according to the order of the estimated poses of the single frames, so as to complete the reconstruction of the static point cloud scene without dynamic obstacles.

[0079] In this embodiment, temporal point cloud data of the vehicle in autonomous driving mode is acquired. This temporal point cloud data includes multiple consecutive frames of laser point cloud data. Frame-by-frame detection is performed on the temporal point cloud data to determine the corresponding single-frame scene target from each frame of laser point cloud data. Target tracking is performed on each single-frame scene target to obtain corresponding trajectory and pose data, which describes the motion trajectory and pose of the single-frame scene target. Pose estimation is performed on each single-frame scene target to obtain the single-frame estimated pose. Based on the trajectory and pose data and the single-frame estimated pose of each single-frame scene target, point cloud separation is performed on single-frame scene targets that are dynamic targets to obtain multiple single-frame static point cloud data. Multiple single-frame static point cloud data are stitched together based on the single-frame estimated pose of each single-frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles. Thus, by accurately separating the dynamic and static point clouds of a single frame using time-series target tracking information and improving the accuracy of laser point cloud pose estimation results using target detection, accurate static 3D scene reconstruction is effectively achieved.

[0080] Figure 2 This is a schematic diagram of the structure of a multi-frame point cloud static scene reconstruction device provided in this embodiment. The multi-frame point cloud static scene reconstruction device may include: an acquisition module 210, a detection module 220, a tracking module 230, an estimation module 240, a determination module 250, and a stitching module 260.

[0081] The acquisition module 210 is used to acquire time-series point cloud data of the vehicle in autonomous driving mode. The time-series point cloud data includes: multiple frames of continuous laser point cloud data.

[0082] The detection module 220 is used to perform frame-by-frame detection on the temporal point cloud data in order to determine the corresponding single-frame scene target from each frame of laser point cloud data.

[0083] The tracking module 230 is used to track the target in each single-frame scene and obtain the corresponding trajectory and pose data. The trajectory and pose data is used to describe the motion trajectory and motion pose of the target in the single-frame scene.

[0084] The estimation module 240 is used to perform pose estimation for each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target.

[0085] The determination module 250 is used to perform point cloud separation on the single-frame scene targets that belong to dynamic targets based on the trajectory and attitude data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, so as to obtain multiple single-frame static point cloud data.

[0086] The stitching module 260 is used to stitch together multiple single-frame static point cloud data based on the single-frame estimated pose corresponding to the target in each single-frame scene, so as to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0087] In this embodiment, optionally, the tracking module 230 is specifically used for:

[0088] Motion modeling is performed on each single-frame scene target based on a preset motion model to obtain the motion pose of each single-frame scene target at different times; two single-frame scene targets with a preset single-frame relationship are obtained, and the preset single-frame relationship is used to describe that two single-frame scene targets in adjacent frames are the same scene target; data association is performed on two single-frame scene targets with a preset single-frame relationship to obtain the motion trajectory corresponding to the same single-frame scene target.

[0089] In this embodiment, optionally, the tracking module 230 is specifically used for:

[0090] Obtain two single-frame scene targets that are in adjacent frames; based on the appearance attribute data of the two single-frame scene targets that are in adjacent frames, determine whether the two single-frame scene targets that are in adjacent frames are the same scene target, so as to obtain two single-frame scene targets with a preset single-frame relationship.

[0091] In this embodiment, optionally, the estimation module 240 is specifically used for:

[0092] From the laser point cloud data corresponding to the scene target in the previous frame associated with each single-frame scene target, the corresponding target point cloud data is removed to obtain the scene point cloud data corresponding to the scene target in the previous frame associated with each single-frame scene target. Inter-frame point cloud matching is then performed between the scene point cloud data corresponding to the scene target in the previous frame associated with each single-frame scene target and the laser point cloud data corresponding to the single-frame scene target to obtain the initial estimated pose for each single-frame scene target. Point cloud matching is then performed between the laser point cloud data corresponding to each single-frame scene target and a preset point cloud sub-map to obtain the current estimated pose for each single-frame scene target. The data coordinate system corresponding to the current estimated pose for each single-frame scene target is the same as the data coordinate system corresponding to the preset point cloud sub-map. Based on the initial estimated pose for each single-frame scene target and the current estimated pose for each single-frame scene target, the single-frame estimated pose for each single-frame scene target is determined.

[0093] In this embodiment, optionally, the determining module 250 is specifically used for:

[0094] Based on the trajectory and attitude data and the estimated pose of each single-frame scene target, the target type of each single-frame scene target is determined. The target type includes dynamic targets and static targets. Point cloud separation is performed on the laser point cloud data corresponding to the single-frame scene target whose target type is dynamic to obtain multiple single-frame static point cloud data.

[0095] In this embodiment, optionally, the determining module 250 is specifically used for:

[0096] If, based on the trajectory and attitude data and the estimated pose of the target in a single frame, it is determined that the target in a single frame is at the same coordinate at all times, then the target type of the target in the single frame is determined to be a static target. If, based on the trajectory and attitude data and the estimated pose of the target in a single frame, it is determined that the target's coordinate position changes at a specific time, then the target type of the target in the single frame is determined to be a dynamic target.

[0097] In this embodiment, optionally, an update module is also included.

[0098] The update module is used to obtain the estimated pose of each laser point cloud data at the acquisition time based on the motion pose and acquisition timestamp of each laser point cloud data; based on the estimated pose of each laser point cloud data at the acquisition timestamp, distortion removal processing is performed on each laser point cloud data to update the temporal point cloud data.

[0099] The multi-frame point cloud static scene reconstruction apparatus provided in this disclosure can execute the above method embodiments. Its specific implementation principle and technical effects can be found in the above method embodiments, and will not be repeated here.

[0100] This application also provides a computer device. Please refer to the following for details. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0101] The computer device includes a memory 310 and a processor 320 that are communicatively connected to each other via a system bus. It should be noted that only a computer device with memory 310 and processor 320 is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented alternatively. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0102] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0103] The memory 310 includes at least one type of readable storage medium, including non-volatile memory or volatile memory, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. RAM may include static RAM or dynamic RAM. In some embodiments, the memory 310 may be an internal storage unit of a computer device, such as the hard disk or RAM of the computer device. In other embodiments, the memory 310 may also be an external storage device of the computer device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, or flash card equipped on the computer device. Of course, the memory 310 may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory 310 is typically used to store the operating system and various application software installed on the computer device, such as the program code of the methods described above. Furthermore, the memory 310 may also be used to temporarily store various types of data that have been output or will be output.

[0104] Processor 320 is typically used to perform overall operations of a computer device. In this embodiment, memory 310 is used to store program code or instructions, including computer operation instructions, and processor 320 is used to execute the program code or instructions stored in memory 310 or process data, such as program code that runs the methods described above.

[0105] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus system can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0106] Another embodiment of this application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads computer-readable program code stored in the computer-readable medium, enabling the processor to execute the functional actions specified in each step or combination of steps in the above method; and to generate means for implementing the functional actions specified in each block or combination of blocks in the block diagram.

[0107] Computer-readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any suitable combination thereof, wherein the memory is used to store program code or instructions, the program code including computer operation instructions, and the processor is used to execute the program code or instructions of the above-described methods stored in the memory.

[0108] The definitions of memory and processor can be found in the description of the foregoing computer device embodiments, and will not be repeated here.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0110] In the various embodiments of this application, the functional units or modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" as described in this application does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims listing several means, several units of these means may be embodied by the same item of hardware. The use of "first," "second," and "third," etc., does not indicate any order and these words should be interpreted as names. Unless otherwise specified, the steps in the above embodiments should not be construed as limiting the order of execution.

[0113] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for reconstructing a static scene from multiple point clouds, characterized in that, include: Acquire temporal point cloud data of a vehicle in autonomous driving mode, wherein the temporal point cloud data includes: multiple frames of continuous laser point cloud data; The temporal point cloud data is detected frame by frame to determine the corresponding single-frame scene target from each frame of the laser point cloud data; For each single-frame scene target, target tracking is performed to obtain corresponding trajectory and pose data, which is used to describe the motion trajectory and motion pose of the single-frame scene target. Pose estimation is performed on each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target; the step of performing pose estimation on each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target includes: removing the corresponding target point cloud data from the laser point cloud data corresponding to the previous frame scene target associated with each single-frame scene target to obtain the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target; and respectively connecting the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target with the laser point cloud data corresponding to the single-frame scene target. Inter-frame point cloud matching is performed on cloud data to obtain the initial estimated pose corresponding to each single-frame scene target; the laser point cloud data corresponding to each single-frame scene target is matched with a preset point cloud sub-map to obtain the current estimated pose corresponding to each single-frame scene target, and the data coordinate system corresponding to the current estimated pose corresponding to each single-frame scene target is the same as the data coordinate system corresponding to the preset point cloud sub-map; based on the initial estimated pose corresponding to each single-frame scene target and the current estimated pose corresponding to each single-frame scene target, the single-frame estimated pose corresponding to each single-frame scene target is determined. Based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, point cloud separation is performed on the single-frame scene targets that belong to dynamic targets to obtain multiple single-frame static point cloud data. Multiple static point cloud data of each single frame are stitched together based on the single frame estimated pose corresponding to each single frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles.

2. The method according to claim 1, characterized in that, The step of tracking the target for each single-frame scene target to obtain the corresponding trajectory and pose data includes: Based on a preset motion model, motion modeling is performed on each single-frame scene target to obtain the motion pose of each single-frame scene target at different times. Two single-frame scene targets with a preset single-frame relationship are obtained, wherein the preset single-frame relationship is used to describe that two single-frame scene targets in adjacent frames are the same scene target; Data association is performed on two single-frame scene targets with a preset single-frame relationship to obtain the motion trajectory corresponding to the same single-frame scene target.

3. The method according to claim 2, characterized in that, The step of obtaining two single-frame scene targets with a preset single-frame relationship includes: Obtain the two single-frame scene targets that are in adjacent frames; Based on the appearance attribute data of two single-frame scene targets in adjacent frames, it is determined whether the two single-frame scene targets in adjacent frames are the same scene target, so as to obtain two single-frame scene targets with a preset single-frame relationship.

4. The method according to claim 1, characterized in that, Based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, point cloud separation is performed on the single-frame scene targets that belong to dynamic targets to obtain multiple single-frame static point cloud data, including: Based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, the target type of each single-frame scene target is determined, and the target type includes: dynamic target and static target; Point cloud separation is performed on the laser point cloud data corresponding to the single-frame scene target whose target type is dynamic target, to obtain multiple single-frame static point cloud data.

5. The method according to claim 4, characterized in that, The step of determining the target type of each single-frame scene target based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target includes: If, based on the trajectory and pose data corresponding to the single-frame scene target and the single-frame estimated pose corresponding to the single-frame scene target, it is determined that the single-frame scene target is at the same position coordinate at all times, then the target type of the single-frame scene target is determined to be a static target. If, based on the trajectory and pose data corresponding to the single-frame scene target and the single-frame estimated pose corresponding to the single-frame scene target, it is determined that the coordinate position of the single-frame scene target changes at a specific moment, then the target type of the single-frame scene target is determined to be a dynamic target.

6. The method according to claim 1, characterized in that, Before performing frame-by-frame detection on the temporal point cloud data, the method further includes: Based on the motion pose and acquisition timestamp of each laser point cloud data at the acquisition time, the estimated pose of each laser point cloud data at the acquisition timestamp is obtained. Based on the estimated pose of each laser point cloud data at the acquisition timestamp, distortion correction processing is performed on each laser point cloud data to update the time-series point cloud data.

7. A device for reconstructing a static scene from multiple point clouds, characterized in that, include: The acquisition module is used to acquire time-series point cloud data of the vehicle in autonomous driving mode, the time-series point cloud data including: multiple frames of continuous laser point cloud data; The detection module is used to perform frame-by-frame detection on the time-series point cloud data to determine the corresponding single-frame scene target from each frame of the laser point cloud data. The tracking module is used to track each target in the single-frame scene and obtain corresponding trajectory and pose data. The trajectory and pose data is used to describe the motion trajectory and motion pose of the target in the single-frame scene. An estimation module is used to perform pose estimation for each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target. Specifically, the estimation module is used to: remove the corresponding target point cloud data from the laser point cloud data corresponding to the previous frame scene target associated with each single-frame scene target to obtain the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target; perform inter-frame point cloud matching between the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target and the laser point cloud data corresponding to the single-frame scene target to obtain an initial estimated pose corresponding to each single-frame scene target; perform point cloud matching between the laser point cloud data corresponding to each single-frame scene target and a preset point cloud sub-map to obtain a current estimated pose corresponding to each single-frame scene target, wherein the data coordinate system corresponding to the current estimated pose of each single-frame scene target is the same as the data coordinate system corresponding to the preset point cloud sub-map; and determine the single-frame estimated pose corresponding to each single-frame scene target based on the initial estimated pose and the current estimated pose. The determination module is used to perform point cloud separation on the single-frame scene targets that belong to dynamic targets based on the trajectory and pose data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, so as to obtain multiple single-frame static point cloud data. The stitching module is used to stitch together multiple static point cloud data of each single frame based on the single frame estimated pose corresponding to each single frame scene target, so as to reconstruct a static point cloud scene that does not contain dynamic obstacles.

8. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method for reconstructing a multi-frame point cloud static scene as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When a computer program is executed by a processor, it implements the method for reconstructing a multi-frame point cloud static scene as described in any one of claims 1 to 6.

Images