Method and device for reconstructing multi-frame point cloud static scene, computer equipment and medium

Abstract

The invention provides a multi-frame point cloud static scene reconstruction method and device, computer equipment and a medium. The method comprises the steps of obtaining time sequence point cloud data of a vehicle in an automatic driving state; performing frame-by-frame detection on the time sequence point cloud data; performing target tracking on each single-frame scene target to obtain corresponding track attitude data; performing pose estimation on each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target; based on the track attitude data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, carrying out point cloud separation on the single-frame scene target belonging to the dynamic target to obtain a plurality of single-frame static point cloud data; and splicing a plurality of single-frame static point cloud data based on the single-frame estimated pose corresponding to each single-frame scene target so as to reconstruct a static point cloud scene which does not contain the dynamic obstacle. Therefore, accurate static three-dimensional scene reconstruction is effectively realized.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of three-dimensional scene reconstruction, and in particular, to a reconstruction method, apparatus, computer equipment, and medium applicable to a multi-frame point cloud static scene. Background Art

[0002] 4D annotation of autonomous driving scenes is the annotation of 3D data with time series information. It not only needs to annotate static objects in 3D space, but also dynamic objects with time series information. Its purpose is to record the geometry and attribute information of static driving scenes, as well as the appearance, attributes, motion trajectory, and posture changes of dynamic objects, so as to provide true values ​​for autonomous driving perception.

[0003] Since a single-frame laser point cloud can only display information about a local area at a certain moment and cannot obtain information about the entire scene, separating the dynamic and static targets of a single-frame point cloud can effectively improve the accuracy of dynamic target modeling in autonomous driving scenarios and improve the efficiency of 4D annotation. Therefore, before using laser point clouds for 4D annotation, it is necessary to reconstruct static three-dimensional scenes based on multi-frame data and separate dynamic and static targets for each frame of laser point cloud data. However, when detecting dynamic and static point clouds, traditional laser point cloud static scenes will have false detections, which will reduce the accuracy of single-frame dynamic and static point cloud separation, and thus affect the reconstruction effect of static point clouds. Summary of the invention

[0004] The embodiments described herein provide a method, apparatus, computer device, and medium for reconstructing a multi-frame point cloud static scene, which overcome the above-mentioned problems.

[0005] In a first aspect, according to the present disclosure, a method for reconstructing a static scene of a multi-frame point cloud is provided, comprising:

[0006] Acquire time-series point cloud data when the vehicle is in an automatic driving state, wherein the time-series point cloud data includes: multiple frames of continuous laser point cloud data;

[0007] Performing frame-by-frame detection on the time-series point cloud data to determine a corresponding single-frame scene target from each frame of the laser point cloud data;

[0008] Tracking each of the single-frame scene targets to obtain corresponding trajectory posture data, wherein the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target;

[0009] Performing pose estimation on each of the single-frame scene targets to obtain a single-frame estimated pose corresponding to each of the single-frame scene targets;

[0010] Based on the trajectory posture data corresponding to each of the single-frame scene targets and the single-frame estimated posture corresponding to each of the single-frame scene targets, point cloud separation is performed on the single-frame scene targets belonging to the dynamic targets to obtain a plurality of single-frame static point cloud data;

[0011] Based on the single-frame estimated pose corresponding to each single-frame scene target, multiple single-frame static point cloud data are spliced ​​to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0012] In a second aspect, according to the content of the present disclosure, a device for reconstructing a static scene of a multi-frame point cloud is provided, comprising:

[0013] An acquisition module is used to acquire time-series point cloud data when the vehicle is in an automatic driving state, wherein the time-series point cloud data includes: multiple frames of continuous laser point cloud data;

[0014] A detection module, used to detect the time-series point cloud data frame by frame, so as to determine the corresponding single-frame scene target from each frame of the laser point cloud data;

[0015] A tracking module, used to track each of the single-frame scene targets to obtain corresponding trajectory posture data, wherein the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target;

[0016] An estimation module, used to estimate the pose of each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target;

[0017] A determination module, configured to separate the point cloud of the single-frame scene target belonging to the dynamic target based on the trajectory posture data corresponding to each of the single-frame scene targets and the single-frame estimated posture corresponding to each of the single-frame scene targets, so as to obtain a plurality of single-frame static point cloud data;

[0018] The splicing module is used to splice multiple single-frame static point cloud data based on the single-frame estimated posture corresponding to each single-frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0019] In a third aspect, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the method for reconstructing a multi-frame point cloud static scene in any of the above embodiments are implemented.

[0020] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for reconstructing a multi-frame point cloud static scene in any of the above embodiments are implemented.

[0021] The method for reconstructing a multi-frame point cloud static scene provided in an embodiment of the present application obtains time-series point cloud data when the vehicle is in an automatic driving state, and the time-series point cloud data includes: multiple frames of continuous laser point cloud data; frame-by-frame detection of the time-series point cloud data is performed to determine the corresponding single-frame scene target from each frame of laser point cloud data; target tracking is performed on each single-frame scene target to obtain corresponding trajectory posture data, and the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target; posture estimation is performed on each single-frame scene target to obtain a single-frame estimated posture corresponding to each single-frame scene target; based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target, point cloud separation is performed on single-frame scene targets that are dynamic targets to obtain multiple single-frame static point cloud data; based on the single-frame estimated posture corresponding to each single-frame scene target, multiple single-frame static point cloud data are spliced ​​to reconstruct a static point cloud scene that does not contain dynamic obstacles. In this way, the single-frame dynamic and static point clouds are accurately separated through time series target tracking information, and target detection is used to improve the accuracy of laser point cloud pose estimation results, effectively realizing accurate static three-dimensional scene reconstruction.

[0022] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below. It should be noted that the drawings described below only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure, wherein:

[0024] Figure 1 It is a flowchart of a method for reconstructing a multi-frame point cloud static scene provided by the present invention.

[0025] Figure 2 It is a structural schematic diagram of a multi-frame point cloud static scene reconstruction device provided by the present invention.

[0026] Figure 3 It is a structural schematic diagram of a computer device provided by the present disclosure.

[0027] It should be noted that the elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work also fall within the scope of protection of the present disclosure.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by a person skilled in the art to which the subject matter of the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the specification and the relevant art, and will not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, a statement that two or more parts are "connected" or "coupled" together shall mean that the parts are joined together directly or through one or more intermediate components.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiments" in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists, A and B exist at the same time, and B exists. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. Terms such as "first" and "second" are only used to distinguish one component (or a part of a component) from another component (or another part of a component).

[0032] In the description of the present application, unless otherwise specified, "plurality" means more than two (including two), and similarly, "multiple groups" means more than two groups (including two).

[0033] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0034] Figure 1 is a flowchart of a method for reconstructing a static scene of a multi-frame point cloud provided by an embodiment of the present disclosure, such as Figure 1As shown, the specific process of the multi-frame point cloud static scene reconstruction method includes:

[0035] S110, acquiring time-series point cloud data when the vehicle is in an automatic driving state, the time-series point cloud data including: multiple frames of continuous laser point cloud data.

[0036] Among them, multiple frames of continuous laser point cloud data are laser point cloud data collected at different times, that is, different laser point cloud data correspond to different collection times.

[0037] In some embodiments, before performing frame-by-frame detection on the time-series point cloud data, the method of this embodiment further includes:

[0038] Based on the motion pose and acquisition timestamp of each laser point cloud data at the acquisition time, the estimated pose of each laser point cloud data at the acquisition timestamp is obtained; based on the estimated pose of each laser point cloud data at the acquisition timestamp, each laser point cloud data is dedistorted to update the time series point cloud data.

[0039] Among them, when dedistorting the laser point cloud data, the uniform speed model is used to perform point cloud dedistortion processing. It is assumed that the carrier moves at a constant speed and constant angular velocity. The posture at the start and end of the single-frame point cloud acquisition time is used as the input of the uniform speed model. Combined with the timestamp of each point, the posture of the carrier at the timestamp can be calculated, and then the coordinate transformation of the point is performed to complete the point cloud dedistortion.

[0040] Based on the motion pose and acquisition timestamp of each laser point cloud data at the acquisition time, an estimated pose of each laser point cloud data at the acquisition timestamp is obtained, which may include: inputting the motion pose and acquisition timestamp of the laser point cloud data at the acquisition time into a uniform speed model, and determining the estimated pose of the laser point cloud data at the acquisition timestamp based on the output of the uniform speed model.

[0041] Based on the estimated pose of each laser point cloud data at the acquisition timestamp, each laser point cloud data is dedistorted, which may include: converting the estimated pose of the laser point cloud data at the acquisition timestamp into a world coordinate system, and performing an error comparison between the converted estimated pose and the actual pose of the laser point cloud data at the acquisition timestamp. If the error is greater than a preset difference, the laser point cloud data is considered to be distorted data and is removed; if the error is less than or equal to the preset difference, the laser point cloud data is considered not to be distorted data and is retained.

[0042] Therefore, by dedistorting the laser point cloud data before performing frame-by-frame detection on the time-series point cloud data, the accuracy of the laser point cloud data for frame-by-frame detection can be effectively guaranteed, avoiding the impact on the detection of single-frame scene targets.

[0043] S120, performing frame-by-frame detection on the time-series point cloud data to determine the corresponding single-frame scene target from each frame of laser point cloud data.

[0044] Among them, when performing frame-by-frame detection on temporal point cloud data, a multi-stage 3D target detection framework based on the Cascade Attention Network (Cascade Attention Network, CasA) can be implemented. CasA consists of a Region Proposal Network (RPN) and a Cascaded Refinement Networks (CRN). CRN contains a new cascade attention module that uses multiple sub-networks and attention modules to aggregate object features from different stages and gradually refine region generation. This detection method can detect targets such as vehicles, pedestrians, and auto users from a single frame of point cloud.

[0045] S130 , tracking each single-frame scene target to obtain corresponding trajectory posture data.

[0046] Among them, the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target.

[0047] The tracker for target tracking is a data association tracker guided by prediction confidence. It uses a constant acceleration (CA) motion model to estimate future positions and outputs prediction confidence for data association. In addition, it uses the geometry, appearance, and motion features of point cloud targets to achieve faster and more accurate data association. Target tracking can determine whether the detection targets of multiple frame point clouds are the same target, as well as the motion status of each detection target.

[0048] In some embodiments, target tracking is performed on each single-frame scene target to obtain corresponding trajectory posture data, including:

[0049] Based on the preset motion model, motion modeling is performed on each single-frame scene target to obtain the motion posture of each single-frame scene target corresponding to different moments; two single-frame scene targets with a preset single-frame relationship are obtained; data association is performed on the two single-frame scene targets with a preset single-frame relationship to obtain the motion trajectory corresponding to the same single-frame scene target.

[0050] Among them, the preset motion model is the CA motion model, which can perform motion modeling on the detection target and estimate the position and posture of the detection target at each moment. For example, the constant acceleration model assumes that the vehicle's motion speed is variable, but the acceleration is constant. Compared with the uniform motion model, it can estimate the vehicle's posture more accurately. The input of the CA motion model is the single-frame scene target and single-frame motion posture of the vehicle, and the output is the moment-by-moment motion posture of the single-frame scene target at each frame moment.

[0051] The preset single-frame relationship is used to describe that two single-frame scene objects in adjacent frames are the same scene object.

[0052] For example, the single-frame scene target A1 and the single-frame scene target A2 are adjacent frames. If the single-frame scene target A1 is the previous frame target of the single-frame scene target A2, and the single-frame scene target A1 and the single-frame scene target A2 are the same target (such as the same pedestrian), then the single-frame scene target A1 and the single-frame scene target A2 have a preset single-frame relationship.

[0053] The purpose of data association is to determine whether a target in the previous frame point cloud and a target in the next frame point cloud are the same target. If so, the data of the two are associated and marked. The trajectory and posture of the target in the time series can be obtained based on the association results of multiple frames of data.

[0054] Data association is performed on two single-frame scene targets with a preset single-frame relationship to obtain the motion trajectory corresponding to the same single-frame scene target, which may include: data association identification of the motion postures of the two single-frame scene targets with a preset single-frame relationship corresponding to different moments (such as sorting them according to time sequence), so as to effectively obtain the motion trajectory and motion posture of the single-frame scene target in the time series.

[0055] In some embodiments, obtaining two single-frame scene objects having a preset single-frame relationship includes:

[0056] Acquire two single-frame scene targets in adjacent frames; determine whether the two single-frame scene targets in adjacent frames are the same scene target based on the appearance attribute data of the two single-frame scene targets in adjacent frames, so as to obtain two single-frame scene targets with a preset single-frame relationship.

[0057] The appearance attribute data may include but are not limited to: geometric shape, appearance characteristics (such as color, size, etc.).

[0058] Based on the appearance attribute data of two single-frame scene targets in adjacent frames, determining whether the two single-frame scene targets in adjacent frames are the same scene target may include: based on the appearance attribute data of the previous frame target of the two single-frame scene targets in adjacent frames, calculating the appearance attribute data of the next frame target corresponding to the previous frame target according to the moving speed, if the similarity between the appearance attribute data of the next frame target corresponding to the previous frame target calculated according to the moving speed and the appearance attribute data of the latter frame target of the two single-frame scene targets in adjacent frames meets a preset threshold, then determining that the two single-frame scene targets in adjacent frames are the same scene target; if the similarity between the appearance attribute data of the next frame target corresponding to the previous frame target calculated according to the moving speed and the appearance attribute data of the latter frame target of the two single-frame scene targets in adjacent frames does not meet the preset threshold, then determining that the two single-frame scene targets in adjacent frames are not the same scene target. Thus, based on the appearance attribute data between the front and back adjacent frames, the same scene target is effectively identified.

[0059] S140 , performing pose estimation on each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target.

[0060] When estimating the pose of each single-frame scene target, a rough estimate is first made based on the point cloud matching method between frames, and then optimized through the point cloud submap matching method to obtain an accurate pose estimation result.

[0061] In some embodiments, performing pose estimation on each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target includes:

[0062] From the laser point cloud data corresponding to the previous frame scene target associated with each single-frame scene target, the corresponding target point cloud data is eliminated to obtain the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target; the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target and the laser point cloud data corresponding to the single-frame scene target are respectively matched between frames to obtain the initial estimated pose corresponding to each single-frame scene target; the laser point cloud data corresponding to each single-frame scene target are respectively matched with the preset point cloud submap to obtain the current estimated pose corresponding to each single-frame scene target; based on the initial estimated pose corresponding to each single-frame scene target and the current estimated pose corresponding to each single-frame scene target, the single-frame estimated pose corresponding to each single-frame scene target is determined.

[0063] Among them, the target point cloud is eliminated from the laser point cloud data of the scene target in the previous frame, and only the point cloud corresponding to the background environment is retained for point cloud registration with the laser point cloud data of the scene target in the current frame. In this way, the influence of dynamic obstacles on the point cloud registration accuracy can be eliminated, so that the initial estimated pose obtained by registration is not affected by the dynamic obstacle point cloud.

[0064] The preset point cloud sub-map is a local point cloud map obtained by aggregating laser point cloud data corresponding to a single-frame scene target at different times.

[0065] The data coordinate system corresponding to the current estimated pose of each single-frame scene target is the same as the data coordinate system corresponding to the preset point cloud submap. That is, before matching with the preset point cloud submap, it is necessary to determine the data coordinate system of the current estimated pose corresponding to the single-frame scene target. If it is not the world coordinate system, it needs to be converted to the world coordinate system to ensure the validity of the point cloud matching.

[0066] Based on the initial estimated pose corresponding to each single-frame scene target and the current estimated pose corresponding to each single-frame scene target, determining the single-frame estimated pose corresponding to each single-frame scene target may include: adjusting the pose of the initial estimated pose corresponding to each single-frame scene target based on the current estimated pose corresponding to each single-frame scene target (such as fusing the current estimated pose corresponding to the single-frame scene target with the initial estimated pose according to different weight ratios, and the weight ratio can be adaptively adjusted according to the scene), and obtaining the single-frame estimated pose corresponding to each single-frame scene target. Thus, a more accurate position estimation result is obtained.

[0067] S150, based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target, separate the point clouds of the single-frame scene targets belonging to the dynamic targets to obtain multiple single-frame static point cloud data.

[0068] Among them, each single-frame scene target may be a dynamic target or a static target, and it is necessary to separate the dynamic and static targets to effectively obtain single-frame static point cloud data.

[0069] In some embodiments, based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target, the single-frame scene target belonging to the dynamic target is separated into point clouds to obtain multiple single-frame static point cloud data, including:

[0070] Based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target, the target type of each single-frame scene target is determined, and the target types include: dynamic targets and static targets; the laser point cloud data corresponding to the single-frame scene targets whose target type is dynamic targets are point cloud separated to obtain multiple single-frame static point cloud data.

[0071] Among them, since the single-frame estimated pose corresponding to the single-frame scene target corresponds to the world coordinate system, the trajectory pose data corresponding to the single-frame scene target can be converted to the world coordinate system in advance to facilitate subsequent operations.

[0072] Based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated pose corresponding to each single-frame scene target, determining the target type of each single-frame scene target may include: if each single-frame estimated pose corresponding to a single-frame scene target is the same as the trajectory estimated pose corresponding to the trajectory posture data / the error is negligible (such as the error is lower than a preset threshold), then determining that the single-frame scene target is at the same position coordinate at all times; if each single-frame estimated pose corresponding to a single-frame scene target is not completely the same as the trajectory estimated pose corresponding to the trajectory posture data / the error is negligible (such as the error is lower than a preset threshold), then determining that the coordinate position of the single-frame scene target changes at a specific time.

[0073] Therefore, the target type of each single-frame scene target is identified through the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target, and the laser point cloud data corresponding to the single-frame scene target whose target type is a dynamic target is performed to effectively separate the dynamic point cloud and realize the separation of dynamic and static point clouds.

[0074] In some embodiments, determining the target type of each single-frame scene target based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target includes:

[0075] If, based on the trajectory posture data corresponding to the single-frame scene target and the single-frame estimated posture corresponding to the single-frame scene target, it is determined that the single-frame scene target is at the same position coordinate at all times, then the target type of the single-frame scene target is determined to be a static target; if, based on the trajectory posture data corresponding to the single-frame scene target and the single-frame estimated posture corresponding to the single-frame scene target, it is determined that the coordinate position of the single-frame scene target changes at a specific time, then the target type of the single-frame scene target is determined to be a dynamic target.

[0076] The specific moment may be one or more moments in the point cloud data collection moments, which is used to indicate that the single-frame scene target is not always in a static state, thereby effectively determining the target type of the single-frame scene target.

[0077] S160, stitching multiple single-frame static point cloud data based on the single-frame estimated pose corresponding to each single-frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0078] Among them, the single-frame estimated pose corresponding to the single-frame scene target includes the estimated pose of the dynamic target and the estimated pose of the static target. The estimated pose of the static target in each frame can be used to splice each single-frame static point cloud data in sequence according to the sequence of the single-frame estimated poses to complete the reconstruction of the static point cloud scene that does not contain dynamic obstacles.

[0079] In this embodiment, the time-series point cloud data of the vehicle in the automatic driving state is obtained, and the time-series point cloud data includes: multiple frames of continuous laser point cloud data; the time-series point cloud data is detected frame by frame to determine the corresponding single-frame scene target from each frame of laser point cloud data; each single-frame scene target is tracked to obtain the corresponding trajectory posture data, and the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target; each single-frame scene target is estimated to obtain the single-frame estimated posture corresponding to each single-frame scene target; based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target, the single-frame scene target belonging to the dynamic target is separated from the point cloud to obtain multiple single-frame static point cloud data; based on the single-frame estimated posture corresponding to each single-frame scene target, multiple single-frame static point cloud data are spliced ​​to reconstruct a static point cloud scene that does not contain dynamic obstacles. In this way, the single-frame dynamic and static point clouds are accurately separated through the time-series target tracking information, and the accuracy of the laser point cloud posture estimation result is improved by using target detection, and accurate static three-dimensional scene reconstruction is effectively realized.

[0080] Figure 2 A structural schematic diagram of a multi-frame point cloud static scene reconstruction device provided in this embodiment, the multi-frame point cloud static scene reconstruction device may include: an acquisition module 210, a detection module 220, a tracking module 230, an estimation module 240, a determination module 250 and a stitching module 260.

[0081] The acquisition module 210 is used to acquire the time-series point cloud data when the vehicle is in the automatic driving state, and the time-series point cloud data includes: multiple frames of continuous laser point cloud data.

[0082] The detection module 220 is used to perform frame-by-frame detection on the time-series point cloud data to determine the corresponding single-frame scene target from each frame of laser point cloud data.

[0083] The tracking module 230 is used to track each single-frame scene target and obtain corresponding trajectory posture data, where the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target.

[0084] The estimation module 240 is used to estimate the pose of each single-frame scene target to obtain the single-frame estimated pose corresponding to each single-frame scene target.

[0085] The determination module 250 is used to separate the point cloud of the single-frame scene target belonging to the dynamic target based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target to obtain multiple single-frame static point cloud data.

[0086] The stitching module 260 is used to stitch multiple single-frame static point cloud data based on the single-frame estimated pose corresponding to each single-frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles.

[0087] In this embodiment, optionally, the tracking module 230 is specifically used to:

[0088] Based on the preset motion model, motion modeling is performed on each single-frame scene target to obtain the motion posture of each single-frame scene target corresponding to different moments; two single-frame scene targets with a preset single-frame relationship are obtained, and the preset single-frame relationship is used to describe that two single-frame scene targets in adjacent frames are the same scene target; data association is performed on the two single-frame scene targets with the preset single-frame relationship to obtain the motion trajectory corresponding to the same single-frame scene target.

[0089] In this embodiment, optionally, the tracking module 230 is specifically used to:

[0090] Acquire two single-frame scene targets in adjacent frames; determine whether the two single-frame scene targets in adjacent frames are the same scene target based on the appearance attribute data of the two single-frame scene targets in adjacent frames, so as to obtain two single-frame scene targets with a preset single-frame relationship.

[0091] In this embodiment, optionally, the estimation module 240 is specifically configured to:

[0092] From the laser point cloud data corresponding to the previous frame scene target associated with each single-frame scene target, the corresponding target point cloud data is eliminated to obtain the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target; the scene point cloud data corresponding to the previous frame scene target associated with each single-frame scene target and the laser point cloud data corresponding to the single-frame scene target are respectively matched between frames to obtain the initial estimated pose corresponding to each single-frame scene target; the laser point cloud data corresponding to each single-frame scene target are respectively matched with the preset point cloud submap to obtain the current estimated pose corresponding to each single-frame scene target, and the data coordinate system corresponding to the current estimated pose corresponding to each single-frame scene target is the same as the data coordinate system corresponding to the preset point cloud submap; based on the initial estimated pose corresponding to each single-frame scene target and the current estimated pose corresponding to each single-frame scene target, the single-frame estimated pose corresponding to each single-frame scene target is determined.

[0093] In this embodiment, optionally, the determination module 250 is specifically configured to:

[0094] Based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target, the target type of each single-frame scene target is determined, and the target types include: dynamic targets and static targets; the laser point cloud data corresponding to the single-frame scene targets whose target type is dynamic targets are point cloud separated to obtain multiple single-frame static point cloud data.

[0095] In this embodiment, optionally, the determination module 250 is specifically configured to:

[0096] If, based on the trajectory posture data corresponding to the single-frame scene target and the single-frame estimated posture corresponding to the single-frame scene target, it is determined that the single-frame scene target is at the same position coordinate at all times, then the target type of the single-frame scene target is determined to be a static target; if, based on the trajectory posture data corresponding to the single-frame scene target and the single-frame estimated posture corresponding to the single-frame scene target, it is determined that the coordinate position of the single-frame scene target changes at a specific time, then the target type of the single-frame scene target is determined to be a dynamic target.

[0097] In this embodiment, optionally, it also includes: an update module.

[0098] The updating module is used to obtain the estimated pose of each laser point cloud data at the acquisition timestamp based on the motion pose of each laser point cloud data at the acquisition time and the acquisition timestamp; based on the estimated pose of each laser point cloud data at the acquisition timestamp, each laser point cloud data is dedistorted to update the time series point cloud data.

[0099] The multi-frame point cloud static scene reconstruction device provided by the present disclosure can execute the above method embodiments. Its specific implementation principles and technical effects can be found in the above method embodiments, and the present disclosure will not repeat them here.

[0100] The present application also provides a computer device. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0101] The computer device includes a memory 310 and a processor 320 that are connected to each other through a system bus. It should be noted that the figure only shows a computer device with a memory 310 and a processor 320, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0102] Computer devices can be computing devices such as desktop computers, notebooks, PDAs, and cloud servers. Computer devices can interact with users through keyboards, mice, remote controls, touch pads, or voice control devices.

[0103] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes a non-volatile memory or a volatile memory, for example, a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc., and the RAM may include a static RAM or a dynamic RAM. In some embodiments, the memory 310 may be an internal storage unit of a computer device, for example, a hard disk or a memory of the computer device. In other embodiments, the memory 310 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, or a flash card (FlashCard) equipped on the computer device. Of course, the memory 310 may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory 310 is generally used to store the operating system and various application software installed on the computer device, such as the program code of the above method. In addition, the memory 310 may also be used to temporarily store various data that have been output or are to be output.

[0104] The processor 320 is generally used to perform the overall operation of the computer device. In this embodiment, the memory 310 is used to store program codes or instructions, the program code includes computer operation instructions, and the processor 320 is used to execute the program codes or instructions stored in the memory 310 or process data, such as running the program code of the above method.

[0105] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus system can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0106] Another embodiment of the present application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads a computer-readable program code stored in the computer-readable medium, so that the processor can execute the functional actions specified in each step or a combination of steps in the above method; and generate a device for implementing the functional actions specified in each block or a combination of blocks in the block diagram.

[0107] Computer-readable media include but are not limited to electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any appropriate combination of the foregoing, the memory is used to store program codes or instructions, the program codes include computer operating instructions, and the processor is used to execute the program codes or instructions of the above methods stored in the memory.

[0108] For the definitions of memory and processor, please refer to the description of the aforementioned computer device embodiment and will not be repeated here.

[0109] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] Each functional unit or module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0112] In the claims, any reference symbols placed between brackets shall not be construed as limiting the claims. The word "comprising" described in the present application does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented with the aid of hardware comprising several different elements and with the aid of a suitably programmed computer. In a unit claim that lists a number of devices, several units of these devices may be embodied by the same hardware item. The use of first, second, and third, etc. does not indicate any order, and these words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

[0113] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for reconstructing a static scene of a multi-frame point cloud, characterized in that: include: Acquire time-series point cloud data when the vehicle is in an automatic driving state, wherein the time-series point cloud data includes: multiple frames of continuous laser point cloud data; Performing frame-by-frame detection on the time-series point cloud data to determine a corresponding single-frame scene target from each frame of the laser point cloud data; Tracking each of the single-frame scene targets to obtain corresponding trajectory posture data, wherein the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target; Performing pose estimation on each of the single-frame scene targets to obtain a single-frame estimated pose corresponding to each of the single-frame scene targets; Based on the trajectory posture data corresponding to each of the single-frame scene targets and the single-frame estimated posture corresponding to each of the single-frame scene targets, point cloud separation is performed on the single-frame scene targets belonging to the dynamic targets to obtain a plurality of single-frame static point cloud data; Based on the single-frame estimated pose corresponding to each single-frame scene target, multiple single-frame static point cloud data are spliced ​​to reconstruct a static point cloud scene that does not contain dynamic obstacles.

2. The method according to claim 1, characterized in that The target tracking of each single-frame scene target to obtain corresponding trajectory posture data includes: Performing motion modeling on each of the single-frame scene targets based on a preset motion model to obtain motion postures of each of the single-frame scene targets corresponding to different moments; Acquire two single-frame scene objects having a preset single-frame relationship, wherein the preset single-frame relationship is used to describe that the two single-frame scene objects in adjacent frames are the same scene object; Data association is performed on two single-frame scene objects having a preset single-frame relationship to obtain a motion trajectory corresponding to the same single-frame scene object.

3. The method according to claim 2, characterized in that The acquiring of the two single-frame scene objects having a preset single-frame relationship comprises: Acquire two single-frame scene objects in adjacent frames; Based on the appearance attribute data of the two single-frame scene objects in adjacent frames, it is determined whether the two single-frame scene objects in adjacent frames are the same scene object, so as to obtain the two single-frame scene objects having a preset single-frame relationship.

4. The method according to claim 1, characterized in that: The step of estimating the pose of each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target includes: From the laser point cloud data corresponding to the previous frame scene target associated with each of the single frame scene targets, the corresponding target point cloud data is eliminated to obtain the scene point cloud data corresponding to the previous frame scene target associated with each of the single frame scene targets; Respectively performing inter-frame point cloud matching on the scene point cloud data corresponding to the previous frame scene target associated with each of the single-frame scene targets and the laser point cloud data corresponding to the single-frame scene target to obtain an initial estimated pose corresponding to each of the single-frame scene targets; Performing point cloud matching on the laser point cloud data corresponding to each of the single-frame scene targets and the preset point cloud submap, respectively, to obtain the current estimated pose corresponding to each of the single-frame scene targets, wherein the data coordinate system corresponding to the current estimated pose corresponding to each of the single-frame scene targets is the same as the data coordinate system corresponding to the preset point cloud submap; Based on the initial estimated pose corresponding to each of the single-frame scene targets and the current estimated pose corresponding to each of the single-frame scene targets, the single-frame estimated pose corresponding to each of the single-frame scene targets is determined.

5. The method according to claim 1, characterized in that The step of performing point cloud separation on the single-frame scene targets belonging to dynamic targets based on the trajectory posture data corresponding to each of the single-frame scene targets and the single-frame estimated posture corresponding to each of the single-frame scene targets to obtain a plurality of single-frame static point cloud data includes: Based on the trajectory posture data corresponding to each of the single-frame scene targets and the single-frame estimated posture corresponding to each of the single-frame scene targets, determining the target type of each of the single-frame scene targets, wherein the target type includes: a dynamic target and a static target; The laser point cloud data corresponding to the single-frame scene target of which the target type is a dynamic target is subjected to point cloud separation to obtain a plurality of single-frame static point cloud data.

6. The method according to claim 5, characterized in that The determining the target type of each single-frame scene target based on the trajectory posture data corresponding to each single-frame scene target and the single-frame estimated posture corresponding to each single-frame scene target includes: If it is determined that the single-frame scene target is at the same position coordinate at all times based on the trajectory posture data corresponding to the single-frame scene target and the single-frame estimated posture corresponding to the single-frame scene target, then the target type of the single-frame scene target is determined to be a static target; If it is determined that the coordinate position of the single-frame scene target changes at a specific moment based on the trajectory posture data corresponding to the single-frame scene target and the single-frame estimated posture corresponding to the single-frame scene target, then it is determined that the target type of the single-frame scene target is a dynamic target.

7. The method according to claim 1, characterized in that Before the frame-by-frame detection of the time series point cloud data, the method further includes: Based on the motion pose and acquisition timestamp of each laser point cloud data at the acquisition time, obtaining the estimated pose of each laser point cloud data at the acquisition timestamp; Based on the estimated pose of each laser point cloud data at the acquisition timestamp, each laser point cloud data is subjected to a dedistortion process to update the time series point cloud data.

8. A device for reconstructing a static scene of a multi-frame point cloud, characterized in that: include: An acquisition module is used to acquire time-series point cloud data when the vehicle is in an automatic driving state, wherein the time-series point cloud data includes: multiple frames of continuous laser point cloud data; A detection module, used to detect the time-series point cloud data frame by frame, so as to determine the corresponding single-frame scene target from each frame of the laser point cloud data; A tracking module, used to track each of the single-frame scene targets to obtain corresponding trajectory posture data, wherein the trajectory posture data is used to describe the motion trajectory and motion posture corresponding to the single-frame scene target; An estimation module, used to estimate the pose of each single-frame scene target to obtain a single-frame estimated pose corresponding to each single-frame scene target; A determination module, configured to separate the point cloud of the single-frame scene target belonging to the dynamic target based on the trajectory posture data corresponding to each of the single-frame scene targets and the single-frame estimated posture corresponding to each of the single-frame scene targets, so as to obtain a plurality of single-frame static point cloud data; The splicing module is used to splice multiple single-frame static point cloud data based on the single-frame estimated posture corresponding to each single-frame scene target to reconstruct a static point cloud scene that does not contain dynamic obstacles.

9. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, a method for reconstructing a static scene of a multi-frame point cloud as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for reconstructing a multi-frame point cloud static scene as described in any one of claims 1 to 7 is implemented.

Images