Environment perception ground truth data generation method and apparatus, device, and medium

Abstract

The application discloses an environment perception ground truth data generation method, device, equipment and medium, and the method comprises the steps of acquiring multi-frame environment three-dimensional point cloud data collected by an aircraft and corresponding aircraft pose data; projecting the multi-frame environment three-dimensional point cloud data to a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, and performing semantic labeling on the target global point cloud map to obtain a global labeled map; determining target point cloud data corresponding to a target frame and target pose data corresponding to the target frame from the multi-frame environment three-dimensional point cloud data; generating a static background grid based on the target pose data and the global labeled map; generating a dynamic target grid based on the target point cloud data and the target global point cloud map; and generating environment perception ground truth data corresponding to the target frame based on the static background grid and the dynamic target grid. The method realizes dynamic and static joint environment perception ground truth data generation and guarantees the spatiotemporal consistency of the ground truth data.

Description

Technical Field

[0001] This invention relates to the field of aircraft data processing technology, and in particular to a method, apparatus, device, and medium for generating environmental perception truth data. Background Technology

[0002] With the rapid development of autonomous driving and low-altitude intelligent driving technologies, 3D environmental perception capabilities have gradually become a focus of current technological attention. 4D Occupancy Grid Maps (4D OCCs), due to their ability to effectively represent the precise geometric structure of static environments and the temporal motion information of dynamic obstacles, have become the next-generation perception paradigm and the foundation for autonomous driving decision-making and planning. To train high-performance 4D OCC end-to-end perception models, the industry requires massive amounts of ground-value environmental perception data with high-precision semantics and occupancy states.

[0003] However, existing environmental perception ground truth data generation schemes have several shortcomings: First, the mapping accuracy is low. Multi-frame point cloud stitching based on SLAM algorithms is prone to pose accumulation drift, leading to ghosting and layering in the point cloud map, resulting in poor global consistency and failing to meet the high-precision requirements of 4D OCC ground truth. Second, the annotation cost is high. Existing schemes mostly adopt frame-by-frame or local annotation modes, resulting in repeated annotation of the same static background in a large number of keyframes. This is costly and makes it difficult to guarantee semantic consistency between frames, thus failing to support the large-scale production of 4D OCC ground truth. Therefore, there is an urgent need for a method that can achieve high-precision, low-cost, and highly consistent environmental perception ground truth data generation. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for generating environmental perception truth data, in order to solve the problems of low mapping accuracy, high annotation cost, and poor inter-frame consistency in related technologies.

[0005] In a first aspect, the present invention provides a method for generating environmental perception truth data, comprising: Acquire multi-frame environmental 3D point cloud data collected by the aircraft, as well as the corresponding aircraft pose data; Based on the aircraft pose data, multi-frame environmental 3D point cloud data is projected onto a preset global coordinate system to obtain a target global point cloud map, and semantic annotation is performed on the target global point cloud map to obtain a globally annotated map. The target point cloud data corresponding to the target frame and the target pose data corresponding to the target frame are determined from the multi-frame environmental 3D point cloud data. The target frame is any frame contained in the multi-frame environmental 3D point cloud data. A static background mesh is generated based on the target pose data and the globally labeled map; Based on the target point cloud data and the target global point cloud map, a dynamic target mesh is generated; Based on the static background grid and the dynamic target grid, the environmental perception ground truth data corresponding to the target frame is generated.

[0006] In some embodiments, acquiring multi-frame environmental 3D point cloud data collected by the aircraft, and the corresponding aircraft pose data, includes: Acquire lidar data, satellite navigation data, and inertial measurement data collected by the aircraft in the target area; The lidar data is analyzed to obtain the initial environmental 3D point cloud data, including the first timestamp; By jointly solving the satellite navigation data and inertial measurement data, initial pose data including the second timestamp is obtained; Based on a preset accuracy threshold, the initial pose data is verified for accuracy, and the initial pose data that passes the accuracy verification is determined as the aircraft pose data. Based on the first and second timestamps, the initial environmental 3D point cloud data is time-aligned to obtain multi-frame environmental 3D point cloud data.

[0007] In some embodiments, based on aircraft pose data, multiple frames of environmental 3D point cloud data are projected onto a preset global coordinate system to obtain a target global point cloud map, including: Based on the aircraft pose data and the preset global coordinate system, coordinate transformation is performed on the multi-frame environmental 3D point cloud data to obtain multi-frame global environmental point cloud data. The initial global point cloud map is obtained by overlaying multiple frames of global environmental point cloud data. Extract planar features from the initial global point cloud map, and perform ghosting detection on the initial global point cloud map based on the planar features to obtain the ghosting detection results; Based on the ghosting detection results, the aircraft pose data is corrected to obtain the corrected pose data; Based on the corrected pose data, ghosting correction is performed on the initial global point cloud map to obtain the target global point cloud map.

[0008] In some embodiments, a static background mesh is generated based on the target pose data and a globally labeled map, including: Based on the target pose data and the preset perception range, the corresponding 3D point cloud data is extracted from the globally labeled map to obtain the background point cloud data; The background point cloud data is processed into a grid to obtain multiple spatial grid cells; Based on the distribution of background point cloud data, the grid occupancy status of multiple spatial grid units is determined; Map the corresponding semantic labels in the globally labeled map to multiple spatial grid cells to determine the grid semantic labels; A static background grid is generated based on the grid occupancy status and grid semantic labels.

[0009] In some embodiments, determining the grid occupancy status of multiple spatial grid cells based on the distribution of background point cloud data includes: Obtain the grid resolution of multiple spatial grid cells, and determine the grid occupancy threshold based on the grid resolution; The number of background point clouds contained in multiple spatial grid cells is counted to obtain the grid point cloud count results; The grid occupancy status of spatial grid cells whose grid point cloud count results are greater than or equal to the grid occupancy threshold is defined as the occupancy status. The grid occupancy status of spatial grid cells whose grid point cloud count is less than the grid occupancy threshold is determined as non-occupancy status.

[0010] In some embodiments, a dynamic target mesh is generated based on target point cloud data and a target global point cloud map, including: Based on the target pose data, the target point cloud data is projected onto a preset global coordinate system to obtain the projected target point cloud data; Calculate the distance between each point in the projected target point cloud data and the nearest neighbor point in the target global point cloud map to obtain point cloud distance data; Based on a preset distance threshold and point cloud distance data, dynamic target point cloud data is determined from the projected target point cloud data; The dynamic target point cloud data is processed into a grid to obtain a dynamic target grid.

[0011] In some embodiments, environmental awareness ground truth data corresponding to the target frame is generated based on a static background mesh and a dynamic target mesh, including: The dynamic target mesh and the static background mesh are fused to obtain a joint mesh; Based on the target pose data, determine the optical center position of the aircraft in the joint grid; Starting from the optical center, multiple probe rays are emitted into the preset field of view; Based on the intersection relationship between the probe ray and the joint grid, the grid occlusion state of the joint grid is determined; Based on the grid occlusion state, generate ground truth environmental perception data corresponding to the target frame.

[0012] In a second aspect, the present invention provides an environmental perception truth data generation device, comprising: The data acquisition module is used to acquire multi-frame environmental 3D point cloud data collected by the aircraft, as well as the corresponding aircraft pose data; The map annotation module is used to project multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, and to perform semantic annotation on the target global point cloud map to obtain a globally annotated map. The target determination module is used to determine the target point cloud data corresponding to the target frame and the target pose data corresponding to the target frame from the multi-frame environmental 3D point cloud data. The target frame is any frame contained in the multi-frame environmental 3D point cloud data. The first mesh generation module is used to generate a static background mesh based on the target pose data and the globally labeled map; The second grid generation module is used to generate a dynamic target grid based on the target point cloud data and the target global point cloud map; The truth generation module is used to generate environmental perception truth data corresponding to the target frame based on the static background grid and the dynamic target grid.

[0013] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the environmental awareness truth data generation method described in the first aspect above.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the environmental perception truth data generation method described in the first aspect above.

[0015] In summary, the environmental perception ground truth data generation method, apparatus, device, and medium disclosed herein include: acquiring multi-frame environmental 3D point cloud data collected by an aircraft, and corresponding aircraft pose data; projecting the multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, and semantically annotating the target global point cloud map to obtain a globally annotated map; determining the target point cloud data corresponding to a target frame and the target pose data corresponding to the target frame from the multi-frame environmental 3D point cloud data, wherein the target frame is any frame contained in the multi-frame environmental 3D point cloud data; generating a static background mesh based on the target pose data and the globally annotated map; generating a dynamic target mesh based on the target point cloud data and the target global point cloud map; and generating environmental perception ground truth data corresponding to the target frame based on the static background mesh and the dynamic target mesh. This method constructs a map by projecting multi-frame environmental 3D point cloud data onto a preset global coordinate system based on aircraft pose data, and performs one-time semantic annotation on the target global point cloud map to obtain a globally annotated map. This eliminates mapping drift error and reduces the cost of repeated annotation across multiple frames to a single fixed cost. By generating a static background mesh from the globally annotated map and combining the target point cloud data with the target global point cloud map to generate a dynamic target mesh, the method achieves the generation of dynamic and static joint environmental perception ground truth data, ensuring the spatiotemporal consistency of the ground truth data. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of an environmental perception truth data generation method according to an embodiment of the present invention; Figure 2 This is another flowchart of the method for generating environmental perception truth data in one embodiment of the present invention; Figure 3 This is another flowchart of the method for generating environmental perception truth data in one embodiment of the present invention; Figure 4 This is another flowchart of the method for generating environmental perception truth data in one embodiment of the present invention; Figure 5 This is another flowchart of the method for generating environmental perception truth data in one embodiment of the present invention; Figure 6 This is another flowchart of the method for generating environmental perception truth data in one embodiment of the present invention; Figure 7This is another flowchart of the method for generating environmental perception truth data in one embodiment of the present invention; Figure 8 This is a schematic block diagram of an environmental perception truth data generation device according to an embodiment of the present invention; Figure 9 This is a schematic block diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0018] As an example, such as Figure 1 As shown, a method for generating environmental perception truth data is provided, including: S101, acquires multi-frame environmental 3D point cloud data collected by the aircraft, as well as the corresponding aircraft pose data; S102, based on the aircraft pose data, project multi-frame environmental 3D point cloud data onto a preset global coordinate system to obtain a target global point cloud map, and perform semantic annotation on the target global point cloud map to obtain a globally annotated map; S103, determine the target point cloud data corresponding to the target frame and the target pose data corresponding to the target frame from the multi-frame environmental 3D point cloud data; S104, Generate a static background mesh based on the target pose data and the globally labeled map; S105, generates a dynamic target mesh based on target point cloud data and target global point cloud map; S106, based on the static background grid and the dynamic target grid, generates the environmental perception ground truth data corresponding to the target frame.

[0019] As an example, in step S101, a sensor system mounted on the aircraft can be used to collect data by performing multi-flight cross-trajectory data in the target area, thereby obtaining multi-frame environmental 3D point cloud data and corresponding aircraft pose data. Among them, multi-frame environmental 3D point cloud data refers to a set of multi-frame 3D point cloud data collected in a time sequence, and aircraft pose data refers to the 6-DOF (6-DoF) pose information of the aircraft corresponding to each frame of point cloud data, including position information and attitude information.

[0020] In one embodiment, the aircraft can be a low-altitude intelligent driving platform such as a low-altitude unmanned aerial vehicle or a manned aircraft, equipped with sensors such as lidar and a high-precision integrated navigation system to collect environmental point cloud data and the aircraft's own position and attitude data. In other words, the aircraft, as a mobile platform carrying sensors, completes the collection of environmental data of the target area by flying along a preset route within the target area.

[0021] In one embodiment, the preset global coordinate system can be a coordinate system with an absolute geographic reference, such as the WGS84 coordinate system or the UTM coordinate system, or it can be a local Cartesian coordinate system with a certain point in the target area as the origin. As long as a unified global spatial reference can be provided, the present invention does not limit it.

[0022] In one embodiment, the environmental three-dimensional point cloud data can be three-dimensional point cloud data collected by lidar, and each point cloud data contains three-dimensional coordinate information and information such as reflection intensity; the aircraft attitude data can contain three-dimensional position coordinates (such as longitude, latitude, and altitude) and three-dimensional attitude angles (such as roll, pitch, and yaw / heading).

[0023] As an example, in step S102, based on the acquired aircraft pose data, the environmental 3D point cloud data of each frame can be transformed from the sensor local coordinate system to the preset global coordinate system, and the transformed environmental 3D point cloud data of each frame can be superimposed to obtain the target global point cloud map; then, the target global point cloud map is fully semantically annotated to obtain a global annotated map with semantic labels.

[0024] In one embodiment, semantic annotation of the target global point cloud map can be performed manually, or it can be automatically pre-annotated using an offline large model and then manually verified, as long as full semantic annotation of the global point cloud map can be achieved. The semantic annotation categories can include common static features in low-altitude scenes such as buildings, power towers, power lines, trees, ground, and water bodies.

[0025] It should be understood that since the global point cloud map is a unique static base map, only a one-time full semantic annotation is needed. This semantic annotation can be directly reused when generating ground truth values ​​for all subsequent temporal keyframes. This drastically reduces the annotation cost of a massive number of keyframes to a single fixed cost. At the same time, because it is only annotated once, it avoids the semantic and boundary jump problems between frames caused by multiple manual annotations, ensuring the absolute consistency of ground truth values.

[0026] As an example, in step S103, any frame can be selected from the multi-frame environmental 3D point cloud data as the target frame, and the target point cloud data and the target pose data corresponding to the target frame can be obtained; wherein, the target frame is any frame contained in the multi-frame environmental 3D point cloud data, that is, it can be any frame in the multi-frame data, and each frame can be used as the target frame when it is selected, and used to generate the environmental perception ground truth data corresponding to the frame.

[0027] In one embodiment, the selection of the target frame can be based on the training requirements of the downstream perception network, such as selecting at fixed time intervals or selecting according to a specific scene. Any frame selected from the multi-frame environmental 3D point cloud data is acceptable, and the present invention does not limit this.

[0028] As an example, in step S104, the current perception range can be determined in the global annotation map based on the target pose data corresponding to the target frame, and the annotation point cloud data within the range can be extracted and processed into a grid to generate a static background grid; wherein, the static background grid refers to three-dimensional grid data containing the static background occupancy state and semantic labels.

[0029] In one embodiment, the preset sensing range can be set according to the input receptive field design specifications of the downstream perceptual neural network, for example, set as a cube-shaped area centered on the aircraft, with a radius of 50 meters in all directions (front, back, left, right) and 20 meters in all directions (up and down). In other words, the preset sensing range is a pre-defined spatial range used to generate single-frame environmental perception ground truth data, and its size and shape can be adjusted according to actual application requirements.

[0030] In other words, based on the aircraft pose data corresponding to the target frame, and using that aircraft pose data as the center, the point cloud of the corresponding region can be automatically cropped from the labeled global static base map according to the preset perception range boundary, and then voxelized, assigning voxel occupancy status and semantic labels. Since the globally labeled map has already completed high-precision semantic annotation, the static background mesh obtained by slicing can directly obtain the corresponding semantic information, thereby ensuring spatial consistency.

[0031] As an example, in step S105, the target point cloud data of the target frame can be spatially differentially compared with the target global point cloud map to extract the dynamic point cloud that does not belong to the static background in the target point cloud data, and the dynamic point cloud can be meshed to obtain the dynamic target mesh; wherein, the dynamic target mesh refers to the three-dimensional mesh data containing the state of dynamic obstacles.

[0032] In one embodiment, the target point cloud data and target pose data are subjected to rigid body transformation by coordinate transformation to obtain projected target point cloud data in the global coordinate system; the distance between the projected target point cloud data and the target global point cloud map is calculated by nearest neighbor search to obtain point cloud distance data; and the point cloud distance data is judged against a preset distance threshold by threshold comparison to realize the extraction of dynamic target point cloud data.

[0033] For example, the preset distance threshold can be set to 0.15 meters. If the Euclidean distance between the point cloud of the current frame (i.e. the target point cloud data) and the corresponding point on the global point cloud map of the target is greater than this threshold, it is determined to be a dynamic object, thereby filtering out the inherent noise of the system and achieving accurate separation of dynamic targets.

[0034] As an example, in step S106, the static background mesh and the dynamic target mesh can be fused together, and the occlusion state of the mesh can be deduced by combining ray tracing technology, thereby generating the environmental perception ground truth data corresponding to the target frame; wherein, the environmental perception ground truth data can be 4D OCC ground truth data, which includes the occupancy state (e.g. occupied, idle, unknown) of each voxel in three-dimensional space and semantic labels.

[0035] In one embodiment, the environmental perception ground truth data (i.e., 4D OCC ground truth) can include three occupancy states: Occupied, Free, and Unknown / Occluded. The Occupied state indicates that the voxel is occupied by an object; the Free state indicates that the voxel is empty and light can pass through; and the Unknown / Occluded state indicates that the voxel is occluded by an object in front, and it is impossible to determine whether it is occupied.

[0036] It should be understood that by introducing ray-casting to simulate the sensor's perspective and jointly inferring voxel occlusion states, ray-casting can accurately simulate real physical occlusion and accurately infer unknown / occluded states. The generated environmental perception ground truth data is temporally continuous and dense, thereby achieving accurate classification of the three occupancy states and improving the physical realism of the ground truth data. In summary, this disclosure proposes a method for generating ground truth environmental perception data, comprising: acquiring multi-frame environmental 3D point cloud data collected by an aircraft, and corresponding aircraft pose data; projecting the multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, and semantically annotating the target global point cloud map to obtain a globally annotated map; determining the target point cloud data corresponding to the target frame and the target pose data corresponding to the target frame from the multi-frame environmental 3D point cloud data, wherein the target frame is any frame contained in the multi-frame environmental 3D point cloud data; generating a static background mesh based on the target pose data and the globally annotated map; generating a dynamic target mesh based on the target point cloud data and the target global point cloud map; and generating ground truth environmental perception data corresponding to the target frame based on the static background mesh and the dynamic target mesh. This method constructs a map by projecting multi-frame environmental 3D point cloud data onto a preset global coordinate system based on aircraft pose data, and performs one-time semantic annotation on the target global point cloud map to obtain a globally annotated map. This eliminates mapping drift error and reduces the cost of repeated annotation across multiple frames to a single fixed cost. By generating a static background mesh from the globally annotated map and combining the target point cloud data with the target global point cloud map to generate a dynamic target mesh, the method achieves the generation of dynamic and static joint environmental perception ground truth data, ensuring the spatiotemporal consistency of the ground truth data.

[0037] As an example, such as Figure 2 As shown, step S101, which involves acquiring multiple frames of 3D point cloud data of the environment collected by the aircraft, and the corresponding aircraft pose data, includes: S201: Acquire lidar data, satellite navigation data, and inertial measurement data collected by the aircraft in the target area; S202, analyze the lidar data to obtain initial environmental 3D point cloud data including the first timestamp; S203, jointly calculate the satellite navigation data and inertial measurement data to obtain the initial pose data including the second timestamp; S204, based on a preset accuracy threshold, performs accuracy verification on the initial pose data, and determines the initial pose data that passes the accuracy verification as the aircraft pose data; S205, based on the first and second timestamps, performs time alignment on the initial environmental 3D point cloud data to obtain multi-frame environmental 3D point cloud data.

[0038] As an example, in step S201, sensors such as lidar, satellite navigation receiver, and inertial measurement unit mounted on the aircraft can be used to collect data by cross-flight along multiple routes in the target area, thereby obtaining lidar data, satellite navigation data, and inertial measurement data. Among them, lidar data is used to generate a three-dimensional point cloud of the environment, and satellite navigation data and inertial measurement data are used to calculate the aircraft's pose (i.e., aircraft pose data).

[0039] In one embodiment, the lidar can be a high-line-count lidar to ensure the density and accuracy of the point cloud; satellite navigation data can include Global Navigation Satellite System (GNSS) data, such as GPS, BeiDou, GLONASS, etc.; inertial measurement data can include acceleration and angular velocity data collected by the inertial measurement unit. In other words, the lidar, satellite navigation, and inertial measurement unit together constitute the sensor combination of the acquisition system, used to acquire environmental information and carrier motion information.

[0040] For example, a data acquisition platform equipped with a high-line-count lidar and an ultra-high-precision positioning and orientation system (POS) performs multi-flight cross-data acquisition in the target area. The POS system typically consists of a high-precision GNSS receiver tightly coupled with a high-level inertial measurement unit (IMU), which can provide the carrier with high-precision three-dimensional position, three-dimensional velocity, and three-dimensional attitude (6-DoF).

[0041] As an example, in step S202, the collected lidar data can be parsed and processed to convert the original lidar measurement data into a three-dimensional point cloud format, and a corresponding first timestamp can be added to each frame of point cloud data to obtain the initial environmental three-dimensional point cloud data; wherein, the first timestamp refers to the acquisition timestamp corresponding to each frame of initial environmental three-dimensional point cloud data.

[0042] In one embodiment, parsing the lidar data may include processing steps such as distance calculation, angle calculation, and coordinate transformation, ultimately generating point cloud data with three-dimensional coordinates and reflection intensity information. Each frame of point cloud data corresponds to a collection time, which is recorded in the form of a first timestamp for subsequent time alignment processing.

[0043] As an example, in step S203, a tightly coupled solution algorithm can be used to jointly solve the satellite navigation data and inertial measurement data to obtain the initial pose data of the aircraft, and add a corresponding second timestamp to each initial pose data; whereby tightly coupled solution refers to a solution method that deeply fuses the raw GNSS observations and the raw IMU observations at the filtering level, which can provide pose results with higher accuracy and higher robustness.

[0044] In one embodiment, satellite navigation data and inertial measurement data can be processed using a post-hoc tightly coupled solution algorithm to obtain the true zero-drift 6-DoF trajectory with an absolute geographic coordinate system, including longitude, latitude, altitude, and pose information such as roll angle, pitch angle, and yaw angle. The second timestamp refers to the solution timestamp corresponding to each pose data point, which corresponds to the first timestamp of the point cloud data.

[0045] It should be understood that tightly coupled solution has higher accuracy and robustness than loosely coupled solution, especially in scenarios where GNSS signals are briefly lost. Tightly coupled systems can rely on the inertial calculation of the IMU to maintain the stability of pose output, thereby ensuring the consistency of global mapping.

[0046] As an example, in step S204, the initial pose data obtained by calculation can be verified based on a preset accuracy threshold, and the pose data that meets the accuracy requirements can be determined as the final aircraft pose data. The preset accuracy threshold is a pre-set pose accuracy standard, and only pose data that meets this standard can be used for subsequent mapping and ground truth generation.

[0047] In one embodiment, the preset accuracy threshold may include a position accuracy threshold and an attitude accuracy threshold.

[0048] As an example, in step S205, the initial environmental 3D point cloud data and the aircraft pose data can be time-aligned based on the first timestamp of the point cloud data and the second timestamp of the pose data, so that each frame of initial environmental 3D point cloud data corresponds to accurate aircraft pose data, thereby obtaining time-aligned multi-frame environmental 3D point cloud data.

[0049] In one embodiment, time alignment may include: interpolating and matching based on timestamps to interpolate pose data to the time corresponding to each frame of point cloud, thereby obtaining accurate pose data corresponding to the initial environmental 3D point cloud data for each frame. In other words, since the sampling frequencies of the LiDAR and POS may be different, time alignment and interpolation are required to match the aircraft pose data at the corresponding time for each frame of initial environmental 3D point cloud.

[0050] For example, if the sampling frequency of the LiDAR is 10Hz and the output frequency of the POS is 100Hz, then the 10Hz pose data corresponding to each frame of the LiDAR point cloud can be obtained by interpolating in the 100Hz pose sequence of the POS through linear interpolation or higher-order interpolation, thereby achieving precise temporal alignment.

[0051] As an example, such as Figure 3 As shown, step S102, which involves projecting multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, includes: S301, based on the aircraft pose data and the preset global coordinate system, performs coordinate transformation on multi-frame environmental 3D point cloud data to obtain multi-frame global environmental point cloud data; S302, overlay multiple frames of global environmental point cloud data to obtain an initial global point cloud map; S303, extract planar features from the initial global point cloud map, and perform ghost detection on the initial global point cloud map based on the planar features to obtain the ghost detection result; S304, Based on the ghosting detection results, the aircraft pose data is corrected to obtain the corrected pose data; S305, based on the corrected pose data, performs ghosting correction on the initial global point cloud map to obtain the target global point cloud map.

[0052] As an example, in step S301, the aircraft pose data corresponding to each frame of environmental 3D point cloud data can be used to transform each frame of environmental 3D point cloud data from the sensor local coordinate system (i.e., the coordinate system when the aircraft carries the sensor to collect data) into a preset global coordinate system, thereby obtaining multiple frames of global environmental point cloud data; where global environmental point cloud data refers to global environmental point cloud data that has been transformed into the preset global coordinate system.

[0053] In one embodiment, the coordinate transformation process can be implemented using a rigid body transformation matrix. Specifically, based on the aircraft's position and attitude, a transformation matrix is ​​constructed from the sensor's local coordinate system to the global coordinate system. Then, each point in each frame of the environmental 3D point cloud data is multiplied by this transformation matrix to obtain the point cloud coordinates in the preset global coordinate system. In other words, through rigid body transformation, the point cloud in each frame's local coordinate system is uniformly mapped to the preset global coordinate system, preparing for subsequent overlay mapping. This method avoids the problem of cumulative drift at its source by directly using absolute pose for projection.

[0054] As an example, in step S302, all multi-frame global environmental point cloud data transformed to a preset global coordinate system can be superimposed, that is, all point cloud data can be merged into the same global coordinate system space to obtain an initial global point cloud map; wherein, the initial global point cloud map refers to the global point cloud map obtained after preliminary superposition.

[0055] As an example, in step S303, planar features in the initial global point cloud map can be extracted, and ghosting detection can be performed on the initial global point cloud map based on these planes to obtain ghosting detection results; where planar features refer to ground features in the point cloud that present a planar shape, such as ground, walls, roofs, etc., and ghosting detection refers to detecting whether there are ghosting, layering, or other phenomena in the point cloud map caused by pose errors.

[0056] In one embodiment, the global consistency of planes in low-altitude scenes can be used for ghosting analysis. Specifically, in low-altitude scenes, the ground, building walls, roofs, etc., all have good planar characteristics; ideally, these planes should be flat and free of ghosting. If there are slight errors in pose, causing ghosting after point cloud superposition, then these planar features will exhibit phenomena such as layering and blurring. By detecting the flatness and layering of the planes, it is possible to determine whether ghosting exists and its degree.

[0057] As an example, in step S304, the aircraft pose data can be fine-tuned based on the ghost detection results to obtain corrected pose data; wherein, the corrected pose data refers to the pose data after fine-tuning, which can eliminate the ghost phenomenon in the point cloud map.

[0058] In one embodiment, error constraints can be constructed, and the pose can be fine-tuned in conjunction with the integrated navigation pose constraints to eliminate ghosting. Specifically, the flatness of the plane is used as the optimization target, and the fine-tuning amount of the pose is used as the optimization variable. The pose is optimized within a limited range to maximize the flatness of the plane, thereby eliminating ghosting.

[0059] It should be understood that this step only makes minor adjustments to the keyframe pose within a limited range, without changing the global absolute coordinate system, thus eliminating ghosting while preserving absolute accuracy.

[0060] As an example, in step S305, the initial global point cloud map can be corrected for ghosting based on the corrected pose data, and a new global point cloud map without ghosting, i.e., the target global point cloud map, can be generated. The target global point cloud map refers to a three-dimensional static background point cloud map with high density and no spatial ghosting after ghosting correction.

[0061] In one embodiment, the ghosting correction process can involve using the corrected aircraft pose data to re-transform and overlay the coordinates of multiple frames of environmental 3D point cloud data, thereby obtaining a ghost-free global point cloud map of the target. Alternatively, the initial global point cloud map can be directly corrected, as long as the ghosting can be eliminated.

[0062] As an example, such as Figure 4 As shown, step S104, which generates a static background mesh based on the target pose data and the globally labeled map, includes: S401, based on the target pose data and the preset perception range, extract the corresponding 3D point cloud data from the globally labeled map to obtain the background point cloud data; S402, performs gridding processing on the background point cloud data to obtain multiple spatial grid cells; S403, based on the distribution of background point cloud data, determine the grid occupancy status of multiple spatial grid cells; S404: Map the corresponding semantic labels in the globally labeled map to multiple spatial grid cells to determine the grid semantic labels; S405 generates a static background grid based on the grid occupancy status and grid semantic labels.

[0063] As an example, in step S401, based on the target pose data of the target frame, the point cloud data of the corresponding area can be automatically cropped from the labeled global map according to the target pose data as the center and the preset perception range boundary, thereby obtaining the background point cloud data; wherein, the background point cloud data refers to the point cloud data corresponding to the global map within the preset perception range.

[0064] In one embodiment, the preset perception range can be set as a cube with L meters (e.g., 50m × 50m) in all directions (front, back, left, and right) and H meters (e.g., 20m) in all directions (up and down) based on the center of the UAV. The setting depends on the input receptive field design specifications of the downstream perceptual neural network. In other words, the perception range is a pre-defined three-dimensional spatial range used to determine the spatial coverage of a single frame of environmental perception ground truth data.

[0065] For example, if the preset perception range is 100m×100m×40m (length×width×height), and the target frame corresponds to the aircraft position at a point in the global coordinate system, then the point cloud data in the globally labeled map within a range of 100m×100m×40m is cropped out with that point as the center and used as the background point cloud data.

[0066] As an example, in step S402, the background point cloud data obtained by cropping can be meshed (i.e., voxelized) to divide the continuous three-dimensional space into multiple discrete spatial mesh units (i.e., voxels), thereby obtaining multiple spatial mesh units; wherein, the spatial mesh unit is the basic unit that constitutes the three-dimensional mesh, and each unit corresponds to a small cube in the three-dimensional space.

[0067] In one embodiment, the voxel size can be set to 0.2m × 0.2m × 0.2m or 0.4m × 0.4m × 0.4m. This setting is based on a key parameter balancing ground truth accuracy and downstream network memory usage. For low-altitude autonomous driving systems to identify obstacles such as thin cables, a 0.2m level is typically preferred. In other words, the grid resolution (i.e., voxel size) is a parameter that can be adjusted according to actual needs; higher resolution results in finer ground truth, but also a larger data volume.

[0068] As an example, in step S403, the grid occupancy status of each spatial grid cell can be determined based on the distribution of background point cloud data in each spatial grid cell; wherein, the grid occupancy status is used to indicate whether the grid cell is occupied by an object.

[0069] In one embodiment, the presence or absence of a grid cell can be determined by counting the number of point clouds contained within each grid cell. If the number of point clouds in a grid cell reaches a certain threshold, the grid cell is considered occupied; otherwise, the grid cell is considered unoccupied (i.e., free).

[0070] As an example, in step S404, the corresponding semantic labels in the global annotation map can be mapped to each spatial grid cell, thereby determining the grid semantic label of each grid cell; where the grid semantic label refers to the semantic category corresponding to each grid cell, such as building, tree, ground, etc.

[0071] In one embodiment, the semantic label mapping method can be as follows: if a grid cell is determined to be occupied, then the semantic label corresponding to the point cloud within that grid is used as the semantic label of that grid. If a grid contains point clouds with multiple semantics, the semantic category with the most occurrences can be taken as the semantic label of that grid, or multiple semantics and their proportions can be retained; this invention does not limit this.

[0072] It should be understood that since the globally labeled map has already completed full semantic annotation, when tiling, it is only necessary to directly map the semantic labels to the grid cells, without the need for re-annotating the semantics. This ensures that the semantic labels of all frames are consistent, fundamentally eliminating the problem of semantic jumps between frames.

[0073] As an example, in step S405, a final static background grid can be generated based on the occupancy status and semantic label of each grid cell; wherein, the static background grid is three-dimensional grid data containing the occupancy status and semantic label of all grid cells, used to represent the static background within the perception range.

[0074] As an example, such as Figure 5 As shown, step S403, which is to determine the grid occupancy status of multiple spatial grid cells based on the distribution of background point cloud data, includes: S501, obtain the grid resolution of multiple spatial grid cells, and determine the grid occupancy threshold based on the grid resolution; S502, count the number of background point clouds contained in multiple spatial grid cells respectively, and obtain the grid point cloud counting result; S503, the grid occupancy status of spatial grid cells whose grid point cloud count results are greater than or equal to the grid occupancy threshold is determined as the occupancy status; S504, determine the grid occupancy status of spatial grid cells whose grid point cloud count is less than the grid occupancy threshold as non-occupancy status.

[0075] As an example, in step S501, the currently set grid resolution can be obtained, and the corresponding grid occupancy threshold can be determined based on the grid resolution; wherein, the grid occupancy threshold refers to the threshold of the number of point clouds used to determine whether a grid cell is occupied.

[0076] In one embodiment, the grid occupancy threshold can be set according to the grid resolution and point cloud density. For example, for a 0.2m resolution grid, if the point cloud density is high, the grid occupancy threshold can be set to 5 points; if the point cloud density is low, the grid occupancy threshold can be set to 3 points. This disclosure does not limit the value of the grid occupancy threshold.

[0077] As an example, in step S502, the number of background point clouds contained in each spatial grid cell can be counted to obtain the point cloud count result corresponding to each grid, i.e., the grid point cloud count result; where the grid point cloud count result refers to the statistical value of the number of point clouds in each grid cell.

[0078] In one embodiment, the point cloud count for each grid cell can be obtained by traversing all background point clouds, determining which grid cell each point belongs to, and then accumulating the point cloud count for each grid cell.

[0079] As an example, in step S503, the point cloud count result of each grid can be compared with the grid occupancy threshold. If the point cloud count result is greater than or equal to the occupancy threshold, the occupancy state of the grid cell is determined to be occupied.

[0080] As an example, in step S504, if the point cloud count result is less than the occupancy threshold, the occupancy state of the grid cell is determined to be non-occupancy. It should be noted that the non-occupancy state here only indicates that there are not enough point clouds in the grid, and it has not yet been distinguished whether it is free or unknown. The free and unknown states will be further determined in subsequent ray tracing steps.

[0081] As an example, such as Figure 6 As shown, step S105, which generates a dynamic target mesh based on the target point cloud data and the target global point cloud map, includes: S601, Based on the target pose data, project the target point cloud data onto the preset global coordinate system to obtain the projected target point cloud data; S602, calculate the distance between each point in the projected target point cloud data and the nearest neighbor point in the target global point cloud map to obtain point cloud distance data; S603, based on a preset distance threshold and point cloud distance data, determines dynamic target point cloud data from the projected target point cloud data; S604 performs gridding processing on dynamic target point cloud data to obtain dynamic target grids.

[0082] As an example, in step S601, the original point cloud data (i.e., target point cloud data) of the target frame can be transformed from the sensor local coordinate system to the preset global coordinate system based on the target pose data corresponding to the target frame, thereby obtaining the projected target point cloud data; wherein, the projected target point cloud data refers to the target frame point cloud data that has been projected into the global coordinate system.

[0083] In one embodiment, the coordinate transformation is the same as that used during mapping, transforming the point cloud from the local coordinate system to the global coordinate system using a rigid body transformation matrix. This ensures that the point cloud of the target frame is in the same coordinate system as the global point cloud map, allowing for subsequent spatial difference comparison.

[0084] As an example, in step S602, each point in the projected target point cloud can be calculated, the nearest point can be found in the target global point cloud map, and the Euclidean distance between the two points can be calculated to obtain the distance value corresponding to each point, i.e., point cloud distance data; where point cloud distance data refers to the set of distances from each point in the target point cloud to the nearest neighbor point in the target global point cloud map.

[0085] In one embodiment, nearest neighbor search can be implemented using spatial indexing structures such as KD-Tree or Octree to improve search efficiency. For each point in the target point cloud, the nearest point is found in the global point cloud map, and the Euclidean distance between the two points is calculated as the distance value of that point.

[0086] It should be understood that since the target global point cloud map is a high-precision map with a static background, if a point in the target point cloud belongs to a static background, then it should be able to find a corresponding point very close to it in the global map; conversely, if a point belongs to a dynamic object, then it cannot find a corresponding point in the global map, and its corresponding point cloud distance data is relatively large.

[0087] As an example, in step S603, the distance value of each point can be judged based on a preset distance threshold, and points with a distance greater than the preset distance threshold are identified as dynamic points, thereby extracting dynamic target point cloud data; wherein, dynamic target point cloud data refers to the point cloud data corresponding to the dynamic object separated from the target point cloud.

[0088] In one embodiment, the preset distance threshold can be set to 0.15 meters. The rationale for this setting is that, due to the high precision of the target global point cloud map, if the Euclidean distance between the target frame point cloud and the corresponding point on the global map is greater than this threshold, it is determined to be a dynamic object, thereby filtering out inherent system noise. In other words, the preset distance threshold is a key parameter for distinguishing between static and dynamic points, and can be adjusted according to the required precision and point cloud noise level. This disclosure does not limit its specific value.

[0089] As an example, in step S604, the extracted dynamic target point cloud data can be meshed (i.e., voxelized) to convert the dynamic point cloud into a mesh form, thereby obtaining a dynamic target mesh; wherein, the dynamic target mesh is three-dimensional mesh data containing the dynamic obstacle occupancy state.

[0090] In one embodiment, the resolution of the dynamic target mesh can be consistent with the resolution of the static background mesh to facilitate subsequent fusion processing. Each mesh cell is marked as occupied if it contains dynamic point clouds; otherwise, it is marked as unoccupied.

[0091] As an example, such as Figure 7 As shown, step S106, which generates ground truth environmental perception data corresponding to the target frame based on the static background mesh and the dynamic target mesh, includes: S701, which merges the dynamic target mesh and the static background mesh to obtain a joint mesh; S702, based on target pose data, determines the optical center position of the aircraft in the joint grid; S703 emits multiple probe rays from the optical center position into a preset field of view; S704, based on the intersection relationship between the probe ray and the joint grid, determines the grid occlusion state of the joint grid; S705 generates ground truth environmental perception data corresponding to the target frame based on the grid occlusion state.

[0092] As an example, in step S701, the dynamic target mesh and the static background mesh can be merged, that is, the occupancy states of the two meshes are combined to obtain a joint mesh; wherein, the joint mesh is a three-dimensional mesh that simultaneously contains static background and dynamic target occupancy information.

[0093] In one embodiment, the fusion rule can be: for the same grid cell, if either the static background grid or the dynamic target grid is occupied, then the cell in the joint grid is occupied; only when both are unoccupied is the cell in the joint grid unoccupied. In other words, the joint grid is the union of the static background and the dynamic target, representing all occupied space within the current sensing range.

[0094] As an example, in step S702, the optical center position of the aircraft in the joint grid can be determined based on the target pose data corresponding to the target frame; where the optical center position refers to the starting position of ray tracing, which corresponds to the position of the optical center of the aircraft's sensor in the joint grid space.

[0095] In one embodiment, the optical center is the optical center of the lidar sensor mounted on the aircraft. Since the sensor is fixedly mounted on the aircraft, the position of the optical center in the global coordinate system can be calculated using the aircraft's pose data and the sensor's mounting extrinsic parameters, and then converted into its relative position in the joint mesh. In other words, the position of the optical center is the starting point for ray tracing; only by determining the position of the optical center can the sensor's detection angle be accurately simulated.

[0096] It should be understood that since ray tracing is performed within the local space of the joint mesh, it is necessary to determine the relative position of the optical center within the joint mesh, which serves as the emission starting point of the ray within the joint mesh. This allows subsequent ray propagation and occlusion detection to be performed in the local coordinate system of the joint mesh, improving computational efficiency.

[0097] As an example, in step S703, multiple probe rays can be emitted in various directions within a preset field of view, starting from the optical center position; where the preset field of view refers to the field angle range of the sensor, and the probe rays are virtual rays used to simulate the sensor's probe beam.

[0098] In one embodiment, the preset field of view can be set according to the actual field of view parameters of the lidar, and the number and density of the detection rays can be set as needed. The denser the rays, the higher the accuracy of the occlusion inference.

[0099] For example, the density of the detection rays can be set according to the angular resolution of the lidar, so that each ray corresponds to an actual detection channel of the lidar, thereby more realistically simulating the sensor's detection process.

[0100] As an example, in step S704, the grid cells through which the ray passes and the grid cells that are blocked can be determined based on the intersection of each probe ray with the joint grid, thereby determining the grid occlusion state of each cell in the joint grid; wherein, the grid occlusion state includes two states: free and unknown / occluded.

[0101] In one embodiment, the specific process of ray-casting is as follows: virtual rays are emitted from the optical center to various angles, and the intersection of the rays with objects in three-dimensional space is calculated. When there is no point cloud in the voxel that the ray penetrates, it is marked as Free; when the ray is blocked by an occupied voxel, the voxels in the region behind it that have not been scanned are marked as Unknown.

[0102] For example, for a wall, a ray originates from the optical center and is blocked upon reaching the wall. The space in front of the wall is marked as Free, the wall itself is marked as Occupied, and the space behind the wall is marked as Unknown. This generates ground truth environmental data with realistic physical occlusion relationships, improving the expressive power of ground truth environmental data.

[0103] As an example, in step S705, the final environmental perception ground truth data can be generated based on the occupancy status, semantic labels, and occlusion status (idle / unknown) of the joint mesh; wherein, the environmental perception ground truth data is 4D OCC ground truth data, which includes the occupancy status (occupied, idle, unknown) and semantic labels of each voxel.

[0104] In one embodiment, the final environmental perception ground truth data includes three occupancy states: Occupied, indicating that the voxel is occupied by a static or dynamic object, and accompanied by a corresponding semantic label; Free, indicating that the voxel is empty, light can pass through, and there is no object; and Unknown, indicating that the voxel is occluded by an object in front, and it is impossible to determine whether it is occupied.

[0105] In summary, this disclosure has the following beneficial effects: 1. By projecting multi-frame environmental 3D point cloud data onto a preset global coordinate system using aircraft pose data, a target global point cloud map is obtained. Semantic annotation is then performed on the target global point cloud map to obtain a globally annotated map. A static background mesh is generated based on the globally annotated map, and a dynamic target mesh is generated based on the target point cloud data and the target global point cloud map. Finally, environmental perception ground truth data is generated, achieving high-precision and high-efficiency generation of environmental perception ground truth data, eliminating mapping drift errors, reducing annotation costs, and improving the spatiotemporal consistency of ground truth data.

[0106] 2. Initial pose data is obtained by jointly solving satellite navigation data and inertial measurement data through a tightly coupled solution algorithm, and the aircraft pose data is determined by accuracy verification based on a preset accuracy threshold. This achieves the acquisition of high-precision pose data and provides an accurate spatial reference for global mapping.

[0107] 3. By extracting planar features from the initial global point cloud map for ghost detection, and correcting the aircraft pose data based on the ghost detection results, ghost elimination of the point cloud map is achieved, improving the accuracy and consistency of the target global point cloud map.

[0108] 4. Background point cloud data is extracted from the globally labeled map using target pose data and preset perception range. After meshing, the grid occupancy status and grid semantic labels are determined to generate a static background grid, realizing automatic slicing generation of static background and ensuring semantic and spatial consistency of static background in each frame.

[0109] 5. The grid occupancy threshold is determined by the grid resolution, and the grid occupancy status is determined by comparing the grid point cloud count results with the grid occupancy threshold. This achieves accurate judgment of the grid occupancy status and improves the accuracy of static background grids.

[0110] 6. Point cloud distance data is obtained by calculating the nearest neighbor distance between the projected target point cloud data and the target global point cloud map, and dynamic target point cloud data is determined based on a preset distance threshold, thus achieving accurate separation of dynamic targets and improving the accuracy of dynamic target grids.

[0111] 7. By fusing dynamic target meshes and static background meshes to obtain a joint mesh, the mesh occlusion state is determined based on the intersection relationship between the probe ray and the joint mesh, generating ground truth data for environmental perception. This achieves accurate deduction of mesh occlusion state and improves the physical authenticity and topological integrity of ground truth data for environmental perception.

[0112] In one embodiment, an environmental perception truth data generation device is provided, which corresponds one-to-one with the environmental perception truth data generation method in the above embodiments. For example... Figure 8As shown, the environmental perception truth data generation device includes a data acquisition module 801, a map annotation module 802, a target determination module 803, a first grid generation module 804, a second grid generation module 805, and a truth generation module 806. Detailed descriptions of each functional module are as follows: The data acquisition module 801 is used to acquire multi-frame environmental 3D point cloud data collected by the aircraft, as well as the corresponding aircraft pose data; The map annotation module 802 is used to project multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, and to perform semantic annotation on the target global point cloud map to obtain a globally annotated map. The target determination module 803 is used to determine the target point cloud data corresponding to the target frame and the target pose data corresponding to the target frame from the multi-frame environmental 3D point cloud data. The target frame is any frame contained in the multi-frame environmental 3D point cloud data. The first mesh generation module 804 is used to generate a static background mesh based on the target pose data and the globally labeled map; The second grid generation module 805 is used to generate a dynamic target grid based on the target point cloud data and the target global point cloud map; The truth generation module 806 is used to generate environmental perception truth data corresponding to the target frame based on the static background grid and the dynamic target grid.

[0113] In one embodiment, the data acquisition module 801 is further configured to acquire lidar data, satellite navigation data, and inertial measurement data collected by the aircraft in the target area; The lidar data is analyzed to obtain the initial environmental 3D point cloud data, including the first timestamp; By jointly solving the satellite navigation data and inertial measurement data, initial pose data including the second timestamp is obtained; Based on a preset accuracy threshold, the initial pose data is verified for accuracy, and the initial pose data that passes the accuracy verification is determined as the aircraft pose data. Based on the first and second timestamps, the initial environmental 3D point cloud data is time-aligned to obtain multi-frame environmental 3D point cloud data.

[0114] In one embodiment, the map annotation module 802 is further configured to perform coordinate transformation on multi-frame environmental 3D point cloud data based on the aircraft pose data and a preset global coordinate system to obtain multi-frame global environmental point cloud data. The initial global point cloud map is obtained by overlaying multiple frames of global environmental point cloud data. Extract planar features from the initial global point cloud map, and perform ghosting detection on the initial global point cloud map based on the planar features to obtain the ghosting detection results; Based on the ghosting detection results, the aircraft pose data is corrected to obtain the corrected pose data; Based on the corrected pose data, ghosting correction is performed on the initial global point cloud map to obtain the target global point cloud map.

[0115] In one embodiment, the first mesh generation module 804 is further configured to extract corresponding three-dimensional point cloud data from a globally labeled map based on the target pose data and a preset perception range to obtain background point cloud data; The background point cloud data is processed into a grid to obtain multiple spatial grid cells; Based on the distribution of background point cloud data, the grid occupancy status of multiple spatial grid units is determined; Map the corresponding semantic labels in the globally labeled map to multiple spatial grid cells to determine the grid semantic labels; A static background grid is generated based on the grid occupancy status and grid semantic labels.

[0116] In one embodiment, the first mesh generation module 804 is further configured to obtain the mesh resolution of a plurality of spatial mesh cells and determine a mesh occupancy threshold based on the mesh resolution; The number of background point clouds contained in multiple spatial grid cells is counted to obtain the grid point cloud count results; The grid occupancy status of spatial grid cells whose grid point cloud count results are greater than or equal to the grid occupancy threshold is defined as the occupancy status. The grid occupancy status of spatial grid cells whose grid point cloud count is less than the grid occupancy threshold is determined as non-occupancy status.

[0117] In one embodiment, the second mesh generation module 805 is further configured to project the target point cloud data onto a preset global coordinate system based on the target pose data to obtain projected target point cloud data; Calculate the distance between each point in the projected target point cloud data and the nearest neighbor point in the target global point cloud map to obtain point cloud distance data; Based on a preset distance threshold and point cloud distance data, dynamic target point cloud data is determined from the projected target point cloud data; The dynamic target point cloud data is processed into a grid to obtain a dynamic target grid.

[0118] In one embodiment, the truth generation module 806 is further configured to fuse the dynamic target mesh and the static background mesh to obtain a joint mesh; Based on the target pose data, determine the optical center position of the aircraft in the joint grid; Starting from the optical center, multiple probe rays are emitted into the preset field of view; Based on the intersection relationship between the probe ray and the joint grid, the grid occlusion state of the joint grid is determined; Based on the grid occlusion state, generate ground truth environmental perception data corresponding to the target frame.

[0119] This invention provides an environmental perception truth data generation device, comprising: a data acquisition module for acquiring multi-frame environmental 3D point cloud data collected by an aircraft, and corresponding aircraft pose data; a map annotation module for projecting the multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, and performing semantic annotation on the target global point cloud map to obtain a globally annotated map; a target determination module for determining the target point cloud data corresponding to a target frame and the target pose data corresponding to the target frame from the multi-frame environmental 3D point cloud data, wherein the target frame is any frame contained in the multi-frame environmental 3D point cloud data; a first mesh generation module for generating a static background mesh based on the target pose data and the globally annotated map; a second mesh generation module for generating a dynamic target mesh based on the target point cloud data and the target global point cloud map; and a truth generation module for generating environmental perception truth data corresponding to the target frame based on the static background mesh and the dynamic target mesh. This device projects multi-frame environmental 3D point cloud data onto a preset global coordinate system based on aircraft pose data to construct a map, and performs one-time semantic annotation on the target global point cloud map to obtain a globally annotated map. This eliminates mapping drift errors and reduces the cost of repeated annotation of multiple frames to a single fixed cost. By generating a static background mesh through the globally annotated map and combining the target point cloud data with the target global point cloud map to generate a dynamic target mesh, it realizes the generation of dynamic and static joint environmental perception ground truth data, ensuring the spatiotemporal consistency of ground truth data.

[0120] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores data used in the environment-aware truth data generation method. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an environment-aware truth data generation method.

[0121] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an environment-aware truth data generation method.

[0122] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for generating environment-aware truth data.

[0123] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), IAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0125] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for generating environmental perception truth data, characterized in that, include: Acquire multi-frame environmental 3D point cloud data collected by the aircraft, as well as the corresponding aircraft pose data; Based on the aircraft pose data, the multi-frame environmental 3D point cloud data is projected onto a preset global coordinate system to obtain a target global point cloud map, and the target global point cloud map is semantically annotated to obtain a globally annotated map. The target point cloud data corresponding to the target frame and the target pose data corresponding to the target frame are determined from the multi-frame environmental 3D point cloud data, wherein the target frame is any frame contained in the multi-frame environmental 3D point cloud data. Based on the target pose data and the globally labeled map, a static background mesh is generated; Based on the target point cloud data and the target global point cloud map, a dynamic target mesh is generated; Based on the static background grid and the dynamic target grid, environmental perception ground truth data corresponding to the target frame is generated.

2. The method according to claim 1, characterized in that, The acquisition of multi-frame environmental 3D point cloud data collected by the aircraft, and the corresponding aircraft pose data, includes: Acquire lidar data, satellite navigation data, and inertial measurement data collected by the aircraft in the target area; The lidar data is analyzed to obtain initial environmental 3D point cloud data including the first timestamp; The satellite navigation data and inertial measurement data are jointly calculated to obtain initial pose data including the second timestamp; Based on a preset accuracy threshold, the initial pose data is verified for accuracy, and the initial pose data that passes the accuracy verification is determined as the aircraft pose data. Based on the first timestamp and the second timestamp, the initial environmental 3D point cloud data is time-aligned to obtain the multi-frame environmental 3D point cloud data.

3. The method according to claim 1, characterized in that, The step of projecting the multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map includes: Based on the aircraft pose data and the preset global coordinate system, coordinate transformation is performed on the multi-frame environmental 3D point cloud data to obtain multi-frame global environmental point cloud data. The multi-frame global environmental point cloud data are overlaid to obtain an initial global point cloud map; Extract planar features from the initial global point cloud map, and perform ghosting detection on the initial global point cloud map based on the planar features to obtain ghosting detection results; Based on the ghosting detection results, the aircraft pose data is corrected to obtain corrected pose data; Based on the corrected pose data, ghosting correction is performed on the initial global point cloud map to obtain the target global point cloud map.

4. The method according to claim 1, characterized in that, The step of generating a static background mesh based on the target pose data and the globally labeled map includes: Based on the target pose data and the preset perception range, the corresponding 3D point cloud data is extracted from the global labeled map to obtain the background point cloud data. The background point cloud data is processed into a grid to obtain multiple spatial grid cells; Based on the distribution of the background point cloud data, the grid occupancy status of multiple spatial grid units is determined; Map the corresponding semantic labels in the global labeled map to the multiple spatial grid cells to determine the grid semantic labels; The static background grid is generated based on the grid occupancy state and the grid semantic label.

5. The method according to claim 4, characterized in that, The determination of the grid occupancy status of multiple spatial grid units based on the distribution of the background point cloud data includes: Obtain the grid resolution of the plurality of spatial grid cells, and determine the grid occupancy threshold based on the grid resolution; The number of background point clouds contained in each of the multiple spatial grid cells is counted to obtain the grid point cloud count result; The grid occupancy status of spatial grid cells whose grid point cloud count result is greater than or equal to the grid occupancy threshold is determined as an occupancy status. The grid occupancy status of spatial grid cells whose grid point cloud count result is less than the grid occupancy threshold is determined as non-occupancy status.

6. The method according to claim 1, characterized in that, The step of generating a dynamic target mesh based on the target point cloud data and the target global point cloud map includes: Based on the target pose data, the target point cloud data is projected onto the preset global coordinate system to obtain the projected target point cloud data; Calculate the distance between each point in the projected target point cloud data and the nearest neighbor point in the target global point cloud map to obtain point cloud distance data; Based on a preset distance threshold and the point cloud distance data, dynamic target point cloud data is determined from the projected target point cloud data; The dynamic target point cloud data is processed into a grid to obtain the dynamic target grid.

7. The method according to claim 1, characterized in that, The step of generating environmental awareness ground truth data corresponding to the target frame based on the static background grid and the dynamic target grid includes: The dynamic target mesh and the static background mesh are fused to obtain a joint mesh; Based on the target pose data, the optical center position of the spacecraft in the joint grid is determined; Starting from the optical center position, multiple probe rays are emitted into the preset field of view; Based on the intersection relationship between the probe ray and the joint grid, the grid occlusion state of the joint grid is determined; Based on the grid occlusion state, the environmental perception ground truth data corresponding to the target frame is generated.

8. An environmental perception truth data generation device, characterized in that, include: The data acquisition module is used to acquire multi-frame environmental 3D point cloud data collected by the aircraft, as well as the corresponding aircraft pose data; The map annotation module is used to project the multi-frame environmental 3D point cloud data onto a preset global coordinate system based on the aircraft pose data to obtain a target global point cloud map, and to perform semantic annotation on the target global point cloud map to obtain a globally annotated map. The target determination module is used to determine the target point cloud data corresponding to the target frame and the target pose data corresponding to the target frame from the multi-frame environmental 3D point cloud data, wherein the target frame is any frame contained in the multi-frame environmental 3D point cloud data. The first mesh generation module is used to generate a static background mesh based on the target pose data and the globally labeled map; The second grid generation module is used to generate a dynamic target grid based on the target point cloud data and the target global point cloud map; The truth generation module is used to generate environmental perception truth data corresponding to the target frame based on the static background grid and the dynamic target grid.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the environmental perception truth data generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the environmental perception truth data generation method as described in any one of claims 1 to 7.

Images