Data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a storage medium, and the method is applied to the technical field of images, and comprises the steps: obtaining a region image of a region where a vehicle is located and a three-dimensional point cloud data complete set; determining a three-dimensional point cloud data subset of a monitored object from the three-dimensional point cloud data full set, wherein the monitored object is an object within a preset collision height range of the vehicle; converting the three-dimensional point cloud data subset of the monitored object into a two-dimensional point cloud data set, and mapping the two-dimensional point cloud data set to the regional image; determining a driving marking line of the vehicle in the regional image based on the two-dimensional point cloud data set; and obtaining a target area image based on the two-dimensional point cloud data set, the driving marking line and the area image. According to the scheme, the matching accuracy of the annotation data and the actual physical space is effectively improved.

Description

Technical Field

[0001] This application relates to the field of image technology, and more particularly to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In current methods of zoning vehicle driving areas, operators often need to manually mark driving and non-driving areas on images of the vehicle's environment. However, manual marking is susceptible to subjective biases, leading to significant discrepancies between the marked areas and the actual scene. Therefore, improving the accuracy of standard data has become a pressing technical problem. Summary of the Invention

[0003] This application provides a data processing method, apparatus, electronic device, and storage medium, aiming to improve the accuracy of image data annotation. The technical solution is as follows:

[0004] In a first aspect, this embodiment provides a data processing method, including: acquiring a regional image and a complete set of three-dimensional point cloud data of the area where the vehicle is located; determining a subset of three-dimensional point cloud data of a monitored object from the complete set of three-dimensional point cloud data, wherein the monitored object is an object within a preset collision height range of the vehicle; converting the subset of three-dimensional point cloud data of the monitored object into a two-dimensional point cloud dataset, and mapping the two-dimensional point cloud dataset onto the regional image; determining the vehicle's driving marking lines in the regional image based on the two-dimensional point cloud dataset; and obtaining a target regional image based on the two-dimensional point cloud dataset, the driving marking lines, and the regional image.

[0005] In the above technical solution, by employing a density-based clustering algorithm to automatically separate a subset of the monitoring object's point cloud from the full 3D point cloud set, a large number of invalid interference noise points in the environmental background are effectively filtered out, achieving accurate extraction of the monitoring object's point cloud data. This avoids the subjective judgment bias inevitably introduced by relying on operators to manually mark areas on the image, ensuring the objective authenticity of the monitoring object data. When converting the 3D point cloud subset of the monitoring object into 2D point cloud data and mapping it to the corresponding regional image, spatial coordinate transformation is performed based on sensor calibration parameters, ensuring the geometric consistency between the 3D physical space points and the 2D image pixel coordinates. This solves the problem of positional distortion of the driving area markings on the regional image caused by inherent perspective distortion, stitching errors, and blind spots in visual images. Finally, based on the density distribution characteristics of the mapped 2D point cloud data on the image plane, accurate driving area corresponding driving marking lines are automatically generated, avoiding the error problem of manually drawing marking lines and significantly improving the matching accuracy between the marking data and the actual physical space position.

[0006] In one feasible implementation, the driving marking lines of a vehicle are determined in a regional image based on a two-dimensional point cloud dataset, including: acquiring simulated visual information of the vehicle on the two-dimensional point cloud dataset; determining a target two-dimensional point cloud dataset from the two-dimensional point cloud dataset based on the simulated visual information; and obtaining the driving marking lines based on the target two-dimensional point cloud dataset.

[0007] In the above technical solution, a rigid coordinate transformation is performed based on the installation height, pitch angle, and field of view of the visual sensor during the data preprocessing stage. This accurately aligns the ground projection point cloud in the vehicle coordinate system to the independent optical coordinate system of each sensor, achieving a physical-level simulation of the three-dimensional spatial point cloud to the two-dimensional image observation perspective. The target two-dimensional point cloud dataset is generated by filtering through the field of view boundary constraints, ensuring that only the point cloud data of the actual observable area of ​​the sensor is retained. By calling a preset algorithm to perform line grouping operations on the target two-dimensional point cloud dataset to generate continuous closed driving marking lines, the blind spot error and subjective judgment bias in manual marking are eliminated. Therefore, the spatial accuracy and physical rationality of the true data of the passable area are effectively improved.

[0008] In one feasible implementation, determining a subset of the 3D point cloud data of the monitored object from the complete set of 3D point cloud data includes: calling a preset clustering algorithm to determine the clustering of the complete set of 3D point cloud data to obtain a subset of the 3D point cloud data of the monitored object.

[0009] In the above technical solution, the local density distribution of point cloud is calculated in real time through a dynamic neighborhood expansion mechanism, which automatically adapts to the spatial characteristics of objects of different sizes. This avoids the problem of over-segmentation or under-segmentation caused by fixed neighborhood radius in manual annotation, and at the same time eliminates the positional deviation of the object boundary subjectively defined by the annotator.

[0010] In one feasible implementation, after obtaining a subset of the 3D point cloud data of the monitored object, the method further includes: obtaining the height information of the monitored object based on the subset of the 3D point cloud data of the monitored object; if the height information of the monitored object is not within a preset height range, then filtering the subset of the 3D point cloud data corresponding to the monitored object.

[0011] In the above technical solution, the three-dimensional point cloud data subset corresponding to the monitored object is filtered by a preset height range, and only the three-dimensional point cloud data whose height information conforms to the preset range is retained. Compared with processing all three-dimensional point cloud data subsets in full, this can effectively improve the overall data processing speed.

[0012] In one feasible implementation, before converting the subset of three-dimensional point cloud data of the monitored object into a two-dimensional point cloud dataset, the method further includes: determining ground point cloud data from the subset of three-dimensional point cloud data and filtering the ground point cloud data.

[0013] In the above technical solution, by determining the ground point cloud data from the subset of 3D point cloud data and filtering it, the ground point cloud that accounts for the largest proportion in the scene and is irrelevant to obstacle monitoring is removed, which significantly reduces the data scale of subsequent processing and thus improves the overall data processing speed.

[0014] In one feasible implementation, the acquisition of the area image and the complete set of 3D point cloud data of the area where the vehicle is located includes: acquiring the first 3D point cloud data corresponding to the first timestamp and the second 3D point cloud data corresponding to the second timestamp, wherein the first timestamp is greater than the second timestamp;

[0015] The coordinate systems of the first and second 3D point cloud data are registered; the registered first and second 3D point cloud data are then superimposed to obtain the complete set of 3D point cloud data.

[0016] In one feasible implementation, before registering the coordinate systems of the first 3D point cloud data and the second 3D point cloud data, the method further includes: acquiring noise data of the first 3D point cloud data and the second 3D point cloud data, and filtering the noise data.

[0017] In one feasible implementation, after obtaining the first three-dimensional point cloud data corresponding to the first timestamp and the second three-dimensional point cloud data corresponding to the second timestamp, the method further includes: performing coordinate system transformation on the first three-dimensional point cloud data and the second three-dimensional point cloud data based on the vehicle's vehicle coordinate system.

[0018] In the above technical solution, by acquiring the first 3D point cloud data corresponding to the first time stamp and the second 3D point cloud data corresponding to the second time stamp, and performing coordinate system transformation between the two based on the vehicle's coordinate system, the spatial consistency of the data is effectively improved. By acquiring and filtering point cloud noise data, the data processing speed can be improved. Furthermore, coordinate system registration is performed on the transformed first 3D point cloud data and second 3D point cloud data to ensure spatiotemporal consistency. The registered first 3D point cloud data and second 3D point cloud data are superimposed to generate a complete set of 3D point cloud data. Compared with methods that rely on single frames or visual stitching, this solution uses multi-temporal point cloud superposition to construct a more continuous and complete spatial representation, which can effectively improve the data accuracy of 3D point cloud data.

[0019] Secondly, this embodiment provides a data processing apparatus, the apparatus comprising:

[0020] The data acquisition unit is used to acquire the area image and the complete set of 3D point cloud data of the area where the vehicle is located.

[0021] The data clustering unit is used to determine a subset of the 3D point cloud data of the monitored object from the full set of 3D point cloud data. The monitored object is an object within the preset collision height range of the vehicle.

[0022] The data conversion unit is used to convert a subset of the three-dimensional point cloud data of the monitored object into a two-dimensional point cloud dataset, and to map the two-dimensional point cloud dataset onto the region image;

[0023] Information annotation unit, used to determine vehicle driving marking lines in a regional image based on a two-dimensional point cloud dataset;

[0024] The image generation unit is used to obtain the target region image based on the two-dimensional point cloud dataset, driving marking lines, and region image.

[0025] Thirdly, this embodiment provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the data processing method as described above.

[0026] Fourthly, this embodiment provides a computer-readable storage medium storing a computer program, which, when executed, implements the data processing method described above.

[0027] In the above technical solution, by employing a density-based clustering algorithm to automatically separate a subset of the monitoring object's point cloud from the full 3D point cloud set, a large number of invalid interference noise points in the environmental background are effectively filtered out, achieving accurate extraction of the monitoring object's point cloud data. This avoids the subjective judgment bias inevitably introduced by relying on operators to manually mark areas on the image, ensuring the objective authenticity of the monitoring object data. When converting the 3D point cloud subset of the monitoring object into 2D point cloud data and mapping it to the corresponding regional image, spatial coordinate transformation is performed based on sensor calibration parameters, ensuring the geometric consistency between the 3D physical space points and the 2D image pixel coordinates. This solves the problem of positional distortion of the driving area markings on the regional image caused by inherent perspective distortion, stitching errors, and blind spots in visual images. Finally, based on the density distribution characteristics of the mapped 2D point cloud data on the image plane, accurate driving area corresponding driving marking lines are automatically generated, avoiding the error problem of manually drawing marking lines and significantly improving the matching accuracy between the marking data and the actual physical space position. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of a data processing method provided in this embodiment;

[0030] Figure 2 This is a flowchart illustrating a data processing method provided in this embodiment;

[0031] Figure 3 This is a schematic diagram of a data processing method provided in this embodiment;

[0032] Figure 4 This is a flowchart illustrating a data processing method provided in this embodiment;

[0033] Figure 5 This is a schematic diagram of a data processing method provided in this embodiment;

[0034] Figure 6 This is a flowchart illustrating a data processing method provided in this embodiment;

[0035] Figure 7 This is a flowchart illustrating a data processing method provided in this embodiment;

[0036] Figure 8 This is a schematic diagram of the structure of a data processing device provided in this embodiment;

[0037] Figure 9 This is a schematic diagram of the structure of an electronic device provided in this embodiment. Detailed Implementation

[0038] To make the features and advantages of this application more apparent and understandable, the technical solutions in this embodiment will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] The technical solutions in this application will now be described clearly and in detail with reference to the accompanying drawings. In the description of this embodiment, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of this embodiment, "multiple" refers to two or more than two.

[0040] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0041] This embodiment provides a data processing method, the execution subject of which is a data processing device or an electronic device equipped with a data processing device. A detailed description follows; it should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0042] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating a data processing method provided in this embodiment. For example... Figure 1 As shown, it should be noted that the vehicle's execution module in this solution includes, but is not limited to, sensors and a processor. The sensors include at least a LiDAR sensor and a vision sensor. The LiDAR sensor achieves 3D spatial ranging and point cloud data generation of the area where the vehicle is located by emitting laser pulses into the area and receiving the reflected laser pulse signals. The vision sensor captures 2D images of the area where the vehicle is located based on optical imaging principles. The processor receives and processes the ranging data from the LiDAR sensor to generate 3D point cloud data, and simultaneously reconstructs the scene corresponding to the area where the vehicle is located based on the 2D images captured by the vision sensor using stereo vision or depth estimation algorithms.

[0043] In real-world scenarios, a vision sensor installed on the vehicle captures image data of the area where the vehicle is located at a preset frequency. Combined with pre-calibrated intrinsic and extrinsic parameters of the vision sensor, a perspective transformation model is used to perform spatial geometric correction and projection transformation on multiple frames of image data. A multi-band fusion algorithm is then applied to eliminate stitching seams, ultimately generating a high-resolution panoramic image covering the vehicle's surrounding field of vision. The intrinsic parameters of the vision sensor describe the physical relationship between the optical center of the vision sensor and the imaging plane, including focal length, principal point coordinates, and lens distortion coefficient. The intrinsic parameters of the vision sensor define the projection geometric model from a three-dimensional spatial point to a two-dimensional image pixel. The extrinsic parameters of the vision sensor characterize the pose state of the vision sensor in global space, including rotation matrices and translation vectors.

[0044] The system uses a lidar sensor to emit laser pulses at a preset scanning frequency to the area where the vehicle is located. Each scan generates a single frame of 3D point cloud data with a timestamp. After the number of scans reaches a preset threshold or the cumulative working time triggers a stop condition, the system integrates the 3D point cloud data from all scans to form a complete set of 3D point cloud data.

[0045] To reduce computational complexity, spatial rasterization preprocessing is performed on the entire 3D point cloud data set. A neighborhood radius and a minimum point threshold are set. Unvisited points are randomly selected as core points, and all point cloud data in their neighborhood are searched. When the number of points in the neighborhood meets the minimum threshold, a new cluster is created and expanded to all density-reachable points. At the same time, the density reachability of boundary points is determined. After traversal, multiple independent clusters are separated. Invalid small clusters are filtered out based on cluster size constraints. Then, continuous frame point cloud motion consistency detection is used to distinguish dynamic object clusters from static background clusters. Finally, a subset of 3D point cloud data corresponding to the monitored object is output.

[0046] The preset algorithm is called to convert a subset of the 3D point cloud data of the monitored object into a subset of 2D point cloud data, and then the subset of 2D point cloud data is projected onto the region image to achieve accurate mapping between the 2D point cloud dataset and the region image.

[0047] The calibration parameters of the vision sensor are obtained, including at least the sensor's installation position, installation height, pitch angle, and field of view. A transformation matrix from the vehicle coordinate system to the vision sensor's optical coordinate system is constructed based on these parameters. The 2D point cloud dataset in the vehicle coordinate system is mapped to the optical coordinate system using this transformation matrix, generating simulated visual information that matches the field of view characteristics. Based on the field of view boundary, 2D point cloud data within the effective coverage area is selected as the target dataset. Spatial distribution analysis is performed on this dataset, and combining geometric continuity constraints and smoothness criteria, a piecewise interpolation algorithm is used to fit driving marking lines that conform to the road topology and meet closure requirements. Finally, a target region image is obtained by overlaying the 2D point cloud dataset, driving marking lines, and region images.

[0048] In this embodiment, a subset of the monitoring object's point cloud is automatically separated from the full 3D point cloud using a density clustering algorithm. This effectively filters out a large number of invalid interference noise points in the environmental background, achieving accurate extraction of the monitoring object's point cloud data. This avoids the subjective judgment bias inevitably introduced by relying on operators to manually mark areas on the image, ensuring the objective authenticity of the monitoring object data. When converting the 3D point cloud subset of the monitoring object into 2D point cloud data and mapping it to the corresponding regional image, spatial coordinate transformation is performed based on sensor calibration parameters. This ensures the geometric consistency between the 3D physical space points and the 2D image pixel coordinates, solving the problem of positional distortion of the driving area markings on the regional image caused by inherent perspective distortion, stitching errors, and blind spots in visual images. Finally, based on the density distribution characteristics of the mapped 2D point cloud data on the image plane, accurate driving area corresponding driving marking lines are automatically generated, avoiding the error problem of manually drawing marking lines and significantly improving the matching accuracy between the marking data and the actual physical space position.

[0049] based on Figure 1The scene diagram shown below will be combined with... Figures 2-7 This embodiment provides a detailed description of a data processing method.

[0050] Please see Figure 2 , Figure 2 This is a flowchart illustrating a data processing method provided in this embodiment. Figure 2 As shown, the method of this embodiment may include the following steps S101-S105.

[0051] S101, acquire the complete set of regional images and 3D point cloud data of the area where the vehicle is located.

[0052] In this embodiment, a visual sensor captures real-time environmental images of the area where the vehicle is located, and the obtained environmental images are stitched together to obtain an area image. A lidar sensor performs laser scanning on the area where the vehicle is located at a preset frequency to obtain three-dimensional point cloud data of the area where the vehicle is located. The obtained three-dimensional point cloud data is integrated to obtain a complete set of three-dimensional point cloud data.

[0053] Specifically, the system uses a visual sensor to synchronously acquire multi-view raw image data streams of the area where the vehicle is located at a high frame rate; real-time image feature matching is performed based on a feature point extraction algorithm; combined with the calibration results of the intrinsic and extrinsic parameters of the visual sensor, spatial geometric correction and projection transformation of multiple frames of images are performed through a perspective transformation model; and a multi-band fusion algorithm is used to eliminate stitching seams, ultimately generating a high-resolution panoramic image covering the vehicle's surround view range.

[0054] The system uses a lidar sensor to emit laser pulses at a preset scanning frequency towards the area where the vehicle is located. Each time the lidar sensor completes a scan of the area, it generates a frame of 3D point cloud data, and the completion time of the current scan is recorded as a timestamp for the 3D point cloud data. Once the lidar sensor has completed a preset threshold of scan cycles or reached a preset time threshold, it stops operating. The 3D point cloud data from all scan cycles are then integrated to generate a complete 3D point cloud dataset.

[0055] It should be noted that in this embodiment, the original 3D point cloud data is timestamped.

[0056] S102, determine the subset of three-dimensional point cloud data of the monitored object from the complete set of three-dimensional point cloud data.

[0057] Specifically, the entire 3D point cloud data set undergoes spatial rasterization preprocessing to reduce computational complexity. A neighborhood radius and a minimum point threshold are set, and unvisited 3D point clouds are randomly selected as core points. All 3D point cloud data within the neighborhood of these core points are searched. If the number of 3D point cloud data within the neighborhood reaches the minimum point threshold, a new cluster is created and expanded to all density-reachable points. Density reachability is determined for boundary points. After traversal, multiple independent clusters are separated, and invalid small clusters are filtered out using cluster size constraints. Finally, dynamic object clusters and static background clusters are distinguished through continuous frame point cloud motion consistency detection, and a subset of 3D point cloud data corresponding to the dynamic object is output. This method does not require a preset target number and can effectively handle point clouds of irregularly shaped objects.

[0058] S103 converts a subset of the 3D point cloud data of the monitored object into a 2D point cloud dataset, and maps the 2D point cloud dataset onto the region image.

[0059] Specifically, the X and Y coordinates of each 3D point cloud in the subset of the monitored object's 3D point cloud data are directly extracted as 2D planar positions. At the same time, the original Z coordinate of each point cloud is used as a height attribute and bound to the corresponding point to form a 2D point cloud dataset with a (X coordinate, Y coordinate, height attribute) structure.

[0060] The ground plane model parameters, including the plane normal vector and reference height, are fitted using a ground filtering algorithm. This ground plane model defines a horizontal reference plane in the vehicle coordinate system. Using the horizontal reference plane as the direction, each 2D point cloud data point is vertically mapped onto the plane, generating a corresponding 2D point set in the ground coordinate system. The projected 2D point set is then converted into continuously distributed data in physical coordinates, eliminating the height dimension and establishing a unified ground reference datum. Finally, a structured 2D point cloud dataset is output.

[0061] The system acquires the calibration parameters of the vehicle's pre-set vision sensors, including installation position, height, pitch angle, yaw angle, and horizontal field of view. Based on the installation parameters of each vision sensor, a fan-shaped scanning area is dynamically constructed. The spatial relationship between the vision sensor's position and the ground plane is calculated through geometric transformations. Two-dimensional point cloud data falling within the vision sensor's field of view is selected, ensuring that only the area actually visible to the vision sensor is processed. Using the vision sensor's intrinsic parameters such as focal length and principal point coordinates, and extrinsic parameters such as the transformation matrix, the coordinates corresponding to the selected two-dimensional point cloud data are converted into corresponding image pixel coordinates for the region image through perspective projection calculations. Pre-set pattern points are then generated at each image pixel coordinate.

[0062] S104, Determine vehicle driving marking lines in a region image based on a two-dimensional point cloud dataset.

[0063] Specifically, the calibration parameters of the vision sensor in the vehicle are obtained, including the installation position, installation height, pitch angle and field of view of the vision sensor in the vehicle, and a transformation matrix from the vehicle coordinate system to the optical coordinate system of the vision sensor is constructed based on the calibration parameters.

[0064] A transformation matrix is ​​used to map the 2D point cloud dataset in the vehicle coordinate system to the optical coordinate system of the vision sensor, generating simulated visual information that matches the field-of-view characteristics of the vision sensor. Based on the field-of-view boundaries in the simulated visual information, 2D point cloud data located within the maximum field-of-view coverage area of ​​the vision sensor are selected and defined as the target 2D point cloud dataset.

[0065] Spatial distribution analysis of the target two-dimensional point cloud is performed. Combining geometric continuity constraints and smoothness criteria, a piecewise interpolation algorithm is used to fit a continuous driving marking line that conforms to the road topology. The driving marking line must meet the closure requirement. Local distortion caused by sparse point cloud or noise is eliminated through iterative optimization. Finally, a high-precision driving marking line with complete structure is output.

[0066] S105, based on the two-dimensional point cloud dataset, driving marking lines, and region images, obtains the target region image.

[0067] Specifically, the 2D point cloud dataset is rasterized, divided into fixed-resolution grid cells, and the number of 2D point cloud data points within each grid cell is counted to generate a point cloud density distribution map. The coordinate point sequence of driving marking lines in the regional image is extracted and converted into vector graphics. Based on the coordinate system of the regional image, the spatial position of the point cloud density distribution map is aligned with that of the regional image through affine transformation. The vector graphics are then directly drawn onto the regional image according to a preset style, prioritizing the display of driving marking lines when they overlap with points in the density distribution map. The integrity of the 2D point cloud data and driving marking lines in the edge regions of the regional image is verified; missing data is supplemented using neighboring point interpolation, and the offset of the marking line ends is corrected. Finally, the fusion result is converted to the target format, the resolution is unified, and color space calibration is performed, ultimately yielding the target regional image generated by superimposing the 2D point cloud dataset, driving marking lines, and the regional image.

[0068] Please refer to the following: Figure 3 , Figure 3 This is a schematic diagram illustrating a data processing method provided in this embodiment. For example... Figure 3 As shown, there is a vehicle A in a running state in the regional image, and there are multiple vehicles in a stopped state around vehicle A. After processing the point cloud data of the area where vehicle A is located, several two-dimensional point cloud datasets and driving annotation lines are mapped in the regional image.

[0069] In this embodiment, a subset of the monitoring object's point cloud is automatically separated from the full 3D point cloud using a density clustering algorithm. This effectively filters out a large number of invalid interference noise points in the environmental background, achieving accurate extraction of the monitoring object's point cloud data. This avoids the subjective judgment bias inevitably introduced by relying on operators to manually mark areas on the image, ensuring the objective authenticity of the monitoring object data. When converting the 3D point cloud subset of the monitoring object into 2D point cloud data and mapping it to the corresponding regional image, spatial coordinate transformation is performed based on sensor calibration parameters. This ensures the geometric consistency between the 3D physical space points and the 2D image pixel coordinates, solving the problem of positional distortion of the driving area markings on the regional image caused by inherent perspective distortion, stitching errors, and blind spots in visual images. Finally, based on the density distribution characteristics of the mapped 2D point cloud data on the image plane, accurate driving area corresponding driving marking lines are automatically generated, avoiding the error problem of manually drawing marking lines and significantly improving the matching accuracy between the marking data and the actual physical space position.

[0070] Please see Figure 4 , Figure 4 This is a flowchart illustrating a data processing method provided in this embodiment. Figure 4 As shown, the method in this embodiment may include the following steps S201-S202.

[0071] S201, Obtain simulated visual information of the vehicle on a two-dimensional point cloud dataset.

[0072] Specifically, based on the installation parameters of the pre-installed vision sensors on the vehicle, including position coordinates, height, pitch angle, and horizontal field of view, a fan-shaped scanning area is constructed. The projection relationship between the camera position and the ground plane is calculated through geometric transformation, and two-dimensional point cloud data with azimuth corners within the field of view are selected to simulate the real-view observation information of the vision sensors on the area where the vehicle is located, which is recorded as simulated visual information.

[0073] S202, determine the target two-dimensional point cloud dataset from the two-dimensional point cloud dataset based on simulated visual information, and obtain the driving marking line based on the target two-dimensional point cloud dataset.

[0074] Specifically, based on simulated visual information, the maximum field of view of the two-dimensional point cloud dataset that the visual sensor can observe is determined. The two-dimensional point cloud data that can be observed within the maximum field of view is identified as the target two-dimensional point cloud data. The target two-dimensional point cloud data are then aggregated to obtain the target two-dimensional point cloud dataset. A piecewise smooth interpolation algorithm is then used to generate continuous closed driving marking lines based on the target two-dimensional point cloud dataset.

[0075] Please refer to the following: Figure 5 , Figure 5This is a schematic diagram illustrating a data processing method provided in this embodiment. For example... Figure 5 As shown, there is a two-dimensional point cloud data composed of multiple two-dimensional point clouds. The two-dimensional point cloud that can be observed within the maximum field of view of the vehicle is taken as the target two-dimensional point cloud, and the target two-dimensional point cloud is merged to form a line to obtain the driving marking line.

[0076] In this embodiment, by performing rigid coordinate transformation based on the installation height, pitch angle, and field of view of the visual sensors during the data preprocessing stage, the ground projection point cloud in the vehicle coordinate system is precisely aligned to the independent optical coordinate system of each sensor, realizing a physical-level simulation of the three-dimensional spatial point cloud to the two-dimensional image observation perspective. The target two-dimensional point cloud dataset is generated by filtering through the field of view boundary constraints to ensure that only the point cloud data of the actual observable area of ​​the sensor is retained. By calling a preset algorithm to perform line grouping operation on the target two-dimensional point cloud dataset to generate continuous closed driving marking lines, the blind spot error and subjective judgment deviation in manual marking are eliminated, thus effectively improving the spatial accuracy and physical rationality of the true data of the passable area.

[0077] In one feasible embodiment, the data processing method provided in this embodiment may include calling a preset clustering algorithm to determine the clustering of the entire set of three-dimensional point cloud data to obtain a subset of the three-dimensional point cloud data of the monitored object.

[0078] Specifically, a density-based spatial clustering algorithm is invoked. By setting a neighborhood search radius and a minimum point count threshold, all unlabeled points in the complete 3D point cloud data set are traversed. Local neighborhoods are dynamically constructed based on the spatial distribution characteristics of each point in the complete 3D point cloud data set. If the number of points in a neighborhood exceeds the threshold, the point is marked as a core point and a new cluster is created. Simultaneously, unvisited points in its neighborhood are recursively expanded, and point sets that satisfy density connectivity are iteratively merged. The generated candidate clusters are geometrically constrained and verified. Based on features such as bounding box volume and point cloud density distribution, noise and invalid clusters are filtered to achieve a subset of the 3D point cloud data of the monitored object that meets the preset morphological conditions.

[0079] In this embodiment, the local density distribution of the point cloud is calculated in real time through a dynamic neighborhood expansion mechanism, which automatically adapts to the spatial characteristics of objects of different sizes. This avoids the problem of over-segmentation or under-segmentation caused by fixed neighborhood radii in manual annotation, and eliminates the positional deviation of the object boundary subjectively defined by the annotator.

[0080] Because many objects in real-world scenarios contain redundant point cloud data that do not affect vehicle movement, this redundant point cloud data can severely impact subsequent data processing speed. Therefore, it is necessary to delete this redundant point cloud data. Please refer to [link / reference]. Figure 6 , Figure 6 This is a flowchart illustrating a data processing method provided in this embodiment. Figure 6 As shown, the method of this embodiment may include the following steps S301-S302.

[0081] S301, based on a subset of the 3D point cloud data of the monitored object, obtains the height information of the monitored object.

[0082] In this embodiment, the 3D point cloud data subset is a local data set segmented from the complete 3D point cloud, representing the geometric shape and spatial distribution of the monitored object. The 3D point cloud data subset consists of three mutually orthogonal spatial dimensions (X, Y, Z) of the point cloud in 3D coordinates, where X describes the absolute position information of the monitored object in the horizontal direction, Y represents the absolute position information of the monitored object in the horizontal direction, and Z represents the absolute position information of the monitored object in the vertical direction.

[0083] The generation of 3D point cloud data subsets relies on spatial segmentation algorithms or object detection techniques. By analyzing the point cloud density and geometric features in the entire 3D point cloud dataset, the dataset is divided into point cloud clusters that independently represent physical entities. Each 3D point cloud data subset contains the 3D coordinate data of all point clouds on the monitored object surface.

[0084] Specifically, coordinate analysis is performed on a subset of the 3D point cloud data of the monitored object to extract the 3D spatial coordinates (X, Y, Z) of each point cloud, where the Z-axis corresponds to the height direction. By statistically analyzing the Z-value distribution of all point clouds within the 3D point cloud data subset, the maximum, minimum, and average height values ​​of the monitored object are calculated.

[0085] The effective height range is defined by combining the preset collision height range. A multi-scale height feature fusion method is used to model the overall height distribution of a subset of 3D point cloud data, including the joint analysis of the absolute height above the ground and the relative height of the monitored object itself, and finally obtains the height information of the monitored object.

[0086] S302, if the height information of the monitored object is not within the preset height range, then the subset of the three-dimensional point cloud data corresponding to the monitored object will be filtered.

[0087] Specifically, the height information of the monitored object is matched with a preset height range. If the height information of the monitored object is greater than the maximum threshold of the preset height range, or less than the minimum threshold of the preset height range, the monitored object is determined to be an invalid monitored object. For example, if the detected object is ground noise or high-altitude interference, the monitored object is determined to be an invalid monitored object, and the corresponding 3D point cloud data subset is deleted.

[0088] For example, the monitored object is a gravel 0.1 meters deep in a depression on the ground or a drone point 3 meters high in the air, with a preset height range of 0.2 meters to 2.5 meters. In this case, the subset of 3D point cloud data corresponding to the gravel and the subset of 3D point cloud data corresponding to the drone are filtered.

[0089] In this embodiment, a subset of the 3D point cloud data corresponding to the monitored object is filtered by a preset height range, and only the number of 3D point cloud data whose height information conforms to the preset range is retained. Compared with processing all subsets of 3D point cloud data, this can effectively improve the overall data processing speed.

[0090] In one feasible embodiment, the data processing method provided in this embodiment may include determining ground point cloud data from a subset of three-dimensional point cloud data and filtering the ground point cloud data.

[0091] In this embodiment, the ground point cloud data is obtained based on the random sampling consensus algorithm, which includes, but is not limited to, the RANSAC algorithm. The random sampling consensus algorithm is an iterative robust method used to estimate mathematical model parameters from data containing a large amount of noise and outliers. It fits the initial model by randomly selecting the minimum point set, and calculates the number of interior points that conform to the model based on a distance threshold. After multiple iterations, the model with the most interior points is selected as the optimal solution.

[0092] Specifically, the parameters of the random sampling consensus algorithm are initialized, including the number of iterations and the distance threshold. The iteration process then begins, where three points are randomly selected in each iteration to fit a candidate plane equation. The distances from all points in the 3D point cloud subset to this candidate plane are calculated. If the distance to a point is less than the set distance threshold, that point is marked as an interior point. After all iterations are completed, the candidate plane with the largest set of interior points is selected as the optimal ground model. Subsequently, the point cloud is segmented based on this optimal ground model, extracting all points with distances less than the distance threshold to form ground point cloud data. This ground point cloud data is then deleted, and finally, a 3D point cloud subset of filtered ground point cloud data is output.

[0093] In one feasible implementation, the input 3D point cloud data subset is spatially meshed to project the 3D point cloud data subset onto a horizontal plane and uniformly divide it into 2D grid cells; for each 2D grid cell, local elevation extreme points are extracted based on a preset neighborhood range, and the elevation difference between all points in the 2D grid cell and the extreme points is calculated.

[0094] By setting a dynamic height threshold through statistical distribution analysis, points with elevation differences less than the threshold are selected as candidate ground points. Combined with the spatial continuity constraints of adjacent grid cells, the candidate ground points are subjected to region growing and morphological optimization to generate a complete ground point cloud surface model. Finally, by setting a safety margin threshold, the ground point cloud and the points in its adjacent buffer are removed from the 3D point cloud data subset, thus completing the filtering operation of the ground point cloud data.

[0095] In this embodiment, by determining and filtering ground point cloud data from a subset of 3D point cloud data, the ground point cloud data that accounts for the largest proportion in the scene and is irrelevant to obstacle detection is removed, which significantly reduces the data scale of subsequent processing and thus improves the overall data processing speed.

[0096] In one feasible implementation, the data processing method provided in this embodiment may include,

[0097] Based on the effective detection requirements of the vehicle vision perception system, spatial boundary constraint parameters for the area where the vehicle is located are set, including lateral distance range, longitudinal distance range, and height range thresholds. A cubic spatial bounding box with its axis parallel to the vehicle body direction is established in the vehicle coordinate system. This bounding box is strictly aligned with the vehicle's driving direction and limits the lateral span of the X-axis, the longitudinal depth of the Y-axis, and the height range of the Z-axis. Subsequently, the spatial coordinates of each 3D point cloud data in the complete 3D point cloud dataset are traversed to check whether they are inside the preset bounding box.

[0098] The system filters 3D point cloud data points that cross a preset bounding box, retaining only those points that are completely within the bounding box. It also sets a configurable safety buffer for the bounding box to ensure that critical points are not mistakenly removed. Finally, it outputs a complete set of 3D point cloud data filtered by spatial range, effectively reducing the amount of data processed and further improving data processing speed.

[0099] For example, spatial boundary constraint parameters are set according to visual perception requirements. The lateral distance range is defined as 4 meters to the left and right of the vehicle, the longitudinal distance range is limited to 1 meter to 60 meters in front of the vehicle, and the height range threshold is set to -1 meter to +3 meters. Based on this, a cubic spatial bounding box with its axis parallel to the vehicle body direction is established in the vehicle coordinate system. This bounding box is strictly aligned with the vehicle's driving direction, precisely limiting the X-axis lateral span to -4 meters to +4 meters, the Y-axis longitudinal depth to 1 meter to 60 meters, and the Z-axis height range to -1 meter to +3 meters. Then, the entire set of 3D point cloud data is traversed. For example, the original 3D point cloud data set has 1.2 million points: when processing the pedestrian point cloud (coordinates X=1.5, Y=25, Z=1.0) at 25 meters on the Y-axis and 1.5 meters on the X-axis, it is determined that it is inside the preset bounding box and is retained; the street light point cloud (X=-4.3) 5 meters away on the left is filtered out because it exceeds the lateral boundary; the point cloud at the ground depression (Z=-1.05) is critically retained because of the preset 0.1-meter safety buffer.

[0100] Data collected in real-world scenarios may exhibit inconsistencies, therefore maintaining data consistency is crucial. Please refer to [link / reference]. Figure 7 , Figure 7 This is a flowchart illustrating a data processing method provided in this embodiment. Figure 7 As shown, the method in this embodiment may include the following steps S401-S405.

[0101] S401, obtain the first three-dimensional point cloud data corresponding to the first timestamp and the second three-dimensional point cloud data corresponding to the second timestamp, wherein the first timestamp is greater than the second timestamp.

[0102] In this embodiment, the first timestamp is the timestamp of the most recent time when the LiDAR sensor completed a scan of the area where the vehicle is located, and the second timestamp is the historical timestamp of the most recent time when the LiDAR sensor scanned the area where the vehicle is located.

[0103] S402, coordinate system transformation is performed on the first three-dimensional point cloud data and the second three-dimensional point cloud data based on the vehicle's vehicle coordinate system.

[0104] In this embodiment, the original coordinate systems of the first and second 3D point cloud data correspond to the coordinate system of the LiDAR. The LiDAR coordinate system is the 3D spatial reference of the LiDAR sensor itself. Its origin is located at the optical center of the LiDAR, the X-axis points in the scanning direction directly in front of the LiDAR sensor, the Y-axis extends to the left perpendicular to the X-axis, and the Z-axis is vertically upward following the right-hand rule. The vehicle coordinate system is a global reference constructed with the vehicle's geometric center or the midpoint of the rear axle as its origin. The X-axis is parallel to the vehicle's longitudinal axis and points in the forward direction, the Y-axis extends laterally to the left, and the Z-axis is vertically upward perpendicular to the vehicle body. It is usually generated by chassis structural parameters and installation positioning measurements. A mapping relationship is established between the two through rigid body transformation. That is, multiple sets of corresponding point clouds are collected using a calibration board or feature matching method, and transformation parameters including rotation matrices and translation vectors are calculated to achieve accurate projection of the LiDAR point cloud onto the vehicle coordinate system, providing a unified spatial reference for multi-sensor data fusion.

[0105] Specifically, based on the pre-calibrated extrinsic transformation matrix between the lidar and vehicle coordinate systems, rigid body transformations are performed on the first and second 3D point cloud data, respectively. The 3D coordinates of each point cloud in both data are multiplied by a rotation matrix and a translation vector to map them from the lidar sensor coordinate system to the vehicle coordinate system. This eliminates coordinate differences caused by the lidar sensor's installation position or vehicle movement, achieving geometric correspondence between the first and second 3D point cloud data under a unified spatial reference.

[0106] S403: Obtain noise data from the first 3D point cloud data and the second 3D point cloud data, and filter the noise data.

[0107] In this embodiment, noise data refers to abnormal points in the first and second three-dimensional point cloud data caused by LiDAR sensor errors, environmental interference, or invalid reflections.

[0108] Specifically, a preset statistical outlier detection algorithm is invoked to dynamically define a local neighborhood containing K nearest neighbors for each point in the first and second 3D point cloud data, for example, K = 30. The mean and standard deviation of the 3D coordinates of the points in the neighborhood are calculated. If the Euclidean distance between the current point and the mean exceeds the preset standard deviation multiple threshold, it is determined to be outlier noise data and is removed.

[0109] S404, registers the coordinate systems of the first 3D point cloud data and the second 3D point cloud data.

[0110] Specifically, geometric features of the first and second 3D point cloud data are extracted in the lidar coordinate system, such as edge points with significant curvature or smooth planar regions, and descriptors are constructed based on the spatial distribution of the features. The correlation between corresponding feature points in the first and second 3D point cloud data is established through feature matching algorithms, and the relative pose transformation parameters of the first and second 3D point cloud data, i.e., rotation matrix and translation vector, are calculated using an optimization method that minimizes the distance error of the matching point pair.

[0111] The first and second 3D point cloud data are transformed into the lidar coordinate system according to the optimized pose parameters, so as to achieve accurate alignment of the first and second 3D point cloud data in a unified spatial reference.

[0112] S405, the registered first 3D point cloud data and the second 3D point cloud data are superimposed to obtain the complete set of 3D point cloud data.

[0113] Specifically, the registered first and second 3D point cloud data are fused and stitched together according to spatial coordinates. For point cloud data in overlapping areas, voxel raster downsampling is used to eliminate redundant data, preserving geometric details while reducing density differences; for non-overlapping areas, spatial stitching is performed directly to construct a 3D point cloud dataset that covers the entire scene and is spatiotemporally continuous.

[0114] In this embodiment, by acquiring the first 3D point cloud data corresponding to the first time stamp and the second 3D point cloud data corresponding to the second time stamp, and performing coordinate system transformation between the two based on the vehicle's coordinate system, the spatial consistency of the data is effectively improved. By acquiring and filtering point cloud noise data, the data processing speed can be improved. Furthermore, coordinate system registration is performed on the transformed first 3D point cloud data and second 3D point cloud data to ensure spatiotemporal consistency. The registered first 3D point cloud data and second 3D point cloud data are superimposed to generate a complete set of 3D point cloud data. Compared with methods that rely on single frames or visual stitching, this scheme uses multi-temporal point cloud superposition to construct a more continuous and complete spatial representation, which can effectively improve the data accuracy of 3D point cloud data.

[0115] based on Figure 1 The following is a scene illustration, which will be combined with... Figure 8 The data processing apparatus provided in this embodiment will be described in detail. It should be noted that... Figure 8 The data processing apparatus in the present application is used to execute the data processing apparatus in the present application. Figures 2-7 The method of the illustrated embodiment is shown only in the parts relevant to this embodiment for ease of explanation. For specific technical details not disclosed, please refer to this application. Figures 2-7In the illustrated embodiment, the data processing device 500 may include a data acquisition unit 501, a data clustering unit 502, a data conversion unit 503, a data annotation unit 504, and an image generation unit 505, as detailed below:

[0116] The data acquisition unit 501 is used to acquire the area image and the complete set of 3D point cloud data of the area where the vehicle is located.

[0117] Data clustering unit 502 is used to determine a subset of the three-dimensional point cloud data of the monitored object from the full set of three-dimensional point cloud data. The monitored object is an object within the preset collision height range of the vehicle.

[0118] The data conversion unit 503 is used to convert a subset of the three-dimensional point cloud data of the monitored object into a two-dimensional point cloud dataset, and to map the two-dimensional point cloud dataset onto the region image.

[0119] Information annotation unit 504 is used to determine the driving annotation lines of vehicles in the regional image based on a two-dimensional point cloud dataset;

[0120] Image generation unit 505 is used to obtain target region image based on two-dimensional point cloud dataset, driving marking lines and region image.

[0121] In some embodiments, the information annotation unit 504 further includes a visual information acquisition unit and an annotation unit.

[0122] The visual information acquisition unit is used to acquire simulated visual information of the vehicle on the two-dimensional point cloud dataset;

[0123] The annotation unit is used to determine the target 2D point cloud dataset from the 2D point cloud dataset based on simulated visual information, and to obtain the driving annotation line based on the target 2D point cloud dataset.

[0124] In some embodiments, the data acquisition unit 501 further includes a data partitioning unit.

[0125] The data partitioning unit is used to invoke a preset clustering algorithm to determine the clustering of the entire 3D point cloud data set, thereby obtaining a subset of the 3D point cloud data of the monitored object.

[0126] In some embodiments, the data acquisition unit 501 further includes a height information acquisition unit and a first data filtering unit.

[0127] The height information acquisition unit is used to obtain the height information of the monitored object based on a subset of the 3D point cloud data of the monitored object;

[0128] The first data filtering unit is used to filter the subset of three-dimensional point cloud data corresponding to the monitored object if the height information of the monitored object is not within the preset height range.

[0129] In some embodiments, the data conversion unit 503 further includes a second data acquisition unit.

[0130] The second data acquisition unit is used to determine ground point cloud data from a subset of 3D point cloud data and filter the ground point cloud data.

[0131] In some embodiments, the data acquisition unit 501 further includes a point cloud data acquisition unit, a coordinate registration unit, and a data integration unit.

[0132] The point cloud data acquisition unit is used to acquire the first three-dimensional point cloud data corresponding to the first timestamp and the second three-dimensional point cloud data corresponding to the second timestamp, wherein the first timestamp is greater than the second timestamp.

[0133] The coordinate registration unit is used to register the coordinate systems of the first 3D point cloud data and the second 3D point cloud data.

[0134] The data integration unit is used to overlay the registered first and second 3D point cloud data to obtain a complete set of 3D point cloud data.

[0135] In some embodiments, the data acquisition unit 501 further includes a third data filtering unit.

[0136] The third data filtering unit is used to acquire noise data from the first 3D point cloud data and the second 3D point cloud data, and to filter the noise data.

[0137] In some embodiments, the data acquisition unit 501 further includes a coordinate transformation unit.

[0138] The coordinate transformation unit is used to perform coordinate system transformation on the first three-dimensional point cloud data and the second three-dimensional point cloud data based on the vehicle's vehicle coordinate system.

[0139] In this embodiment, a subset of the monitoring object's point cloud is automatically separated from the full 3D point cloud using a density clustering algorithm. This effectively filters out a large number of invalid interference noise points in the environmental background, achieving accurate extraction of the monitoring object's point cloud data. This avoids the subjective judgment bias inevitably introduced by relying on operators to manually mark areas on the image, ensuring the objective authenticity of the monitoring object data. When converting the 3D point cloud subset of the monitoring object into 2D point cloud data and mapping it to the corresponding regional image, spatial coordinate transformation is performed based on sensor calibration parameters. This ensures the geometric consistency between the 3D physical space points and the 2D image pixel coordinates, solving the problem of positional distortion of the driving area markings on the regional image caused by inherent perspective distortion, stitching errors, and blind spots in visual images. Finally, based on the density distribution characteristics of the mapped 2D point cloud data on the image plane, accurate driving area corresponding driving marking lines are automatically generated, avoiding the error problem of manually drawing marking lines and significantly improving the matching accuracy between the marking data and the actual physical space position.

[0140] Furthermore, the data processing apparatus provided in the above embodiments and the data processing method embodiments belong to the same concept, and the implementation process can be found in the method embodiments, which will not be repeated here.

[0141] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. In addition, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0142] Please see Figure 9 This is a schematic diagram of the structure of an electronic device provided in this embodiment. Figure 9 As shown, the electronic device 600 includes a processor 601 and a memory 602. The processor 601 and the memory 602 are electrically connected.

[0143] Processor 601 is the control center of electronic device 600 and may include one or more processing cores. Processor 601 connects various parts of the electronic device using various interfaces and lines. By running or calling computer programs stored in memory 602, and by calling data stored in memory 602, it executes various functions and processes data of the electronic device, thereby providing overall control over the electronic device. Optionally, processor 601 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 601 may integrate one or more of the following: CPU, Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user page, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 601 and may be implemented separately using a communication chip.

[0144] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the computer programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0145] Furthermore, memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 602 may also include a memory controller to provide processor 601 with access to memory 602.

[0146] In this embodiment, the processor 601 in the electronic device 600 loads the instructions corresponding to the processes of one or more computer programs into the memory 602 according to the following steps, and the processor 601 runs the computer programs stored in the memory 602 to realize various functions, as follows:

[0147] Acquire the complete set of regional images and 3D point cloud data of the area where the vehicle is located;

[0148] A subset of the 3D point cloud data of the monitored object is determined from the full set of 3D point cloud data. The monitored object is the object within the preset collision height range of the vehicle.

[0149] A subset of the 3D point cloud data of the monitored object is converted into a 2D point cloud dataset, and the 2D point cloud dataset is mapped onto the region image.

[0150] Determine vehicle driving lines in regional images based on two-dimensional point cloud datasets;

[0151] The target region image is obtained based on the two-dimensional point cloud dataset, driving marking lines, and region images.

[0152] Optionally, the processor 601, when executing the process of determining the vehicle's driving marking lines in the region image based on the two-dimensional point cloud dataset, specifically performs the following steps: acquiring simulated visual information of the vehicle on the two-dimensional point cloud dataset; determining a target two-dimensional point cloud dataset from the two-dimensional point cloud dataset based on the simulated visual information; and obtaining the driving marking lines based on the target two-dimensional point cloud dataset.

[0153] Optionally, the processor 601, when executing the process of determining a subset of the three-dimensional point cloud data of the monitored object from the complete set of three-dimensional point cloud data, specifically executes: calling a preset clustering algorithm to determine the clustering of the complete set of three-dimensional point cloud data, thereby obtaining a subset of the three-dimensional point cloud data of the monitored object.

[0154] Optionally, after obtaining the subset of three-dimensional point cloud data of the monitored object, the processor 601 specifically performs the following: based on the subset of three-dimensional point cloud data of the monitored object, obtain the height information of the monitored object; if the height information of the monitored object is not within the preset height range, then filter the subset of three-dimensional point cloud data corresponding to the monitored object.

[0155] Optionally, before executing the conversion of a subset of the 3D point cloud data of the monitored object into a 2D point cloud dataset, the processor 601 specifically performs the following: determining ground point cloud data from the subset of 3D point cloud data and filtering the ground point cloud data.

[0156] Optionally, the processor 601, when executing the acquisition of the area image and the complete set of 3D point cloud data of the area where the vehicle is located, specifically performs the following: acquiring the first 3D point cloud data corresponding to the first time stamp and the second 3D point cloud data corresponding to the second time stamp, wherein the first time stamp is greater than the second time stamp; registering the coordinate systems of the first 3D point cloud data and the second 3D point cloud data; and superimposing the registered first 3D point cloud data and the second 3D point cloud data to obtain the complete set of 3D point cloud data.

[0157] Optionally, before performing coordinate system registration on the first three-dimensional point cloud data and the second three-dimensional point cloud data, the processor 601 specifically performs the following: acquiring noise data from the first three-dimensional point cloud data and the second three-dimensional point cloud data, and filtering the noise data.

[0158] Optionally, after the processor 601 executes the acquisition of the first three-dimensional point cloud data corresponding to the first time stamp and the second three-dimensional point cloud data corresponding to the second time stamp, it specifically performs the following: coordinate system transformation on the first three-dimensional point cloud data and the second three-dimensional point cloud data based on the vehicle's vehicle coordinate system.

[0159] In this embodiment, a subset of the monitoring object's point cloud is automatically separated from the full 3D point cloud using a density clustering algorithm. This effectively filters out a large number of invalid interference noise points in the environmental background, achieving accurate extraction of the monitoring object's point cloud data. This avoids the subjective judgment bias inevitably introduced by relying on operators to manually mark areas on the image, ensuring the objective authenticity of the monitoring object data. When converting the 3D point cloud subset of the monitoring object into 2D point cloud data and mapping it to the corresponding regional image, spatial coordinate transformation is performed based on sensor calibration parameters. This ensures the geometric consistency between the 3D physical space points and the 2D image pixel coordinates, solving the problem of positional distortion of the driving area markings on the regional image caused by inherent perspective distortion, stitching errors, and blind spots in visual images. Finally, based on the density distribution characteristics of the mapped 2D point cloud data on the image plane, accurate driving area corresponding driving marking lines are automatically generated, avoiding the error problem of manually drawing marking lines and significantly improving the matching accuracy between the marking data and the actual physical space position.

[0160] In addition, the device provided in this embodiment may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute a data processing method provided in the above embodiment.

[0161] This embodiment also provides a computer-readable storage medium storing a computer program. When the computer program is run on a computer, it causes the computer to perform the aforementioned method steps to implement a data processing method provided in the above embodiment.

[0162] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement a data processing method provided in the above embodiment.

[0163] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0164] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0165] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the related couplings or direct couplings or communication connections shown or discussed may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0166] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: Acquire the complete set of regional images and 3D point cloud data of the area where the vehicle is located; A subset of the three-dimensional point cloud data of the monitored object is determined from the complete set of the three-dimensional point cloud data, wherein the monitored object is an object within the preset collision height range of the vehicle; A subset of the three-dimensional point cloud data of the monitored object is converted into a two-dimensional point cloud dataset, and the two-dimensional point cloud dataset is mapped onto the region image; The driving marking lines of the vehicle are determined in the regional image based on the two-dimensional point cloud dataset; Based on the two-dimensional point cloud dataset, driving marking lines, and the region image, the target region image is obtained.

2. The method according to claim 1, characterized in that, Determining the vehicle's driving marking lines in the region image based on the two-dimensional point cloud dataset includes: Obtain the vehicle's simulated visual information about the two-dimensional point cloud dataset; Based on the simulated visual information, a target two-dimensional point cloud dataset is determined from the two-dimensional point cloud dataset, and driving marking lines are obtained based on the target two-dimensional point cloud dataset.

3. The method according to claim 1, characterized in that, The step of determining a subset of the three-dimensional point cloud data of the monitored object from the complete set of three-dimensional point cloud data includes: A preset clustering algorithm is invoked to determine the clustering of the complete set of 3D point cloud data, thereby obtaining a subset of the 3D point cloud data of the monitored object.

4. The method according to claim 3, characterized in that, After obtaining the subset of three-dimensional point cloud data of the monitored object, the method further includes: Based on a subset of the 3D point cloud data of the monitored object, the height information of the monitored object is obtained; If the height information of the monitored object is not within the preset height range, then the subset of the three-dimensional point cloud data corresponding to the monitored object will be filtered.

5. The method according to claim 1, characterized in that, Before converting the subset of three-dimensional point cloud data of the monitored object into a two-dimensional point cloud dataset, the method further includes: Ground point cloud data is determined from the subset of the three-dimensional point cloud data, and the ground point cloud data is filtered.

6. The method according to claim 1, characterized in that, The acquisition of the complete set of regional images and 3D point cloud data of the area where the vehicle is located includes: Obtain the first 3D point cloud data corresponding to the first timestamp and the second 3D point cloud data corresponding to the second timestamp, wherein the first timestamp is greater than the second timestamp; The coordinate systems of the first 3D point cloud data and the second 3D point cloud data are registered. The registered first 3D point cloud data and the second 3D point cloud data are overlaid to obtain a complete set of 3D point cloud data.

7. The method according to claim 6, characterized in that, Before registering the coordinate systems of the first 3D point cloud data and the second 3D point cloud data, the method further includes: Obtain noise data from the first 3D point cloud data and the second 3D point cloud data, and then filter the noise data.

8. The method according to claim 6, characterized in that, After obtaining the first 3D point cloud data corresponding to the first timestamp and the second 3D point cloud data corresponding to the second timestamp, the method further includes: The coordinate system of the first three-dimensional point cloud data and the second three-dimensional point cloud data are transformed based on the vehicle's vehicle coordinate system.

9. A data processing apparatus, characterized in that, The device includes: The data acquisition unit is used to acquire the area image and the complete set of 3D point cloud data of the area where the vehicle is located. A data clustering unit is used to determine a subset of the three-dimensional point cloud data of the monitored object from the full set of the three-dimensional point cloud data, wherein the monitored object is an object within a preset collision height range of the vehicle; The data conversion unit is used to convert a subset of the three-dimensional point cloud data of the monitored object into a two-dimensional point cloud dataset, and to map the two-dimensional point cloud dataset onto the region image; An information annotation unit is used to determine the driving annotation line of the vehicle in the regional image based on the two-dimensional point cloud dataset; The image generation unit is used to obtain a target region image based on the two-dimensional point cloud dataset, driving marking lines, and the region image.

10. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable program code; A processor is configured to call and run the executable program code from the memory, causing the electronic device to perform the data processing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the data processing method as described in any one of claims 1 to 8.

Images