A method for recovering a road boundary curve based on a BEV binary image and related equipment

Abstract

The application discloses a road boundary curve recovery method based on a BEV binary graph and related equipment. According to the scheme, a BEV binary graph based on a height difference is constructed as an intermediate representation, a large amount of training data can be generated at a low cost by using a simulator, a trained model can be directly and seamlessly migrated and deployed in a real physical world, the algorithm landing threshold and data cost are greatly reduced, the false detection rate caused by environmental noise can be reduced, and finally the parameterized equation of each road boundary is obtained, the recovery of the road boundary curve is completed, and the method can be widely applied to technical fields such as machine learning.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method and related equipment for road boundary curve recovery based on BEV binary maps. Background Technology

[0002] While visual sensors (Cameras) are widely used in urban autonomous driving, they face significant challenges at construction sites: roads are often covered in mud and gravel, causing the loss of color and texture features; strong lighting changes, dust, and nighttime working environments make color- and edge-gradient-based visual algorithms almost ineffective. Furthermore, existing LiDAR perception algorithms are mostly designed for structured urban roads, relying on clear curb heights or regular geometric features, while road boundaries at construction sites often appear as irregular mounds of earth, discontinuous ditches, or temporary fencing, accompanied by significant noise introduced by vibrations from construction equipment and terrain undulations.

[0003] Vision-based methods face fatal flaws in construction scenarios. First, construction roads are typically composed of compacted soil, asphalt, or gravel, lacking high-contrast white or yellow lane markings, making it impossible for visual networks to extract effective texture features. Second, dust, glare from puddles, and low-light conditions during nighttime operations all drastically reduce camera image quality, leading to serious missed and false detections.

[0004] Traditional lidar road boundary detection primarily relies on rules to extract boundaries based on the geometric attributes of point clouds (such as height, slope, and curvature). However, rule-based methods are extremely sensitive to parameters. On unstructured roads, the road surface itself may contain potholes, speed bumps, or ruts formed by heavy machinery. These non-boundary geometric undulations are easily misidentified as road edges. Furthermore, dust clouds in the air at construction sites generate a large number of outliers, and a single geometric rule is insufficient to distinguish these outliers from real obstacles, resulting in a high false alarm rate. Deep learning-based road edge detection (radar / point cloud) requires the collection of large amounts of real sensor data to achieve generalization.

[0005] In summary, existing technical solutions cannot simultaneously address the geometric complexity of unstructured environments, semantic filtering of dynamic interference, low computing power limitations of edge devices, and the Sim-to-Real generalization problem of model training. Summary of the Invention

[0006] The main objective of this application is to propose a road boundary curve recovery method and related equipment based on BEV binary maps, which can reduce the threshold for algorithm implementation and data costs, and also reduce the false detection rate caused by environmental noise.

[0007] To achieve the above objectives, one aspect of this application proposes a method for road boundary curve recovery based on BEV binary maps, comprising: Sensing components are pre-configured on the target engineering machinery equipment, and point cloud data and image data of the environment in which the target engineering machinery equipment is located are acquired through the sensing components; The point cloud data is spatiotemporally fused, then transformed to the vehicle center coordinate system, and the image data is inferred and recognized to obtain the dynamic objects in the image and their corresponding object information. Based on the point cloud data obtained by fusion and the object information of the dynamic object, determine the state label of each point in the point cloud data; Based on point cloud data with status labels, ground points and non-ground points are separated using the Patchwork algorithm; The separated point cloud is dimensionality reduced and projected onto the bird's-eye view grid plane to construct the BEV feature grid; Traverse each grid cell in the BEV feature grid, perform height difference feature extraction and binarization to obtain a BEV binary map; The BEV binary image is input into a lightweight LaneNet network for inference, and the output is a road boundary probability map and an embedded feature map; wherein, the road boundary probability map is used to represent the probability that each pixel belongs to the road boundary; Based on the road boundary probability map and the embedded feature map, instance segmentation processing is performed to cluster discrete pixels into independent road boundary lines to obtain an instance segmentation map, wherein the value of each pixel in the instance segmentation map represents the lane line label to which the pixel belongs; The discrete pixels are fitted to a mathematical curve, and the parameterized equations for each road boundary are output.

[0008] In some embodiments, the step of projecting the separated point cloud into a bird's-eye view grid plane to construct a BEV feature grid includes: Define a two-dimensional matrix that covers the region of interest surrounding the target engineering machinery equipment; Iterate through each point in the point cloud and calculate the grid index to which the point belongs based on its coordinates; The grid cells to which all points belong are determined, and a list data structure is assigned to each grid cell. The list data structure is used to store the attributes of all points falling into the current grid cell, thereby constructing the BEV feature grid.

[0009] In some embodiments, the step of traversing each grid cell in the BEV feature grid, performing height difference feature extraction and binarization to obtain a BEV binary map includes: Traverse each BEV grid cell, and for the height set within the BEV grid cell, calculate the height difference between two preset points in the height set; Candidate boundary grids are determined based on whether the height difference is within the preset road boundary height range; After removing points containing dynamic object labels within the candidate boundary grid, the remaining grid is assigned a value of 1 at the corresponding pixel position in the BEV image, and a value of 0 at the other pixel positions, thus obtaining a BEV binary image.

[0010] In some embodiments, the step of performing instance segmentation processing based on the road boundary probability map and the embedded feature map, clustering discrete pixels into independent road boundary lines to obtain an instance segmentation map, includes: Based on the road boundary probability map, points with high confidence are selected as the foreground point set; The embedding feature vector of each pixel is concatenated with the physical space coordinates of that pixel on the BEV binary map, and used as the clustering input feature. The DBSCAN algorithm is used to perform clustering processing on the clustering input features, and the output is an instance segmentation map.

[0011] In some embodiments, separating ground points and non-ground points based on point cloud data carrying state labels using the Patchwork algorithm includes: The point cloud data carrying status labels is projected onto the polar coordinate system, and the space is divided into multiple concentric annular regions and sectors according to radial distance and azimuth angle; For each sector obtained by division, select several points with lower heights and filter out extreme underground noise points by combining the average Z value. The remaining points are used as initial ground seed points. The covariance matrix of these seed points is calculated by principal component analysis algorithm, the normal vector is extracted and the initial plane equation is fitted, and the fitted plane is determined. The core parameters of the fitted plane are extracted and thresholded. Points that pass the threshold check are identified as ground points, while those that fail are identified as non-ground points. The core parameters include the elevation of the plane, the verticality / normal vector of the plane, and the flatness of the plane.

[0012] In some embodiments, the step of inputting the BEV binary map into a lightweight LaneNet network for inference and outputting a road boundary probability map and an embedded feature map includes: The Encoder-Decoder structure of the LaneNet network is modified to be lightweight by replacing the standard convolutional layers of the LaneNet network with depthwise separable convolutions and replacing the transposed convolutions in the decoder part with bilinear interpolation. The BEV binary map is inferred using the lightweight modified LaneNet network, and the road boundary probability map and embedded feature map are output.

[0013] In some embodiments, fitting discrete pixels to a mathematical curve and outputting a parameterized equation for each road boundary includes: Iterate through each instance number, extract the BEV coordinates of all pixels belonging to that instance number, and transform them back to the vehicle physical coordinate system. A cubic polynomial model is used to solve for the transformed coordinate parameters, thereby obtaining the coefficients of the parameterized equations of the corresponding road boundaries, and then determining the parameterized equations of each road boundary.

[0014] Another aspect of this application embodiment provides a road boundary curve restoration device based on a BEV binary map, comprising: The first module is used to pre-configure sensing components on the target engineering machinery equipment and acquire point cloud data and image data of the environment in which the target engineering machinery equipment is located through the sensing components. The second module is used to perform spatiotemporal fusion on the point cloud data, transform the point cloud data to the vehicle center coordinate system, and perform reasoning and recognition on the image data to obtain dynamic objects and corresponding object information in the image. The third module is used to determine the state label of each point in the point cloud data based on the point cloud data obtained by fusion and the object information of the dynamic object. The fourth module is used to separate ground points and non-ground points based on point cloud data with status labels using the Patchwork algorithm; The fifth module is used to reduce the dimensionality of the separated point cloud and project it onto the bird's-eye view grid plane to construct the BEV feature grid; The sixth module is used to traverse each grid cell in the BEV feature grid, perform height difference feature extraction and binarization processing, and obtain a BEV binary map. The seventh module is used to input the BEV binary map into a lightweight LaneNet network for inference and output a road boundary probability map and an embedded feature map; wherein, the road boundary probability map is used to represent the probability that each pixel belongs to the road boundary; The eighth module is used to perform instance segmentation processing based on the road boundary probability map and the embedded feature map, clustering discrete pixels into independent road boundary lines to obtain an instance segmentation map, wherein the value of each pixel in the instance segmentation map represents the lane line label to which the pixel belongs; The ninth module is used to fit discrete pixels into mathematical curves and output the parametric equations for each road boundary.

[0015] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0017] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0018] The embodiments of this application include at least the following beneficial effects: This application provides a road boundary curve recovery method and related equipment based on BEV binary maps. This scheme constructs a BEV binary map based on height difference as an intermediate representation, which can use a simulator to generate an unlimited amount of training data at low cost. The trained model can be directly and seamlessly transferred and deployed to the real physical world, which greatly reduces the threshold for algorithm implementation and data cost, can reduce the false detection rate caused by environmental noise, and finally obtains the parameterized equation of each road boundary to complete the recovery of the road boundary curve. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application; Figure 2 This is a flowchart of the overall steps provided in the embodiments of this application; Figure 3 This is a road boundary curve restoration result diagram provided in the embodiments of this application; Figure 4 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0021] It is understood that the terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0024] Before providing a detailed description of the embodiments of this application, some related technologies involved in the embodiments of this application will be described first, as follows: To address the problem of road boundary curve restoration, relevant technologies have employed methods such as vision-based restoration and deep learning-based restoration.

[0025] Among these, vision-based methods face fatal flaws in construction scenarios. First, construction roads are typically composed of compacted soil, asphalt, or gravel, lacking high-contrast white or yellow lane markings, making it impossible for visual networks to extract effective texture features. Second, dust, glare from puddles, and low-light conditions during nighttime operations all cause a sharp decline in camera image quality, resulting in serious missed and false detections.

[0026] Traditional lidar road boundary detection primarily relies on rules based on the geometric attributes of point clouds (such as height, slope, and curvature) for extraction. However, rule-based methods are extremely sensitive to parameters. On unstructured roads, the road surface itself may contain potholes, speed bumps, or ruts formed by heavy machinery. These non-boundary geometric undulations are easily misidentified as road edges. Furthermore, dust clouds in the air at construction sites generate a large number of outliers, and a single geometric rule is insufficient to distinguish these outliers from real obstacles, resulting in a high false alarm rate.

[0027] Furthermore, deep learning-based roadside detection (radar / point cloud) requires the collection of a large amount of real sensor data to achieve generalization requirements.

[0028] In summary, existing technical solutions cannot simultaneously address the geometric complexity of unstructured environments, semantic filtering of dynamic interference, low computing power limitations of edge devices, and the Sim-to-Real generalization problem in model training.

[0029] Based on this, this application proposes a road boundary curve recovery method based on BEV binary maps, which can reduce the threshold for algorithm implementation and data costs, and also reduce the false detection rate caused by environmental noise. This solution pre-configures a sensing component on the target construction machinery equipment, and acquires point cloud data and image data of the environment in which the target construction machinery equipment is located through the sensing component; the point cloud data is spatiotemporally fused, and then transformed to the vehicle center coordinate system; the image data is then used for inference and recognition to obtain dynamic objects and corresponding object information in the image; based on the fused point cloud data and the object information of the dynamic objects, the state label of each point in the point cloud data is determined; based on the point cloud data carrying state labels, ground points and non-ground points are separated using the Patchwork algorithm; the separated point cloud is then dimensionality-reduced and projected onto a bird's-eye view grid plane to construct a BEV feature grid; Each grid cell in the BEV feature grid is traversed, and height difference features are extracted and binarized to obtain a BEV binary map. The BEV binary map is then input into a lightweight LaneNet network for inference, outputting a road boundary probability map and an embedded feature map. The road boundary probability map represents the probability that each pixel belongs to a road boundary. Based on the road boundary probability map and the embedded feature map, instance segmentation is performed, clustering discrete pixels into independent road boundary lines to obtain an instance segmentation map. In the instance segmentation map, the value of each pixel represents the lane number to which that pixel belongs. The discrete pixels are fitted to a mathematical curve, and the parametric equation for each road boundary is output.

[0030] The road boundary curve restoration method and related equipment based on BEV binary maps provided in this application relate to the technical fields of machine learning, computer vision, intelligent transportation, and image recognition. The road boundary curve restoration method based on BEV binary maps provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the road boundary curve restoration method based on BEV binary maps, but is not limited to the above forms.

[0031] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0032] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0033] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided in an embodiment of this application. (Refer to...) Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0034] Server 101 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0035] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0036] Terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc. It can also be a vehicle-mounted terminal of the various device types described above, but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment does not impose any limitations.

[0037] Exemplary based on Figure 1 The implementation environment shown in this application embodiment provides a road boundary curve recovery method based on BEV binary map. The following description uses the application of this BEV binary map-based road boundary curve recovery method in server 101 as an example. It can be understood that this method can also be applied to terminal 102.

[0038] Reference Figure 2 , Figure 2 The flowchart illustrates a method for restoring road boundary curves based on BEV binary maps applied to a server, as provided in this application embodiment. The execution entity of this method can be any of the aforementioned computer devices (including a server or terminal). (Refer to...) Figure 2 The method may include the following steps: S201. Pre-configure sensing components in the target engineering machinery equipment, and acquire point cloud data and image data of the environment in which the target engineering machinery equipment is located through the sensing components; S202. The point cloud data is spatiotemporally fused, and then the point cloud data is transformed to the vehicle body center coordinate system. The image data is then inferred and recognized to obtain the dynamic objects in the image and the corresponding object information. S203. Based on the point cloud data obtained by fusion and the object information of the dynamic object, determine the state label of each point in the point cloud data. S204. Based on the point cloud data carrying status labels, separate ground points and non-ground points using the Patchwork algorithm; S205. Dimensionally reduce the separated point cloud and project it onto the bird's-eye view grid plane to construct the BEV feature grid; S206. Traverse each grid cell in the BEV feature grid, perform height difference feature extraction and binarization to obtain a BEV binary map. S207. Input the BEV binary image into a lightweight LaneNet network for inference, and output a road boundary probability map and an embedded feature map; wherein, the road boundary probability map is used to represent the probability that each pixel belongs to the road boundary; S208. Based on the road boundary probability map and the embedded feature map, perform instance segmentation processing to cluster discrete pixels into independent road boundary lines to obtain an instance segmentation map, wherein the value of each pixel in the instance segmentation map represents the lane line label to which the pixel belongs. S209. Fit discrete pixels to a mathematical curve and output the parameterized equation for each road boundary.

[0039] Specifically, regarding step S201 above, taking construction machinery (e.g., a road roller) as the target construction machinery equipment as an example, in order to achieve all-round perception, a "camera + lidar" perception kit is installed at both the front and rear of the construction machinery (e.g., the road roller). This may include the following steps: 1. Intrinsic parameter calibration: Perform intrinsic parameter calibration for each camera to obtain distortion coefficients and intrinsic parameter matrices for image distortion correction; 2. Radar-camera extrinsic parameter calibration: Accurately calculate the radar coordinate system using a calibration board or joint optimization algorithm. {L} To the camera coordinate system {C} The rigid body transformation matrix, including the rotation matrix RL 2 C Translation vector TL 2 C This step is fundamental to projecting 3D point clouds into 2D images; 3. Multi-radar extrinsic parameter calibration: calibrating the coordinate system of the two radars before and after calibration relative to the vehicle's center coordinate system. {V} The transformation relationship is used to integrate multiple point clouds into a unified vehicle coordinate system and eliminate spatial position differences.

[0040] Specifically, step S202 above may include the following steps: The system synchronously collects point cloud data from the front and rear radars and image data from the front and rear cameras in real time.

[0041] Point cloud data format: The original data is N× A matrix of 4, containing {x,y,z,intensity} ,in( x,y,z () represents three-dimensional spatial coordinates. intensity The intensity is the reflection intensity.

[0042] Spatiotemporal fusion: Based on calibration parameters, the forward radar point cloud is fused... P front and backward radar point cloud P rear Unified transformation to the vehicle body center coordinate system: P fused =( R front ·P front + Tfront ) ∪ ( R rear ·P rear + T rear ), in, P fused Represents unification to the vehicle-centered coordinate system {V} The following is a collection of full-coverage point clouds. R front and R rear These are the extrinsic rotation matrices for the forward and backward radars, respectively. T front and T rear It is a translation vector. P front and P rear For the forward and backward original point clouds, ∪ This indicates the merging / stitching of point cloud sets. The resulting point cloud is... P fused It can cover 360 degrees around the vehicle or key operational areas, solving the problem of limited field of view (FOV) of a single radar.

[0043] Image object detection: In parallel, this application embodiment uses the YOLO object detection network to infer the images from the front and rear cameras, identify dynamic objects (people, trucks, cones, etc.) in the scene, and output the corresponding 2D bounding boxes and category labels.

[0044] It should be noted that in most cases, forward-looking radar and camera are sufficient, but the embodiments of this application fuse front and rear radar and camera in order to further enhance stability.

[0045] Specifically, for step S203 above, in order to utilize the rich semantic information of vision to assist geometric perception, this scheme performs a "projection + table lookup" operation, which may include the following steps: 1. Projection: Utilizing the extrinsic parameter matrix of the radar-camera system ( R L2C , T L2C ) and camera internal parameters K Each point in the merged 3D point cloud P i ( x, y, z Projected onto the image plane ( u, v ): , in,s This represents the homogeneous scale factor in the perspective projection process. K For the camera intrinsic parameter matrix, R L2C For rotation matrix, T L2C It is a translation vector.

[0046] 2. Semantic injection: For projection points ( u,v The system employs a point-aligned bounding box (AABB) detection algorithm to check whether a point falls within a bounding box detected by YOLO. Specifically, the system extracts the coordinates of the top-left corner of the bounding box output by YOLO. x min , y min ) and the coordinates of the bottom right corner ( x max , y max For any projection point ( u, v If satisfied x min ≤u≤x max and y min ≤v≤y max If the point falls within the bounding box, then it is determined that the point is within the bounding box.

[0047] If the point falls within the bounding box of a "dynamic object" (such as Person, Vehicle), then assign that 3D point... P i Label it as "Dynamic". If it falls within the box for "Static Objects" (such as Cone), label it accordingly. This step expands the semantics of 2D images to the attributes of 3D point clouds, providing a basis for subsequent removal of dynamic interference.

[0048] Specifically, step S204 above may include the following steps: The point cloud data carrying status labels is projected onto the polar coordinate system, and the space is divided into multiple concentric annular regions and sectors according to radial distance and azimuth angle; For each sector obtained by division, select several points with lower heights and filter out extreme underground noise points by combining the average Z value. The remaining points are used as initial ground seed points. The covariance matrix of these seed points is calculated by principal component analysis algorithm, the normal vector is extracted and the initial plane equation is fitted, and the fitted plane is determined. The core parameters of the fitted plane are extracted and thresholded. Points that pass the threshold check are identified as ground points, while those that fail are identified as non-ground points. The core parameters include the elevation of the plane, the verticality / normal vector of the plane, and the flatness of the plane.

[0049] Before extracting road boundaries, ground points and non-ground points must be separated. This solution uses the Patchwork algorithm, which is specifically designed for non-flat terrain. The core processing steps of this algorithm can be described in the following three main stages: 1. Concentric Zone Model (CZM): Since the point cloud generated by LiDAR scanning is extremely dense at close range and sparse at distant range, traditional uniform grid partitioning easily leads to unbalanced feature extraction. The purpose of CZM is to establish a non-uniform polar grid, allocating point cloud space with appropriate density to regions at different distances, thereby significantly reducing computational complexity while maintaining feature representation capabilities. Patchwork first projects the 3D point cloud onto a polar coordinate system, dividing the space into multiple concentric ring regions (Zones) and sectors (Bins) according to radial distance and azimuth angle. Specific implementation: This application divides the space surrounding the vehicle into a central area, a quarter area, a half area, and an outer area, from near to far. Different sized sectors are used for areas at different distances, ensuring that each sector receives a point cloud of appropriate density. 2. Region-wise Ground Plane Fitting (R-GPF): Within each independent sector (Bin) divided by CZM, the algorithm assumes that the local ground is approximately flat and performs independent plane fitting on it. Initial seed point extraction: Within the current sector, select several points with relatively low height (Z coordinate), and filter out extreme underground noise points by combining the average Z value, and use them as initial ground seed points. PCA plane fitting: The principal component analysis (PCA) algorithm is used to calculate the covariance matrix of these seed points, extract the normal vector, and fit the initial plane equation.

[0050] Iterative convergence: Calculate the orthogonal distance from all points within the current sector to the fitted plane. If the distance is less than a set threshold (e.g., 15cm), these points are used as new seed points, and PCA fitting is performed again. This process is iterated several times (usually around 3 times) to make the fitted plane gradually approximate the actual local terrain undulations; 3. Ground Likelihood Estimation (GLE): The GLE module is responsible for validating the plane fitted to each sector. Specific implementation: The system extracts three core parameters of the fitted plane for evaluation: the plane's elevation, the plane's uprightness / normal vector, and the plane's flatness. Decision output: Only the fitted points within a sector that passes the GLE threshold check will be officially marked as "ground points" (marked as 1); otherwise, all points within that sector will be considered obstacles or "non-ground points" (marked as 0).

[0051] Algorithm logic: Patchwork uses a concentric zone model (CZM) to divide the point cloud into multiple zones and sectors based on distance and angle.

[0052] Advantages: Compared to simple RANSAC global plane fitting, Patchwork can adapt to the slope, undulation and gullies of the construction site, accurately separating ground points from non-ground points.

[0053] Ultimately, each point receives a label. g∈{ 0 , 1 } Indicates whether it belongs to the ground.

[0054] It should be noted that the embodiments of this application use the Patchwork algorithm to separate ground points from non-ground points. In other scenarios, the Patchwork algorithm can be replaced by RANSAC (Random Sample Consensus) plane fitting, CSF (Cloth Simulation Filtering) algorithm, or a ground segmentation network based on deep learning (such as GndNet). The purpose of the embodiments of this application is to assist in separating the ground from obstacles, thereby serving the subsequent BEV feature extraction. Therefore, the content that uses relevant algorithms to achieve this purpose is within the protection scope of this application and will not be described in detail here.

[0055] Specifically, step S205 above may include the following steps: Define a two-dimensional matrix that covers the region of interest surrounding the target engineering machinery equipment; Iterate through each point in the point cloud and calculate the grid index to which the point belongs based on its coordinates; The grid cells to which all points belong are determined, and a list data structure is assigned to each grid cell. The list data structure is used to store the attributes of all points falling into the current grid cell, thereby constructing the BEV feature grid.

[0056] It should be noted that the system described in this application embodiment is a grayscale or binary image of Bird's Eye View (BEV), and the pixel value of the image does not represent the intensity of reflection, but rather the "height difference" or "height change rate", which is different from the relevant prior art.

[0057] In some embodiments, this application projects the processed 3D point cloud into a 2D bird's-eye view (BEV) grid plane after dimensionality reduction. This is a crucial step in connecting 3D perception with 2D image networks. The specific implementation process includes: 1. Mesh Definition: Define a two-dimensional matrix that covers the region of interest (ROI) around the vehicle, i.e., H×W There are 10 grid cells. In this scheme, the size of each cell is 0. . twenty three × 0 . twenty three; 2. Information Aggregation: Traverse all point clouds and, based on their ( x,y The coordinate calculation belongs to the grid index. r,c In autonomous driving, the vehicle center (0) is often used. , 0) Placed at the physical center of the BEV mesh. Assume the total physical length and width of the BEV mesh are respectively... X and Y The conversion formula is: ， in, s For the physical resolution of the grid (here) s =0 . 23 meters). x min ,y min X is the minimum boundary value set for the region of interest (ROI) perceived by the vehicle in the physical coordinate system; grids The total number of grid cells in the X-axis direction (vehicle's forward / backward direction), Y grids This represents the total number of grid cells along the Y-axis (the left-right direction of the vehicle). In this way, every point finds its assigned grid cell; in other words, the grid cells are also aggregating the points belonging to them. Various hyperparameters, such as grid resolution and the area of ​​interest (ROI), can be adjusted according to the scene requirements and will not be elaborated upon here.

[0058] Each grid cell will be assigned a list data structure to store all points that fall into that cell (if there are...). N (each) attributes: height set Z = {z 1 ,z 2 ,...,zN } Semantic tag set L = {l 1 ,l 2 ,...,l N } Ground marker set G = {g 1 , g 2 ,...,g N } .

[0059] In some embodiments, the step of traversing each grid cell in the BEV feature grid, performing height difference feature extraction and binarization to obtain a BEV binary map includes: Traverse each BEV grid cell, and for the height set within the BEV grid cell, calculate the height difference between two preset points in the height set; Candidate boundary grids are determined based on whether the height difference is within the preset road boundary height range; After removing points containing dynamic object labels within the candidate boundary grid, the remaining grid is assigned a value of 1 at the corresponding pixel position in the BEV image, and a value of 0 at the other pixel positions, thus obtaining a BEV binary image.

[0060] Specifically, step S206 above may include the following steps: Traverse each BEV grid cell, and for the height set within the BEV grid cell, calculate the height difference between two preset points in the height set; Candidate boundary grids are determined based on whether the height difference is within the preset road boundary height range; After removing points containing dynamic object labels within the candidate boundary grid, the remaining grid is assigned a value of 1 at the corresponding pixel position in the BEV image, and a value of 0 at the other pixel positions, thus obtaining a BEV binary image.

[0061] It should be noted that step S206 above aims to extract robust boundary features from noisy data and is the core processing step of this scheme. The implementation process of this step can be as follows: Traverse each BEV mesh cell and perform the following calculations: 1. Robust height difference calculation: For the height set within the mesh Z Calculate its 95th percentile (95 th The difference between the 5th percentile and the 1st percentile: Δ H = Percentile (Z, 95) Percentile ( Z, 5).

[0062] Principle: Traditional Z max Z min The method is highly susceptible to the influence of flypoints from lidar (caused by dust or specular reflection). For example, even a single suspended flypoint in a normal, flat ground grid can cause a significant difference between its maximum and minimum values, leading to misjudgments. Using... P 95 P The 5 statistics, which eliminate outliers at both ends, can robustly reflect the vertical distribution span of the main point cloud within the grid. 2. Threshold filtering: Determine Δ H Is it within the preset road boundary height range (e.g., 5)? cm< Δ H< 30 cm Mesh grids that meet the conditions are considered candidate boundary grids; 3. Semantic and Ground Filtering: If the grid contains a large number of points with the semantic label "dynamic object" (injected in step 5.3), they are directly removed to avoid mistaking vehicles or pedestrians for road boundaries; ground information from Patchwork is used to further assist in the judgment. 4. Generate a binary image: Finally, the retained mesh is used in the BEV image. I input The corresponding pixel position is assigned a value of 1, and the rest are 0.

[0063] This binary image clearly delineates the locations of geometrical abrupt changes in the scene (such as curbs and embankments), and removes most of the noise and interference from dynamic objects, making it an ideal input for subsequent neural networks.

[0064] It should be noted that this BEV binary map representation can be generated infinitely using computer simulation without the need for real point cloud acquisition. This is because the representation pattern is sufficiently simple, with virtually no Sim-to-Real Gap. Therefore, the training cost of this model is very low.

[0065] In addition, the embodiments of this application may also use variance or standard deviation to characterize high dispersion; or adopt the method of adjusting quantiles, the relevant content will not be elaborated further.

[0066] Specifically, step S207 above may include the following steps: The Encoder-Decoder structure of the LaneNet network is modified to be lightweight by replacing the standard convolutional layers of the LaneNet network with depthwise separable convolutions and replacing the transposed convolutions in the decoder part with bilinear interpolation. The BEV binary map is inferred using the lightweight modified LaneNet network, and the road boundary probability map and embedded feature map are output.

[0067] In some embodiments, the specific implementation process of step S207 above may include: The generated H×W The binary image is fed into a lightweight LaneNet optimized for edge computing: 1. Network Architecture: Maintain the classic Encoder-Decoder structure of LaneNet to extract multi-scale contextual features; 2. Lightweight modification: Depthwise Separable Convolution: Replaces standard convolutional layers with depthwise separable convolutions. The computational cost of standard convolution is... D 2 K · M·N·D 2 F Depth-separable convolution is D 2 K ·M·D 2 F + M· N·D 2 F (in D K The kernel size is [size]. M,N Number of input / output channels D F (This refers to the feature map size). This can reduce the computational cost and the number of parameters.

[0068] Channel reduction: Reduce the number of feature channels (Filters) in all layers to half that of the original network.

[0069] Efficient upsampling: In the decoder section, the traditional transposed convolution is abandoned, and bilinear interpolation is used for 2x upsampling. Bilinear interpolation has no learnable parameters, is extremely fast, and can produce smooth edges when processing binary geometry, avoiding the checkerboard effect common in transposed convolution; 3. Two-branch output: Binary Branch: Output H×W× A probability map of size 1, representing the probability that each pixel belongs to the road boundary (Foreground / Background); Embedding Branch: Output H × W × C (like C = 4) feature map. This branch is trained using Discriminative Loss to minimize the distance between pixels belonging to the same lane line in the embedding space and maximize the distance between pixels belonging to different lane lines.

[0070] Specifically, step S208 above may include the following steps: Based on the road boundary probability map, points with high confidence are selected as the foreground point set; The embedding feature vector of each pixel is concatenated with the physical space coordinates of that pixel on the BEV binary map, and used as the clustering input feature. The DBSCAN algorithm is used to perform clustering processing on the clustering input features, and the output is an instance segmentation map.

[0071] In some embodiments, the specific implementation process of step S208 above may include: This application embodiment utilizes network output for instance segmentation, clustering discrete pixels into independent road boundary lines. Specifically: 1. Threshold filtering: Based on the output of the Binary branch, select points with high confidence as the foreground point set; 2. Feature Concatenation: Traditional LaneNet only utilizes embedding features for clustering. This application proposes an enhancement strategy: concatenating the embedding feature vector of a pixel with its physical spatial coordinates on the BEV map. x,y The concatenation of these features serves as the input feature for clustering. F cluster =[Embed1 , Embed2 ,..., Embed C ,α·x,α·y ]; in, F cluster For feature set, α These are the weighting coefficients for spatial coordinates. This stitching method utilizes the prior knowledge that road boundaries are continuously distributed in space, making it less likely for spatially distant but similarly embedded noise points to be incorrectly clustered together; 3. DBSCAN Clustering: The DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is used to cluster the feature set. F cluster Perform clustering; The specific execution of the DBSCAN algorithm includes: based on the parameter: neighborhood radius. and minimum points MinPts ,calculate F cluster China The neighborhood density of points is used to identify core points that meet the density requirements. Then, based on density-reachability, the process recursively expands outward from the core points, merging density-connected pixels into the same road boundary instance. Finally, low-density outliers that do not belong to the neighborhood of any core point are automatically removed. This process can achieve effective aggregation of discontinuous boundaries without pre-setting the number of lane lines. Advantages: DBSCAN does not require pre-specifying the number of clusters (which is unknown in unstructured roads) and can effectively identify and remove noise points, making it very suitable for handling discontinuous and irregular road boundaries; 4. Instance Output: Output an instance map, where the value of each pixel represents the lane line ID to which it belongs.

[0072] Specifically, step S209 above may include the following steps: Iterate through each instance number, extract the BEV coordinates of all pixels belonging to that instance number, and transform them back to the vehicle physical coordinate system. A cubic polynomial model is used to solve for the transformed coordinate parameters, thereby obtaining the coefficients of the parameterized equations of the corresponding road boundaries, and then determining the parameterized equations of each road boundary.

[0073] In some embodiments, the specific implementation process of step S209 of this application is as follows: The discrete pixels are fitted into a mathematical curve for use by the planning and control module.

[0074] 1. Inverse coordinate transformation: Traverse each instance ID, extract the BEV coordinates of all pixels belonging to that ID, and transform them back to the vehicle physical coordinate system; 2. Curve fitting: A cubic polynomial model was used. y = ax 3 + bx 2 + cx + d Solve the coefficients using the least squares method. a, b, c, d For curves with large curvature, piecewise fitting or spline curves can also be used; 3. Output: The final output is the parametric equations for each road boundary, such as... Figure 3 As shown, the red and green curves represent the fitted left and right road boundaries, which are smooth and accurate.

[0075] In summary, compared with the prior art, this application has the following advantages: 1. This application abandons the direct use of raw point cloud coordinates or reflection intensity as network input, and instead constructs an intermediate representation, namely a BEV binary map based on height difference. This design allows the scheme to generate an unlimited amount of training data at low cost using simulators. The trained model can be directly and seamlessly transferred and deployed to the real physical world, with almost no Domain Shift / Sim-to-Real Gap, greatly reducing the threshold for algorithm deployment and data costs.

[0076] 2: When constructing BEV mesh features, this scheme uses the difference between the 5th percentile and the 95th percentile of the height, which has noise resistance and can filter out abnormal data at both ends, thereby enhancing robustness and significantly reducing the false detection rate caused by environmental noise.

[0077] 3. To address the computing power limitations of edge computing devices, this solution uses depthwise separable convolution. This method replaces standard convolutions with a single convolution, significantly reducing the number of parameters and compressing the overall network channel count to half of the original. In the decoder stage, bilinear interpolation upsampling is used instead of computed transposed convolutions. This level of network compression is achieved through a simpler intermediate representation designed in this application; something that existing image-based lane line networks and radar point cloud-based direct road edge detection methods struggle to do.

[0078] In summary, the road boundary curve recovery method based on BEV binary maps proposed in this application can be widely applied to autonomous special vehicles operating in unstructured environments without high-precision maps for various road and bridge construction projects.

[0079] Another aspect of this application embodiment provides a road boundary curve restoration device based on a BEV binary map, comprising: The first module is used to pre-configure sensing components on the target engineering machinery equipment and acquire point cloud data and image data of the environment in which the target engineering machinery equipment is located through the sensing components. The second module is used to perform spatiotemporal fusion on the point cloud data, transform the point cloud data to the vehicle center coordinate system, and perform reasoning and recognition on the image data to obtain dynamic objects and corresponding object information in the image. The third module is used to determine the state label of each point in the point cloud data based on the point cloud data obtained by fusion and the object information of the dynamic object. The fourth module is used to separate ground points and non-ground points based on point cloud data with status labels using the Patchwork algorithm; The fifth module is used to reduce the dimensionality of the separated point cloud and project it onto the bird's-eye view grid plane to construct the BEV feature grid; The sixth module is used to traverse each grid cell in the BEV feature grid, perform height difference feature extraction and binarization processing, and obtain a BEV binary map. The seventh module is used to input the BEV binary map into a lightweight LaneNet network for inference and output a road boundary probability map and an embedded feature map; wherein, the road boundary probability map is used to represent the probability that each pixel belongs to the road boundary; The eighth module is used to perform instance segmentation processing based on the road boundary probability map and the embedded feature map, clustering discrete pixels into independent road boundary lines to obtain an instance segmentation map, wherein the value of each pixel in the instance segmentation map represents the lane line label to which the pixel belongs; The ninth module is used to fit discrete pixels into mathematical curves and output the parametric equations for each road boundary.

[0080] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0081] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described road boundary curve recovery method based on BEV binary maps. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0082] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0083] Please see Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and called and executed by the processor 901 using the road boundary curve recovery method based on BEV binary maps according to the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0084] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described road boundary curve recovery method based on BEV binary maps.

[0085] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0086] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0088] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0089] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0090] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0092] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for road boundary curve recovery based on BEV binary maps, characterized in that, include: Sensing components are pre-configured on the target engineering machinery equipment, and point cloud data and image data of the environment in which the target engineering machinery equipment is located are acquired through the sensing components; The point cloud data is spatiotemporally fused, then transformed to the vehicle center coordinate system, and the image data is inferred and recognized to obtain the dynamic objects in the image and their corresponding object information. Based on the point cloud data obtained by fusion and the object information of the dynamic object, determine the state label of each point in the point cloud data; Based on point cloud data with status labels, ground points and non-ground points are separated using the Patchwork algorithm; The separated point cloud is dimensionality reduced and projected onto the bird's-eye view grid plane to construct the BEV feature grid; Traverse each grid cell in the BEV feature grid, perform height difference feature extraction and binarization to obtain a BEV binary map; The BEV binary image is input into a lightweight LaneNet network for inference, and the output is a road boundary probability map and an embedded feature map; wherein, the road boundary probability map is used to represent the probability that each pixel belongs to the road boundary; Based on the road boundary probability map and the embedded feature map, instance segmentation processing is performed to cluster discrete pixels into independent road boundary lines to obtain an instance segmentation map, wherein the value of each pixel in the instance segmentation map represents the lane line label to which the pixel belongs; The discrete pixels are fitted to a mathematical curve, and the parameterized equations for each road boundary are output.

2. The method for road boundary curve recovery based on BEV binary map according to claim 1, characterized in that, The step of projecting the separated point cloud into the bird's-eye view grid plane to construct the BEV feature grid includes: Define a two-dimensional matrix that covers the region of interest surrounding the target engineering machinery equipment; Iterate through each point in the point cloud and calculate the grid index to which the point belongs based on its coordinates; The grid cells to which all points belong are determined, and a list data structure is assigned to each grid cell. The list data structure is used to store the attributes of all points falling into the current grid cell, thereby constructing the BEV feature grid.

3. The method for road boundary curve recovery based on BEV binary map according to claim 1, characterized in that, The process of traversing each grid cell in the BEV feature grid, extracting height difference features, and binarizing the data to obtain a BEV binary map includes: Traverse each BEV grid cell, and for the height set within the BEV grid cell, calculate the height difference between two preset points in the height set; Candidate boundary grids are determined based on whether the height difference is within the preset road boundary height range; After removing points containing dynamic object labels within the candidate boundary grid, the remaining grid is assigned a value of 1 at the corresponding pixel position in the BEV image, and a value of 0 at the other pixel positions, thus obtaining a BEV binary image.

4. The method for road boundary curve recovery based on BEV binary map according to claim 1, characterized in that, The step of performing instance segmentation processing based on the road boundary probability map and the embedded feature map, clustering discrete pixels into independent road boundary lines to obtain an instance segmentation map, includes: Based on the road boundary probability map, points with high confidence are selected as the foreground point set; The embedding feature vector of each pixel is concatenated with the physical space coordinates of that pixel on the BEV binary map, and used as the clustering input feature. The DBSCAN algorithm is used to perform clustering processing on the clustering input features, and the output is an instance segmentation map.

5. The method for road boundary curve recovery based on BEV binary map according to claim 1, characterized in that, The step of separating ground points and non-ground points based on point cloud data carrying state labels using the Patchwork algorithm includes: The point cloud data carrying status labels is projected onto the polar coordinate system, and the space is divided into multiple concentric annular regions and sectors according to radial distance and azimuth angle; For each sector obtained by division, select several points with lower heights and filter out extreme underground noise points by combining the average Z value. The remaining points are used as initial ground seed points. The covariance matrix of these seed points is calculated by principal component analysis algorithm, the normal vector is extracted and the initial plane equation is fitted, and the fitted plane is determined. The core parameters of the fitted plane are extracted and thresholded. Points that pass the threshold check are identified as ground points, while those that fail are identified as non-ground points. The core parameters include the elevation of the plane, the verticality / normal vector of the plane, and the flatness of the plane.

6. The method for road boundary curve recovery based on BEV binary map according to claim 1, characterized in that, The step of inputting the BEV binary map into a lightweight LaneNet network for inference, and outputting a road boundary probability map and an embedded feature map, includes: The Encoder-Decoder structure of the LaneNet network is modified to be lightweight by replacing the standard convolutional layers of the LaneNet network with depthwise separable convolutions and replacing the transposed convolutions in the decoder part with bilinear interpolation. The BEV binary map is inferred using the lightweight modified LaneNet network, and the road boundary probability map and embedded feature map are output.

7. The method for road boundary curve recovery based on BEV binary map according to claim 1, characterized in that, The process of fitting discrete pixels to a mathematical curve and outputting the parameterized equation for each road boundary includes: Iterate through each instance number, extract the BEV coordinates of all pixels belonging to that instance number, and transform them back to the vehicle physical coordinate system. A cubic polynomial model is used to solve for the transformed coordinate parameters, thereby obtaining the coefficients of the parameterized equations of the corresponding road boundaries, and then determining the parameterized equations of each road boundary.

8. A road boundary curve recovery device based on BEV binary map, characterized in that, include: The first module is used to pre-configure sensing components on the target engineering machinery equipment and acquire point cloud data and image data of the environment in which the target engineering machinery equipment is located through the sensing components. The second module is used to perform spatiotemporal fusion on the point cloud data, transform the point cloud data to the vehicle center coordinate system, and perform reasoning and recognition on the image data to obtain dynamic objects and corresponding object information in the image. The third module is used to determine the state label of each point in the point cloud data based on the point cloud data obtained by fusion and the object information of the dynamic object. The fourth module is used to separate ground points and non-ground points based on point cloud data with status labels using the Patchwork algorithm; The fifth module is used to reduce the dimensionality of the separated point cloud and project it onto the bird's-eye view grid plane to construct the BEV feature grid; The sixth module is used to traverse each grid cell in the BEV feature grid, perform height difference feature extraction and binarization processing, and obtain a BEV binary map. The seventh module is used to input the BEV binary map into a lightweight LaneNet network for inference and output a road boundary probability map and an embedded feature map; wherein, the road boundary probability map is used to represent the probability that each pixel belongs to the road boundary; The eighth module is used to perform instance segmentation processing based on the road boundary probability map and the embedded feature map, clustering discrete pixels into independent road boundary lines to obtain an instance segmentation map, wherein the value of each pixel in the instance segmentation map represents the lane line label to which the pixel belongs; The ninth module is used to fit discrete pixels into mathematical curves and output the parametric equations for each road boundary.

9. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images