Lane marking methods, devices, electronic equipment and storage media

Abstract

This disclosure provides a lane marking method, apparatus, electronic device, and storage medium, relating to the field of artificial intelligence technology, and particularly to the fields of data processing, image processing, and autonomous driving. The specific implementation involves: displaying point cloud data collected on the road; responding to a selection operation on a target point in the point cloud data, determining a first lane line based on the target point; and adjusting the first lane line based on road surface information to obtain a second lane line. According to embodiments of this disclosure, the efficiency and accuracy of lane marking can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to the fields of data processing, image processing and autonomous driving. Background Technology

[0002] As point cloud technology matures, point cloud data is increasingly being used in the field of autonomous driving. To obtain lane line information on roads, it is necessary to accurately label the lane lines on the collected point cloud data. However, currently, the consistency and quality of lane line labeling results are poor, resulting in low labeling efficiency. Summary of the Invention

[0003] This disclosure provides a lane marking method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of this disclosure, a lane marking method is provided, comprising:

[0005] Displaying point cloud data collected on the road;

[0006] In response to the selection operation of the target point on the point cloud data, the first lane line is determined based on the target point;

[0007] The first lane line is adjusted based on the road surface information to obtain the second lane line.

[0008] According to another aspect of this disclosure, a lane marking device is provided, comprising:

[0009] The display module is used to display point cloud data collected on the road.

[0010] The lane line determination module is used to determine the first lane line based on the target point in response to the selection operation of the target point in the point cloud data;

[0011] The lane line adjustment module is used to adjust the first lane line based on road surface information to obtain the second lane line.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] The memory is communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0017] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0018] The technical solution of this disclosure allows users to quickly mark lane lines by selecting target points in the displayed point cloud data, while ensuring that the lane lines fit the road surface, thus improving the efficiency and accuracy of lane line marking.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1 This is a schematic flowchart of a lane marking method provided in an embodiment of this disclosure;

[0022] Figure 2 This is a schematic diagram illustrating an application example of an embodiment of this disclosure;

[0023] Figure 3 This is a schematic block diagram of a lane marking device provided in one embodiment of the present disclosure;

[0024] Figure 4 This is a schematic block diagram of a lane marking device provided in another embodiment of this disclosure;

[0025] Figure 5 This is a schematic block diagram of a lane marking device provided in another embodiment of this disclosure;

[0026] Figure 6 This is a block diagram of an electronic device used to implement the lane marking method of the embodiments of this disclosure. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] Figure 1 This disclosure illustrates a lane marking method according to an embodiment. This method can be applied to a lane marking device, which can be deployed in an electronic device. The electronic device can be, for example, a single or multi-device terminal, server, or other processing device. The terminal can be a mobile device, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or other user equipment (UE); the server can be a single-device server or a server cluster. In some possible implementations, the method can also be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the method may include the following steps S110 to S130.

[0029] Step S110: Display the point cloud data collected on the road.

[0030] Optionally, point cloud data may include 4D (4-Dimension) point clouds. A 4D point cloud can be understood as a set of a large number of points in a three-dimensional coordinate system, with the addition of a time dimension. In other words, point cloud data includes multiple frames of 3D (3-Dimension) point clouds over a continuous time period. An exemplary scenario is that a vehicle can obtain multiple frames of 3D point clouds (i.e., point cloud data) continuously using onboard radar while driving on the road.

[0031] For example, point cloud data collected on a road may include point clouds corresponding to lane lines, point clouds corresponding to the road surface, and point clouds corresponding to obstacles.

[0032] Optionally, the point cloud data can be displayed by displaying each frame of 3D point cloud in the 4D point cloud separately. For example, for each frame of 3D point cloud, the 3D point cloud can be projected onto a specific plane and the planar projection map can be displayed on the interactive interface; or, each frame of 3D point cloud can be displayed in the 3D display space on the interactive interface.

[0033] Optionally, an algorithm can be used to identify the category of each point in the point cloud data, and different display styles can be used for points of different categories.

[0034] Step S120: In response to the selection operation for the target point on the point cloud data, determine the first lane line based on the target point.

[0035] In this embodiment of the disclosure, the user can find the position of the lane line by observing the point cloud data, regard the points on the lane line as target points, and complete the selection operation for the target points in the interactive interface. The lane line marking device can read the target points selected by the user and determine the first lane line based on the target points.

[0036] In this embodiment of the disclosure, there can be multiple target points. The first lane line can be determined based on multiple target points. For example, the smallest bounding graph containing multiple target points can be determined as the first lane line, or the lane line can be determined by fitting.

[0037] In some practical application examples, the interactive interface can provide a variety of functions. Users can select the line drawing function on the interactive interface, enter the sub-interface corresponding to the function, and mark points on the point cloud data corresponding to the lane line. The lane line marking device can automatically generate the first lane line based on the user's markings.

[0038] Step S130: Adjust the first lane line based on the road surface information to obtain the second lane line.

[0039] In this embodiment of the disclosure, the road surface information may include the position information of the road surface point cloud in the point cloud data. Optionally, the height of the first lane line can be adjusted using the road surface information to ensure that the second lane line conforms to the road surface.

[0040] Optionally, the point cloud data can be preprocessed (e.g., semantic segmentation) based on an algorithm to obtain road regions in the point cloud data, and road segmentation markers can be added to these road regions. If the point cloud data contains preprocessed road segmentation markers, the point cloud with these markers is used as the point cloud corresponding to the road surface, and the first lane line is attached to the point cloud corresponding to the road surface.

[0041] In some practical applications, the user interface can be set to automatically adjust the line drawing location, allowing the user to choose to enable it or enabling it by default. When the line drawing location is enabled, drawing target points on the line using the line drawing function will automatically call the algorithm function, returning the road surface information predicted by the algorithm. After determining the first lane line based on the target points, the first lane line can be fitted to the road surface based on the road surface information.

[0042] In some embodiments, a user can observe point cloud data and / or the corresponding simulated image to identify lane line attributes (e.g., solid / phasic, color, etc.) and select or input lane line attributes in an interactive interface. Based on these lane line attributes, the attributes of the second lane line can be determined. Alternatively, the lane line marking device can identify the point cloud data and / or the corresponding simulated image to determine the lane line attributes, and thus determine the attributes of the second lane line.

[0043] In related technologies, lane line annotation in 4D point cloud relies on individual experience and skill levels, resulting in poor consistency and quality of annotation results, low efficiency, high labor costs, unsuitability for large-scale data processing, and difficulty in handling complex or fuzzy point cloud data. To address at least one of these problems, this disclosure embodiment allows users to generate annotated lane lines simply by selecting target points. According to the method described above, users can quickly annotate lane lines through a simple interface, ensuring that the lane lines conform to the road surface. This improves the efficiency and accuracy of lane line annotation, providing more reliable road information for autonomous driving assistance systems and contributing to enhanced safety and reliability.

[0044] Figure 2 A schematic diagram illustrating an application example of the lane marking method in an embodiment of this disclosure is shown. For example... Figure 2 As shown, a point cloud acquisition device (e.g., radar) configured on vehicle 21 collects point cloud data on the road. The point cloud data is transmitted or input to lane marking device 22, which provides a user interface with lane drawing functionality. This interface displays the point cloud data to the user and detects the user's selection of target points. For example, the interface provides a drawing tool, allowing the user to drag the brush to select target points. A large number of points within the brush's coverage area become target points, forming the first lane line. If the user activates the automatic road surface adjustment switch, the first lane line is adjusted based on road information. For example, the height information of each point in the first lane line is adjusted to match the height information of the corresponding points on the road surface, resulting in a second lane line. This second lane line is a more accurate marking result that conforms to the road surface. The relevant information of this second lane line is transmitted or input to other electronic devices, such as server 23, to provide lane line information to assist autonomous driving decisions or as sample data to train a high-precision lane line recognition model.

[0045] In some embodiments, the point cloud data collected on the road is displayed, including:

[0046] A top-down view of the point cloud is determined based on point cloud data collected on the road.

[0047] Based on a preset color adjustment method, the point cloud top view is color-adjusted and then the point cloud data is displayed by showing the point cloud top view; the color adjustment method is used to highlight the lines on the point cloud top view.

[0048] In this embodiment of the disclosure, the point cloud top view can be a two-dimensional image formed by projecting point cloud data in three-dimensional space onto a top-view plane. In some examples of practical applications, point cloud data can be projected onto multiple planes to obtain multiple views corresponding to the point cloud data, providing different display perspectives. Users can select the display perspective of the point cloud data to display the point cloud top view on the interactive interface. Using the point cloud top view to present point cloud data allows for a direct and intuitive presentation of lane lines from a top-down perspective, avoiding the influence of point cloud height information on the user's observation and judgment of lane lines.

[0049] Optionally, preset color adjustment methods may include sharpening the point cloud top view, adjusting contrast, adjusting hue and saturation, and eliminating shadows. Optionally, multiple preset color adjustment methods can be provided in the interactive interface for users to choose from. The color of the point cloud top view is adjusted according to the user's selected color adjustment method to highlight the lines on the point cloud top view, so that users can accurately observe and find the lane lines on the point cloud top view, which is convenient for selecting target points on the lane lines later.

[0050] Optionally, users can trigger preset color adjustment methods multiple times. Each color adjustment will result in a different display effect. Users can trigger color adjustments multiple times until they accurately find the lane lines on the point cloud top view.

[0051] According to the above embodiments, by adjusting the color of the point cloud top view, the lines on the point cloud top view are highlighted, so that users can accurately observe and find the lane lines on the point cloud top view, improve the user experience, and further improve the efficiency and accuracy of lane line marking.

[0052] In some embodiments, road surface information includes a ground model;

[0053] Lane marking methods also include:

[0054] Based on the target point, the initial ground point cloud is determined from the point cloud data;

[0055] The ground model is obtained by fitting the initialized ground point cloud.

[0056] In this embodiment of the disclosure, all points in the point cloud data can be coarsely filtered based on the location of the target point, filtering out points far from the target point. Optionally, points that are significantly different from the ground height can also be filtered out, thereby determining the initial ground point cloud. For example, the point cloud near the target point can be determined as the initial ground point cloud in the point cloud data.

[0057] In this embodiment of the disclosure, a planar model can be fitted from a random subset of sample points in an initialized ground point cloud, and the fitted planar model can be regarded as a ground model. A ground model can be understood as a mathematical model used to represent the ground, and it can be converted into a plane in a coordinate system.

[0058] Optionally, after determining the first lane line based on the target point, the height information (z coordinate) of each point can be re-determined based on the planar coordinates (e.g., x and y coordinates) of each point in the first lane line and the ground model, thereby adjusting the height of the first lane line to obtain the second lane line so that the second lane line can fit the ground model.

[0059] In one example, the standard deviation can be calculated based on the distance between each point in the ground point cloud and the plane corresponding to the ground model. If the standard deviation meets the requirements (i.e., the standard deviation is small), the planar coordinates of each point in the first lane line can be substituted into the ground model to obtain the predicted height close to the ground.

[0060] According to the above embodiments, a ground model with high fitting accuracy and robustness can be obtained. Therefore, adjusting the height of the first lane line based on the ground model can further improve the efficiency and accuracy of lane line labeling.

[0061] Optionally, when the standard deviation is large, a cylindrical space surrounding the target point can be determined in the point cloud data. The cylindrical space can be divided into multiple buckets according to the height direction to obtain multiple bucket windows. The number of points taken from low height to high height exceeds a threshold in the bucket window. The lowest point in the bucket window is used as the predicted height for ground contact. The first lane line is adjusted based on the predicted height so that the second lane line is ground contact.

[0062] In some embodiments, a ground model is obtained by fitting an initialized ground point cloud, including:

[0063] Based on the initialized ground point cloud, N iterations are performed to obtain the ground model; where N is an integer not less than 2, and the i-th iteration in the N iterations includes:

[0064] Determine the i-th ground point cloud; where, when i equals 1, the i-th ground point cloud is point cloud data randomly sampled from the initialized ground point cloud; when i is an integer not less than 2 and not greater than N, the i-th ground point cloud is obtained by filtering from the i-1 ground point cloud based on the distance between the points in the i-1 ground point cloud and the i-1 planar model.

[0065] The i-th ground point cloud is fitted to obtain the i-th planar model;

[0066] If the i-th planar model meets the preset conditions, the i-th planar model is determined as the ground model; where i is an integer not less than 2 and not greater than N.

[0067] In this embodiment of the disclosure, a portion of points are randomly sampled from the initialized ground point cloud to form a first ground point cloud. A first planar model is obtained by fitting the first ground point cloud. If the first planar model meets preset conditions, it is used as the ground model. If it does not meet the conditions, a portion of points close to the first ground model are selected from the first ground point cloud to obtain a second ground point cloud. A second planar model is obtained by fitting the second point cloud, and then it is determined whether it meets the conditions. This process is repeated until a planar model that meets the conditions is obtained.

[0068] Optionally, the method for selecting the i-th ground point cloud from the (i-1)-th ground point cloud may include: calculating the residual of each point in the (i-1)-th ground point cloud based on the distance between each point in the (i-1)-th ground point cloud and the (i-1)-th planar model, determining whether each point belongs to the in-plane point based on the residual (e.g., points with residuals less than a threshold are in-plane points), and obtaining the i-th ground point cloud based on the in-plane points.

[0069] Optionally, in the i-th iteration, some points can be randomly sampled from the (i-1)-th planar point cloud or the initialized planar point cloud, and the union of these points with the points in the aforementioned plane can be obtained to get the i-th ground point cloud.

[0070] According to the above embodiments, the fitting accuracy and robustness of the ground model can be improved through multiple iterative operations.

[0071] In one example, the ground model can be determined using RANSAC (Random Sample Consensus), which may include:

[0072] Step 1: Within the maximum number of iterations (maxIterations), perform the following calculations to update the planar model in each round:

[0073] Step 11: Randomly sample a portion of points from the initialized ground point cloud to form the ground point cloud (sampleIndices);

[0074] Step 12: Fit a planar model using randomly sampled points;

[0075] Step 13: Calculate the residuals based on the distances from all points in the ground point cloud to the planar model. Points within the threshold are considered to be points in the current planar model.

[0076] Step 14: Compare whether the current model is better, update the plane model parameters and inliers.

[0077] Step 2: Output the optimal linear model parameters, and use the optimal inlier as the ground point result (i.e., the ground model).

[0078] In some embodiments, determining an initialized ground point cloud from point cloud data based on a target point includes:

[0079] Based on the target point, determine the cylindrical space surrounding the target point in the point cloud data;

[0080] Divide the cylindrical space into buckets along the height direction, and determine M bucket windows, where M is an integer not less than 2;

[0081] Determine the target bucket window based on the number of points within the bucket window;

[0082] The initial ground point cloud is determined based on multiple points within the target bucket window.

[0083] Optionally, the cylindrical space can be the smallest cylindrical space that encloses all target points. Alternatively, a reasonable radius value can be calculated based on the position of each target point, or a radius value can be preset. Based on the center of all target points and this radius value, the cylindrical space can be determined, and this cylindrical space can enclose at least some of the target points.

[0084] In this embodiment, the point clouds contained in different bucket windows can be non-overlapping or partially overlapping. For example, M bucket windows can be determined by sliding windows. Here, the sliding window method is, for example, determining the first bucket window, sliding upwards a certain distance to obtain the second bucket window, and so on to determine M bucket windows. Specifically, the number of bucket windows can be preset by the user, or it can be the ground thickness / bucket height. The ground thickness can be determined based on the distance between the highest and lowest points in the cylindrical space, and the bucket height (i.e., the distance slid upwards when bucketing) can be set by the user.

[0085] In one example, when the real-world ground environment is relatively simple, we can iterate through the number of points in each of the M bucket windows and determine the bucket window with the most points among the M bucket windows, which is the target bucket window.

[0086] In one example, determining the target bucket can include:

[0087] Step 1: Determine the points within the radius of the plane surrounding the target point, that is, the points within a cylindrical space.

[0088] Step 2: Divide the cylinder into bins according to its height, resulting in multiple bin windows. The number of bins can be calculated as ground thickness / bin height (ground_height / height_bin_size). The ground thickness can be determined based on the distance between the highest and lowest points within the cylindrical space, and the bin height can be set by the user.

[0089] Step 3: For normal ground point clouds (i.e., the real ground environment is relatively simple), start from the lowest height bucket window, slide the bucket windows upwards, and find the window that contains the largest number of points.

[0090] According to the above embodiments, by using a bucketing strategy to perform coarse filtering on all points in the point cloud data, point clouds far from the road surface can be accurately filtered out, reducing the number of subsequent fitted point clouds, thereby improving the efficiency and accuracy of fitted point clouds, while reducing the computational requirements.

[0091] In this disclosure, various methods for determining target bucket windows are also provided, and exemplary methods are provided below.

[0092] Method 1:

[0093] Based on the number of points within the bucket window, determine the target bucket window, including:

[0094] When the ratio between the number of points in the j-th bucket window out of M bucket windows and the number of points in the cylindrical space is greater than a first threshold, the j-th bucket window is determined as the target bucket window, where j is a positive integer not greater than M.

[0095] For example, when the real-world ground environment is complex (e.g., the point cloud data includes underground reflection points, roadside points, vegetation points, etc.), starting from the lowest-height bucket window, the point cloud quantity ratio is determined based on the ratio of the number of points in the first bucket window to the number of points in the cylindrical space. If the point cloud quantity ratio is less than a first threshold, the bucket window can be slid upwards, and the point cloud quantity ratio is determined based on the ratio of the number of points in the second bucket window to the number of points in the cylindrical space, and so on, until the point cloud quantity ratio corresponding to the j-th bucket window is greater than the first threshold. At this point, the j-th bucket window can be pre-determined as the target bucket window. That is, without traversing every bucket window in the M bucket windows, the bucket window corresponding to the smallest point cloud quantity ratio greater than the first threshold can be determined as the target bucket window.

[0096] According to the above embodiments, while accurately determining the target bucket window, terminating the determination of the target bucket window in advance, compared to traversing the number of points in each bucket window, can improve efficiency and reduce the requirements for computing power.

[0097] Method 2:

[0098] Based on the number of points within the bucket window, determine the target bucket window, including:

[0099] When the ratio between the number of points in the k-th bucket window and the maximum number of points in the maximum window is greater than the second threshold, the k-th bucket window is determined as the target bucket window; where the maximum number of points in the maximum window is the maximum number of points from the 1-th bucket window to the (k-1)-th bucket window in the M bucket windows, and k is a positive integer not greater than M.

[0100] In this embodiment, when the local ground environment is complex (e.g., the point cloud data includes underground reflection points, roadside points, vegetation points, etc.), starting from the lowest-height bucket window, a window growth rate threshold (min_gradient_rate) is determined based on the ratio of the number of points in the second bucket window to the number of points in the first bucket window. If the window growth rate threshold is less than the second threshold, the bucket window can be slid upwards. A window growth rate threshold is then determined based on the ratio of the number of points in the third bucket window to the bucket window with the largest number of points in both the first and second bucket windows. This process continues until the ratio of the number of points in the k-th bucket window to the maximum number of points in the largest window exceeds the second threshold. This indicates that the k-th bucket window has encountered a peak, and it can be pre-determined as the target bucket window. In other words, it is possible to determine the bucket window with a growth rate threshold greater than the second threshold as the target bucket window without traversing every single one of the M bucket windows.

[0101] According to the above embodiments, while accurately determining the target bucket window, the determination of the target bucket window is terminated in advance compared to traversing the number of points in each bucket window, thus reducing the computational requirements.

[0102] As can be seen, the lane marking method provided in this disclosure significantly improves the efficiency and accuracy of lane marking, minimizes the computational requirements, and optimizes the user experience. The lane marking method of this disclosure provides more reliable road information for autonomous driving assistance systems, helping to improve the system's safety and reliability.

[0103] According to embodiments of this disclosure, this disclosure also provides a lane marking device. Figure 3 A schematic block diagram of a lane marking device according to an embodiment of the present disclosure is shown, such as Figure 3 As shown, the device includes:

[0104] Display module 310 is used to display point cloud data collected on the road;

[0105] Lane line determination module 320 is used to determine a first lane line based on the target point in response to a selection operation on the point cloud data for a target point;

[0106] The lane line adjustment module 330 is used to adjust the first lane line based on road surface information to obtain the second lane line.

[0107] In some embodiments, road surface information includes a ground model;

[0108] like Figure 4 As shown, the lane marking device may also include:

[0109] The point cloud determination module 420 is used to determine the initial ground point cloud from the point cloud data based on the target point;

[0110] The point cloud processing module 430 is used to fit the initialized ground point cloud to obtain a ground model.

[0111] In some embodiments, the point cloud processing module 430 is used for:

[0112] Based on the initialized ground point cloud, N iterations are performed to obtain the ground model; where N is an integer not less than 2, and the i-th iteration in the N iterations includes:

[0113] Determine the i-th ground point cloud; where, when i equals 1, the i-th ground point cloud is point cloud data randomly sampled from the initialized ground point cloud; when i is an integer not less than 2 and not greater than N, the i-th ground point cloud is obtained by filtering from the i-1 ground point cloud based on the distance between the points in the i-1 ground point cloud and the i-1 planar model.

[0114] The i-th ground point cloud is fitted to obtain the i-th planar model;

[0115] If the i-th planar model meets the preset conditions, the i-th planar model is determined as the ground model; where i is an integer not less than 2 and not greater than N.

[0116] In some embodiments, such as Figure 5 As shown, the point cloud determination module 420 includes:

[0117] The spatial determination submodule 510 is used to determine the cylindrical space surrounding the target point in the point cloud data based on the target point;

[0118] The bucket submodule 520 is used to divide the cylindrical space into buckets in the height direction, and to determine M bucket windows, where M is an integer not less than 2;

[0119] The window determination submodule 530 is used to determine the target bucket window based on the number of points in the bucket window;

[0120] The point cloud determination submodule 540 is used to determine the initial ground point cloud based on multiple points within the target bucket window.

[0121] In some embodiments, the window determining submodule 530 is configured to:

[0122] When the ratio between the number of points in the j-th bucket window out of M bucket windows and the number of points in the cylindrical space is greater than a first threshold, the j-th bucket window is determined as the target bucket window, where j is a positive integer not greater than M.

[0123] In some embodiments, the window determining submodule 530 is configured to:

[0124] When the ratio between the number of points in the k-th bucket window and the maximum number of points in the maximum window is greater than the second threshold, the k-th bucket window is determined as the target bucket window; where the maximum number of points in the maximum window is the maximum number of points from the 1-th bucket window to the (k-1)-th bucket window in the M bucket windows, and k is a positive integer not greater than M.

[0125] In some embodiments, the display module 310 is configured to:

[0126] A top-down view of the point cloud is determined based on point cloud data collected on the road.

[0127] Based on a preset color adjustment method, the point cloud top view is color-adjusted and then the point cloud data is displayed by showing the point cloud top view; the color adjustment method is used to highlight the lines on the point cloud top view.

[0128] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0129] The acquisition, storage, and application of personal information of the target object involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0130] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0131] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0132] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 607 into random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 804. An input / output (I / O) interface 605 is also connected to bus 604.

[0133] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0134] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the lane marking method. For example, in some embodiments, the lane marking method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the lane marking method described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the lane marking method by any other suitable means (e.g., by means of firmware).

[0135] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0139] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0140] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0141] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0142] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for marking lane lines, comprising: Use a top-down view of the point cloud to display the point cloud data collected on the road; In response to a selection operation for a target point on the point cloud data, a first lane line is determined based on the target point; Based on the target point, a cylindrical space surrounding the target point is determined in the point cloud data; The cylindrical space is divided into buckets along the height direction, and M bucket windows are determined, where M is an integer not less than 2; The target bucket window is determined based on the number of points within the bucket window; Based on multiple points within the target bucket window, determine the initialized ground point cloud; A ground model is obtained by fitting the initialized ground point cloud; The standard deviation is calculated based on the distance between each point in the ground point cloud and the plane corresponding to the ground model. If the standard deviation meets the requirements, the plane xy coordinates of each point in the first lane line are substituted into the ground model to obtain the predicted height close to the ground, which is used as the height information z coordinate of each point in the first lane line to obtain the second lane line. If the standard deviation does not meet the requirements, a cylindrical space surrounding the target point is determined in the point cloud data. The cylindrical space is then divided into multiple buckets according to the height direction to obtain multiple bucket windows. For bucket windows where the number of points taken from low to high height exceeds a threshold, the lowest point in the bucket window is taken as the predicted height close to the ground. Based on this predicted height, the first lane line is adjusted to obtain the second lane line close to the ground.

2. The method according to claim 1, wherein, The process of fitting the initialized ground point cloud to obtain a ground model includes: Based on the initialized ground point cloud, N iterations are performed to obtain the ground model; where N is an integer not less than 2, and the i-th iteration in the N iterations includes: Determine the i-th ground point cloud; where, when i equals 1, the i-th ground point cloud is point cloud data randomly sampled from the initialized ground point cloud; when i is an integer not less than 2 and not greater than N, the i-th ground point cloud is obtained by filtering from the (i-1)-th ground point cloud based on the distance between the points in the (i-1)-th ground point cloud and the (i-1)-th planar model. The i-th ground point cloud is fitted to obtain the i-th planar model; If the i-th planar model meets the preset conditions, the i-th planar model is determined as the ground model; where i is an integer not less than 2 and not greater than N.

3. The method according to claim 1, wherein, Determining the target bucket window based on the number of points within the bucket window includes: When the ratio between the number of points in the j-th bucket window and the number of points in the cylindrical space is greater than a first threshold, the j-th bucket window is determined as the target bucket window, where j is a positive integer not greater than M.

4. The method according to claim 1, wherein, Determining the target bucket window based on the number of points within the bucket window includes: When the ratio between the number of points in the kth bucket window and the maximum number of points in the maximum window is greater than a second threshold, the kth bucket window is determined as the target bucket window; wherein, the maximum number of points in the maximum window is the maximum number of points from the 1st bucket window to the (k-1th bucket window) of the M bucket windows, and k is a positive integer not greater than M.

5. The method according to any one of claims 1-4, wherein, The point cloud data displayed on the road includes: A top-down view of the point cloud is determined based on point cloud data collected on the road. Based on a preset color adjustment method, the point cloud top view is color-adjusted, and the point cloud data is displayed by displaying the point cloud top view; wherein, the color adjustment method is used to highlight the lines on the point cloud top view.

6. A lane marking device, comprising: The display module is used to display point cloud data collected on the road using a top-down view of the point cloud; A lane line determination module is used to determine a first lane line based on the target point in response to a selection operation for a target point in the point cloud data. The point cloud determination module is used to: determine a cylindrical space surrounding the target point in the point cloud data based on the target point; divide the cylindrical space into buckets in the height direction to determine M bucket windows, where M is an integer not less than 2; determine a target bucket window based on the number of points in the bucket window; and determine an initialized ground point cloud based on multiple points in the target bucket window. The point cloud processing module is used to fit the initialized ground point cloud to obtain a ground model; The lane line adjustment module is used to: calculate the standard deviation based on the distance between each point in the ground point cloud and the plane corresponding to the ground model; if the standard deviation meets the requirements, substitute the plane xy coordinates of each point in the first lane line into the ground model to obtain the predicted height close to the ground, which is used as the height information z coordinate of each point in the first lane line to obtain the second lane line; if the standard deviation does not meet the requirements, determine the cylindrical space surrounding the target point in the point cloud data, divide the cylindrical space into buckets according to the height direction to obtain multiple bucket windows, select bucket windows from low height to high height where the number of points exceeds a threshold, use the lowest point in the bucket window as the predicted height close to the ground, and adjust the first lane line based on the predicted height to obtain the second lane line close to the ground.

7. The apparatus according to claim 6, wherein, The point cloud processing module is used for: Based on the initialized ground point cloud, N iterations are performed to obtain the ground model; where N is an integer not less than 2, and the i-th iteration in the N iterations includes: Determine the i-th ground point cloud; where, when i equals 1, the i-th ground point cloud is point cloud data randomly sampled from the initialized ground point cloud; when i is an integer not less than 2 and not greater than N, the i-th ground point cloud is obtained by filtering from the (i-1)-th ground point cloud based on the distance between the points in the (i-1)-th ground point cloud and the (i-1)-th planar model. The i-th ground point cloud is fitted to obtain the i-th planar model; If the i-th planar model meets the preset conditions, the i-th planar model is determined as the ground model; where i is an integer not less than 2 and not greater than N.

8. The apparatus according to claim 6, wherein, The window determination submodule is used for: When the ratio between the number of points in the j-th bucket window and the number of points in the cylindrical space is greater than a first threshold, the j-th bucket window is determined as the target bucket window, where j is a positive integer not greater than M.

9. The apparatus according to claim 6, wherein, The window determination submodule is used for: When the ratio between the number of points in the kth bucket window and the maximum number of points in the maximum window is greater than a second threshold, the kth bucket window is determined as the target bucket window; wherein, the maximum number of points in the maximum window is the maximum number of points from the 1st bucket window to the (k-1th bucket window) of the M bucket windows, and k is a positive integer not greater than M.

10. The apparatus according to any one of claims 6-9, wherein, The display module is used for: A top-down view of the point cloud is determined based on point cloud data collected on the road. Based on a preset color adjustment method, the point cloud top view is color-adjusted, and the point cloud data is displayed by displaying the point cloud top view; wherein, the color adjustment method is used to highlight the lines on the point cloud top view.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images