Lane line acquisition method and related device

Abstract

Embodiments of the present application provide a lane line acquisition method and a related device. The method may be used in the field of intelligent driving. The method comprises: acquiring coordinate information corresponding to a lane line in a road environment and point cloud data of the road environment, the lane line comprising a lane center line and / or a lane boundary line, and the coordinate information comprising coordinates of a position point on the lane line; and further, obtaining an updated lane line by means of a machine learning model on the basis of the coordinate information and the point cloud data. In a process of generating a lane line in a road environment, coordinate information of a position point on the lane line is used, and point cloud data of the road environment is also used, which is beneficial to improving the accuracy of the obtained updated lane line.

Description

A method for acquiring lane lines and related equipment

[0001] This application claims priority to Chinese Patent Application No. 202411550049.4, filed with the State Intellectual Property Office of China on October 31, 2024, entitled "A method for obtaining lane lines and related equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to intelligent driving technology, and more particularly to a method for acquiring lane lines and related equipment. Background Technology

[0003] With the rapid development of intelligent driving technology, lane lines are needed in more and more scenarios. For example, lane center lines and / or lane boundary lines are needed when building map information. Also, intelligent driving systems in vehicles need to use lane center lines when assisting driving. Therefore, a solution for obtaining lane lines is urgently needed. Summary of the Invention

[0004] This application provides a method and related equipment for obtaining lane lines, and provides a scheme for obtaining lane lines; in addition, in the process of generating lane lines in a certain road environment, not only the coordinate information of the position points on the lane lines are used, but also the point cloud data of the road environment is used, which helps to improve the accuracy of the updated lane lines.

[0005] This application provides the following technical solution:

[0006] Firstly, this application provides a method for obtaining lane lines, which can be used in the field of intelligent driving. In this method, an execution device can acquire at least one set of coordinate information corresponding to lane lines in a road environment. For example, the lane lines in this application may include lane center lines and / or lane boundary lines. Each set of coordinate information in the aforementioned at least one set of coordinate information may include the first coordinates of at least one location point on the lane line. The execution device can also acquire at least one point cloud data of the road environment. For example, each point cloud data of the road environment may include the second coordinates of at least one location point within the road environment. Furthermore, based on the at least one set of coordinate information corresponding to lane lines in the road environment and the at least one point cloud data of the road environment, the execution device can obtain each updated lane line in at least one updated lane line within the road environment through a machine learning model.

[0007] For example, if the lane lines only include lane center lines (or lane boundary lines), then at least one updated lane line may include at least one set of sequentially arranged coordinates corresponding one-to-one with at least one first updated lane center line (or updated lane boundary line). That is, each updated lane line may include a set of sequentially arranged coordinates corresponding to a first updated lane center line (or updated lane boundary line), and each set of sequentially arranged coordinates includes at least two sequentially arranged coordinates. The line connecting the aforementioned at least two coordinates may represent a first updated lane center line (or updated lane boundary line), the first coordinate in the aforementioned sequentially arranged at least two coordinates may indicate the starting point of a first updated lane center line (or updated lane boundary line), and the last coordinate in the sequentially arranged at least two coordinates may indicate the ending point of a first updated lane center line (or updated lane boundary line).

[0008] For example, the order of at least two coordinates in each sequence can indicate the direction of the updated lane line, which can be from the first coordinate among the aforementioned at least two coordinates to the last coordinate among the aforementioned at least two coordinates.

[0009] Optionally, each updated lane line may include not only a set of sequentially arranged coordinates, but also a group identifier (ID) corresponding to the aforementioned set of sequentially arranged coordinates. Different group identifiers are used to distinguish between different sets of sequentially arranged coordinates.

[0010] If the lane lines include lane center lines and lane boundary lines, each updated lane line not only includes a set of sequentially arranged coordinates corresponding to the updated lane line, but may also include the category of the updated lane line. For example, the category of the updated lane line can be lane center line or lane boundary line, etc.

[0011] In this implementation, the coordinate information corresponding to the lane lines in the road environment and the point cloud data of the road environment are obtained. The aforementioned coordinate information includes the coordinate information of the position points on the lane lines. Then, based on the aforementioned coordinate information and the point cloud data, the updated lane lines are obtained through a machine learning model, providing a solution for obtaining updated lane lines. In addition, in the process of generating lane lines in a certain road environment, not only the coordinate information of the position points on the lane lines is used, but also the point cloud data of the road environment is used, which helps to improve the accuracy of the obtained updated lane lines.

[0012] In one possible implementation, the execution device obtains an updated lane line using a machine learning model based on at least one set of coordinate information corresponding to lane lines in the road environment and at least one point cloud data of the road environment. This may include: the execution device performing a projection operation based on the aforementioned at least one set of coordinate information to obtain a first image. In other words, the execution device may project all position points indicated by the at least one set of coordinate information onto the first image based on all first coordinates (or updated first coordinates) included in the at least one set of coordinate information. The first image may include multiple position points on the lane line within the road environment. The execution device also performs a projection operation based on the aforementioned at least one point cloud data to obtain a second image. In other words, the execution device may project all position points indicated by at least one point cloud data onto the second image based on all second coordinates (or updated second coordinates) included in at least one point cloud data. The position points in the second image correspond to the position points in the point cloud data, that is, the position points in the point cloud data are projected from three dimensions to two dimensions to obtain the position points in the second image. The execution device then inputs the first image and the second image into the machine learning model, and processes the first image and the second image through the first machine learning model to obtain the updated lane line.

[0013] In this implementation, a projection operation is performed on at least one set of first coordinate information corresponding to lane lines in the road environment to obtain a first image, and a projection operation is performed on at least one first point cloud data of the road environment to obtain a second image. The first image and the second image are then input into a first machine learning model, and the updated lane lines are obtained through the first machine learning model. Since at least one set of first coordinate information and at least one first point cloud data are reduced to the form of images, the computer resources consumed by the processing of the first machine learning model are reduced, and the speed of obtaining the updated lane lines is improved.

[0014] In one possible implementation, the machine learning model includes a feature extraction module and a feature processing module. The feature extraction module processes the aforementioned at least one first feature map to generate at least one updated lane line. Exemplarily, the feature processing module obtains at least one second feature map based on the aforementioned at least one first feature map, and then generates at least one updated lane line in the road environment based on the aforementioned at least one second feature map. Optionally, the size of each second feature map in the at least one second feature map can be equal to the size of the first image, and the size of the feature map used by the feature processing module in the machine learning model to generate at least one updated lane line based on the at least one second feature map remains unchanged.

[0015] In this implementation, the updated lane lines generated by the first machine learning model are described using the size of the first image. Since the size of the second feature map is equal to the size of the first image, and the size of the feature map used in generating the updated lane lines based on the second feature map remains unchanged, the updated lane lines are predicted based on a feature map with the same size as the first image. If the updated lane lines are predicted based on a feature map with a size smaller than the first image, then after generating the updated lane lines, they need to be upsampled to the size of the first image. If the updated lane lines are predicted based on a feature map with a size larger than the first image, then after generating the updated lane lines, they need to be downsampled to the size of the first image. Both the aforementioned upsampling and downsampling may further amplify the errors carried in the predicted updated lane lines. However, in this application, the updated lane lines are predicted based on a feature map with the same size as the first image, eliminating the need for upsampling and downsampling operations, which is beneficial for obtaining more accurate updated lane lines.

[0016] In one possible implementation, the feature processing module of the machine learning model includes at least one neural network layer (hereinafter referred to as the "first neural network layer" for convenience), which is used in the process of generating updated lane lines based on at least one second feature map. The first neural network layer is used to keep the size of the feature map unchanged and reduce the number of feature maps.

[0017] For example, the feature processing module in the first machine learning model can directly determine at least one first feature map as at least one second feature map; or, the feature processing module in the first machine learning model can also process at least one first feature map to obtain at least one second feature map, etc., which can be determined according to the actual application scenario.

[0018] Optionally, the feature processing module of the first machine learning model further includes at least one second neural network layer, which is used to process at least one first feature map to obtain at least one second feature map.

[0019] In this implementation, the process of generating updated lane lines based on the second feature map maintains the same size of the feature map while reducing the number of feature maps, providing a specific approach to generating updated lane lines and improving the feasibility of this solution. Furthermore, reducing the number of feature maps while keeping the size of the feature map constant facilitates the continuous extraction of effective information carried in the feature maps, which is beneficial for generating updated lane lines based on effective feature information and improving the accuracy of the obtained updated lane lines.

[0020] In one possible implementation, where the lane lines include lane centerlines, the execution device obtains updated lane lines through a machine learning model. This includes: the execution device generating at least one updated lane line and at least one piece of first information through the machine learning model. The at least one updated lane line includes at least one first updated lane centerline. Optionally, the at least one updated lane line also includes at least one updated lane boundary line. Each piece of first information indicates a region in the road environment where the number of lane centerlines changes. The method further includes: the execution device obtaining the connection relationships between different first updated lane centerlines within the at least one piece of first information.

[0021] For example, each of the first pieces of information may include a set of coordinates corresponding to a set of closed-loop position points, which are used to indicate the area in the road environment where the number of lane center lines changes. Thus, the at least one piece of information can indicate the area in the road environment where the number of one or more lane center lines changes.

[0022] In this implementation, the first machine learning model not only generates the first updated lane centerline, but also additionally generates first information to indicate the areas where the number of lane centerlines changes in the road environment. Then, using at least one piece of first information, the connection relationship between different lane centerlines in at least one lane centerline is obtained. Since lane centerlines may bifurcate or converge to the same lane centerline in areas where the number of lane centerlines changes, the connection relationship between lane centerlines in these areas is relatively complex. After locating the areas where the number of lane centerlines changes using the first information, the connection relationship between different lane centerlines in at least one lane centerline is determined, which helps to improve the accuracy of the final lane line topology.

[0023] In one possible implementation, the execution device obtains the connection relationship between different lane center lines among at least one lane center line based on at least one first piece of information, including: the execution device updates at least one first updated lane center line based on at least one first piece of information to obtain at least one second updated lane center line, wherein the at least one second updated lane center line indicates the connection relationship between different first updated lane center lines among at least one first updated lane center line.

[0024] For ease of description, any one of the at least one pieces of first information is referred to as target information in this application. For example, the execution device can determine, based on all coordinates included in each of the at least one first updated lane centerlines, which of the aforementioned at least one first updated lane centerlines (hereinafter referred to as "target lane centerlines") intersect within the area where the number of lane centerlines indicated by the target information changes. That is, it determines that multiple coordinates within the area where the number of lane centerlines indicated by the target information changes are contained within at least three target lane centerlines. The execution device can update the aforementioned at least three target lane centerlines to achieve the connection between the aforementioned at least three target lane centerlines, resulting in at least three second updated lane centerlines that correspond one-to-one with the at least three target lane centerlines.

[0025] Optionally, after determining at least three target lane centerlines, the execution device can, based on the at least three target lane centerlines, determine the coordinates of the first endpoint of each target lane centerline within the area where the number of lane centerlines indicated by the target information changes. The first endpoint can be the starting point or ending point of the target lane centerline. Based on the coordinates of the first endpoints of the at least three target lane centerlines, the execution device can determine the coordinates of the center points of at least three first endpoints that correspond one-to-one with the at least three target lane centerlines. The execution device can update the coordinates of the first endpoints of each target lane centerline to the coordinates of the aforementioned center points, thereby realizing the update of each target lane centerline among the at least three target lane centerlines, that is, realizing the connection between the at least three target lane centerlines.

[0026] In this implementation, the connection relationship between different first updated lane center lines is obtained by updating the first updated lane center line. At least one second updated lane center line can directly indicate the connection relationship between different lane center lines in at least one lane center line. Therefore, the second updated lane center line is not only more accurate, but also does not require additional information to execute the connection relationship between different lane center lines, which also helps to improve the convenience of subsequent use of the second updated lane center line.

[0027] In one possible implementation, at least one set of coordinate information corresponding to lane lines in the road environment is obtained based on the vehicle's position while driving in the road environment. For example, the coordinates of the aforementioned multiple vehicle positions can be directly determined as the first coordinates of multiple points on the lane centerline, i.e., assuming the vehicle is traveling along the lane centerline in the road environment. And / or, the aforementioned multiple vehicle positions can be used to determine the first coordinates of multiple points on the lane boundary lines. Optionally, moving the vehicle position to the left by a predetermined length can be considered as a point on the left boundary line of the lane, and moving the vehicle position to the right by a predetermined length can be considered as a point on the right boundary line of the lane. The predetermined length can be understood as half the lane width.

[0028] In this implementation, the idea is to obtain the first coordinate information corresponding to the lane lines in the road environment based on the vehicle's position when driving in the road environment. Since vehicles often drive along the center line of the lane when driving in the road environment, it is reasonable to obtain the first coordinate information based on the vehicle's position when driving in the road environment, thus providing a simpler method for obtaining the first coordinate information.

[0029] In one possible implementation, each set of coordinate information corresponding to the lane lines in the road environment and each point cloud data of the road environment are in the world coordinate system. In this implementation, if the updated lane lines are in the world coordinate system, it is convenient for different vehicles to reuse the generated updated lane lines. If the first coordinate information and the first point cloud data are both in the world coordinate system, the execution device no longer needs to perform coordinate system transformation on the first coordinate information and the first point cloud data, which helps to reduce the number of steps performed by the execution device and reduce the computer resources consumed by the execution device in the process of generating the updated lane lines.

[0030] Secondly, this application provides a lane line acquisition device, which can be used in the field of intelligent driving. The lane line acquisition device includes: an acquisition module, used to acquire coordinate information corresponding to lane lines in the road environment, wherein the lane lines include lane center lines and / or lane boundary lines, and the coordinate information includes the coordinates of position points on the lane lines; the acquisition module is also used to acquire point cloud data of the road environment; and a processing module, used to obtain updated lane lines based on the coordinate information and point cloud data through a machine learning model.

[0031] In the second aspect, the lane line acquisition device is also used to perform the steps of the execution device in the first aspect and various possible implementations of the first aspect. The meanings of the terms in the second aspect and various possible implementations of the second aspect, as well as the beneficial effects of each possible implementation, can be referred to the descriptions in the various possible implementations of the first aspect, and will not be repeated here.

[0032] Thirdly, this application provides an apparatus including a processor and a memory, the processor being coupled to the memory, the memory being used to store a program; and the processor being used to execute the program in the memory, causing the apparatus to perform the method described in the first aspect.

[0033] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect.

[0034] Fifthly, this application provides a computer program product comprising a program that, when run on a computer, causes the computer to perform the method described in the first aspect.

[0035] Sixthly, this application provides a chip system including a processor for supporting the implementation of the functions involved in the foregoing aspects, such as transmitting or processing data and / or information involved in the foregoing methods. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the terminal device or communication device. This chip system may be composed of chips or may include chips and other discrete devices.

[0036] The third to sixth aspects of this application correspond to the first aspect or multiple possible ways of the first aspect, and have corresponding beneficial effects. Attached Figure Description

[0037] Figure 1 is a structural diagram of an artificial intelligence main framework provided in this application;

[0038] Figure 2 is a system architecture diagram of the lane line acquisition system provided in an embodiment of this application;

[0039] Figure 3 is a flowchart illustrating a method for obtaining lane lines provided in an embodiment of this application.

[0040] Figure 4 is a schematic diagram of inputting data into a first machine learning model according to an embodiment of this application;

[0041] Figure 5 is a schematic diagram of updating the lane centerline according to an embodiment of this application;

[0042] Figure 6 is a schematic diagram of the change process of the feature map in the feature processing module of the first machine learning model provided in the embodiment of this application;

[0043] Figure 7 is a schematic flowchart of a training method for a model provided in an embodiment of this application;

[0044] Figure 8 is a schematic diagram illustrating the beneficial effects provided by an embodiment of this application;

[0045] Figure 9 is a schematic diagram of a lane line acquisition device provided in an embodiment of this application;

[0046] Figure 10 is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation

[0047] The embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, and not all, of the embodiments of this application. Those skilled in the art will recognize that, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0048] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0049] In the embodiments of this application, "instruction" can include direct and indirect instructions, as well as explicit and implicit instructions. The information indicated by a certain piece of information (hereinafter referred to as instruction information) is called the information to be instructed. In specific implementation, there are many ways to indicate the information to be instructed, such as, but not limited to, directly indicating the information to be instructed, such as the information to be instructed itself or its index. It can also indirectly indicate the information to be instructed by indicating other information, where there is an association between the other information and the information to be instructed; or it can indicate only a part of the information to be instructed, while the other parts are known or pre-agreed upon. For example, the instruction can be implemented by using a pre-agreed (e.g., protocol predefined) arrangement of various information, thereby reducing the instruction overhead to a certain extent. This application does not limit the specific method of instruction. It is understood that for the sender of the instruction information, the instruction information can be used to indicate the information to be instructed; for the receiver of the instruction information, the instruction information can be used to determine the information to be instructed.

[0050] First, the overall workflow of the artificial intelligence system is described, as shown in Figure 1. Figure 1 is a structural diagram of one aspect of the artificial intelligence framework provided in this application. The framework is then elaborated on from two dimensions: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed by technology) to the industrial ecosystem of the system.

[0051] (1) Infrastructure

[0052] The infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. Communication with the outside world is achieved through sensors; computing power is provided by intelligent chips, which can specifically employ hardware acceleration chips such as central processing units (CPUs), embedded neural network processing units (NPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). The basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0053] (2) Data

[0054] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0055] (3) Data processing

[0056] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0057] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training on data, including symbolization and formalization.

[0058] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0059] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0060] (4) General ability

[0061] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0062] (5) Smart Products and Industry Applications

[0063] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They encapsulate overall artificial intelligence solutions, productize intelligent information decision-making, and realize practical applications. Their application areas mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart homes, intelligent healthcare, intelligent security, intelligent driving, and smart cities.

[0064] The method provided in this application can be applied to the field of intelligent driving, for example, to scenarios in intelligent driving where lane lines are required. Exemplarily, the lane lines in this application include lane center lines and / or lane boundary lines; wherein, lane boundary lines can be lines existing in real roads used to divide different lanes; lane center lines can be understood as the virtual centerline of the lane, and can be lines that do not exist in real roads, and lane center lines can be used to locate lanes.

[0065] For example, when an intelligent driving system in a vehicle plans its trajectory, it can acquire a map of the surrounding traffic environment. This map may include lane lines. For instance, the traffic environment surrounding the vehicle in this application may include the traffic environment within the field of view of sensors deployed on the vehicle. Another example is that the intelligent driving system can use lane center lines in the surrounding traffic environment to plan the vehicle's trajectory. Yet another example is that when providing assisted driving functions, the intelligent driving system can use lane center lines in the surrounding traffic environment to determine the vehicle's position within the lane. And yet another example is that when the intelligent driving system provides navigation functions, it can acquire map information corresponding to the surrounding traffic environment, etc. The application scenarios of this application are not exhaustively listed here.

[0066] For example, the vehicle in this application may be a car, truck, motorcycle, bus, boat, airplane, helicopter, recreational vehicle, amusement park vehicle, tram, golf cart, or train, etc., and no special limitation is made in this application.

[0067] To provide a lane line acquisition solution, this application discloses that: a first device not only acquires the coordinate information corresponding to lane lines in the road environment (hereinafter referred to as "first coordinate information" for ease of distinction), wherein the lane lines include lane center lines and / or lane boundary lines, and the coordinate information includes the coordinates of the position points on the lane lines; it also acquires the point cloud data of the road environment (hereinafter referred to as "first point cloud data" for ease of distinction); and then, based on the aforementioned coordinate information and first point cloud data, obtains updated lane lines through a machine learning model, thereby providing a lane line acquisition solution; furthermore, in the process of generating lane lines in a certain road environment, not only the coordinate information of the position points on the lane lines is used, but also the point cloud data of the road environment is used, which helps to improve the accuracy of the obtained updated lane lines.

[0068] Before describing the specific implementation process of the method provided in this application in detail, the lane line acquisition system provided in the embodiment of this application will be introduced with reference to Figure 2. Please refer to Figure 2, which is a system architecture diagram of the lane line acquisition system provided in the embodiment of this application. In Figure 2, the lane line acquisition system 200 includes a training device 210, a database 220, an execution device 230, a data storage system 240, and a client device 250. The execution device 230 includes a computing module 231.

[0069] The database 220 stores a training dataset. During the training phase of the machine learning model 201, the training device 210 generates the machine learning model 201 and iteratively trains it using the training dataset to obtain a machine learning model 201 that has undergone training operations. The machine learning model 201 can be specifically represented as a neural network or as a non-neural network model. In this embodiment, the machine learning model 201 is described using a neural network as an example.

[0070] The machine learning model 201, trained by the training device 210, can be deployed to the computing module 231 of the execution device 230. The execution device 230 can access data, code, etc., from the data storage system 240, or store data, instructions, etc., in the data storage system 240. The data storage system 240 can be located within the execution device 230, or it can be an external storage device relative to the execution device 230. In the application phase of the machine learning model 201, the machine learning model 201 can be used to obtain updated lane lines in the road environment based on at least one set of coordinate information corresponding to lane lines in the road environment and point cloud data of the road environment.

[0071] In some embodiments of this application, please refer to FIG2. The execution device 230 and the client device 250 can be separate independent devices. The execution device 230 is configured with an input / output (I / O) interface to interact with the client device 250. For example, the client device 250 can send the above-mentioned coordinate information and / or the above-mentioned point cloud data to the execution device 230 through the I / O interface. The execution device 230 can send the updated lane lines to the client device 250 through the I / O interface.

[0072] It is worth noting that Figure 2 is merely a schematic diagram of one architecture of the lane line acquisition system provided in this application embodiment, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. For example, in some other embodiments of this application, the execution device 230 and the client device can be integrated into the same device, so the user can directly interact with the execution device 230. Exemplarily, the execution device 230 can be a module in the host CPU of the client device that uses a deep learning model for data processing. The execution device 230 can also be a graphics processing unit (GPU) or neural network processor (NPU) in the client device, with the GPU or NPU acting as a coprocessor mounted on the host processor, and the host processor assigning tasks. As another example, in some other embodiments of this application, the training device 210 and the execution device 230 can be integrated into the same device, etc., which can be determined according to the actual application scenario.

[0073] Based on the above description, the following describes the specific implementation process of the application stage and the training stage of the machine learning model (hereinafter referred to as the "first machine learning model" for ease of description) in the method provided in the embodiments of this application.

[0074] I. Application Phase

[0075] Please refer to Figure 3, which is a flowchart illustrating a lane line acquisition method provided in an embodiment of this application. The lane line acquisition method provided in this embodiment may include:

[0076] 301. Obtain first coordinate information corresponding to lane lines in the road environment, wherein the lane lines include lane center lines and / or lane boundary lines, and the first coordinate information includes the coordinates of position points on the lane lines.

[0077] For example, the lane centerline can be understood as a virtual centerline of the lane. The lane centerline can be a line that does not exist in the real road environment, and it can be used to locate the lane. The lane boundary line can be a line that exists in the real road environment, and it can be used to divide different lanes. Each set of first coordinate information corresponding to the lane lines in the road environment may include: the first coordinates of a point on the lane centerline in the road environment, and / or the first coordinates of a point on the lane boundary line in the road environment. The first coordinates can be in a first coordinate system.

[0078] For example, the first coordinate system may include position parameters along the X-axis and Y-axis, with the X and Y axes perpendicular to each other. Optionally, the first coordinate system may also include position parameters along the Z-axis. Optionally, the first coordinate system may be a world coordinate system, which can also be referred to as an Earth coordinate system. In this case, the X-axis may align with the longitude direction in the Earth coordinate system, the Y-axis may align with the latitude direction in the Earth coordinate system, and the Z-axis may align with the elevation direction in the Earth coordinate system. In this application, elevation can also be referred to as height. Alternatively, the first coordinate system may be a vehicle coordinate system, which can be the vehicle coordinate system of the vehicle that generated the above coordinate information; or, the first coordinate system may be other coordinate systems, etc., which can be determined based on the actual application scenario.

[0079] For example, in one case, each set of first coordinate information is obtained based on the vehicle's position when the vehicle is traveling in that road environment.

[0080] For a specific implementation of obtaining a set of first coordinate information based on the vehicle's position while driving in the road environment, for example, when the vehicle is driving in the road environment, the vehicle position can be acquired by a first sensor at a first frequency, thereby obtaining the coordinates of each of the multiple vehicle positions; for example, the first sensor can be a global positioning system (GPS), or a BeiDou system, or other positioning systems. The coordinates of each vehicle position may include the vehicle's position parameters in the X-axis direction and the Y-axis direction in the world coordinate system; optionally, the coordinates of each vehicle position may also include the vehicle's position parameters in the Z-axis direction in the world coordinate system.

[0081] For example, the coordinates of the aforementioned multiple vehicle positions can be directly determined as the first coordinates of multiple points on the lane centerline, assuming the vehicle travels along the lane centerline in this road environment. And / or, the aforementioned multiple vehicle positions can be used to determine the first coordinates of multiple points on the lane boundary line. Optionally, moving the vehicle position to the left by a preset length can be considered a point on the left boundary line of the lane, and moving the vehicle position to the right by a preset length can be considered a point on the right boundary line of the lane. The preset length can be understood as half the lane width. For example, the position parameters in the X-axis direction and / or Y-axis direction of each vehicle position's coordinates can be adjusted to move the vehicle position to the left or right by a preset length. Then, after the vehicle travels through this road environment once, a set of coordinate information corresponding to that road environment can be obtained.

[0082] In this application embodiment, a method is proposed to obtain the first coordinate information corresponding to the lane lines in the road environment based on the vehicle position when the vehicle is driving in the road environment. Since the vehicle is often driving along the center line of the lane in the road environment, it is reasonable to obtain the first coordinate information based on the vehicle position when the vehicle is driving in the road environment, thus providing a simpler method for obtaining the first coordinate information.

[0083] In another case, each set of first coordinate information is obtained based on the image of the road environment and / or the point cloud data of the road environment.

[0084] A specific implementation method for obtaining a set of first coordinate information based on the image and / or point cloud data of the road environment is provided. For example, when the vehicle is driving in the road environment, the second sensor can collect images of the environment around the vehicle at a second frequency, and / or the third sensor can collect second point cloud data of the environment around the vehicle at a third frequency. The second frequency and the third frequency can be the same or different. For example, the collected images can be images under a perspective view (PV). The "environment around the vehicle" in this application can be understood as the environment within the field of view of the sensors deployed on the vehicle (such as the aforementioned second or third sensor). After the vehicle drives in the road environment once, it can obtain multiple images of the road environment and multiple second point cloud data.

[0085] For example, the second sensor can be a photoelectric sensor, which can be a camera or an event camera; the third sensor can be an ultrasonic sensor, a lidar sensor, a millimeter-wave radar sensor, or other sensors capable of measuring and obtaining point cloud data, etc. The embodiments of this application do not exhaustively list them.

[0086] For example, the vehicle can input the collected images and / or second point cloud data into the second machine learning model to obtain the first coordinates and / or the second coordinates of the position point on the center line of the lane in the road environment generated by the second machine learning model. After the vehicle travels through the road environment once, it can obtain a set of coordinate information corresponding to the road environment. It should be noted that after the vehicle has completed traveling through the road environment, it can input all the collected images and / or all the second point cloud data into the second machine learning model together to directly obtain a set of coordinate information corresponding to the road environment. Alternatively, during the vehicle's travel in the road environment, the vehicle can also call the second machine learning model multiple times, and each call to the second machine learning model generates a portion of the coordinates included in the aforementioned set of coordinate information. By calling the second machine learning model multiple times, a set of coordinate information corresponding to the road environment can be generated. The specifics can be determined in conjunction with the actual application scenario.

[0087] For example, the second machine learning model can be a convolutional neural network, a fully connected neural network, a residual neural network, an attention-based neural network, a multilayer perceptron (MLP), or other types of machine learning models.

[0088] In another case, each set of first coordinate information is obtained based on the vehicle's position when it is driving in the road environment, as well as images and / or point cloud data of the road environment.

[0089] Regarding a specific implementation method for obtaining a set of first coordinate information based on the vehicle's position while driving in the road environment, and the image and / or point cloud data of the road environment, for example, when the vehicle is driving in the road environment, the vehicle position can be acquired by a first sensor at a first frequency, thereby obtaining the coordinates of each of the multiple vehicle positions. Based on the coordinates of each of the multiple vehicle positions, a set of first sub-coordinate information corresponding to the road environment is obtained. It should be noted that the specific implementation method of "obtaining the first sub-coordinate information corresponding to the road environment based on the coordinates of each of the multiple vehicle positions" can refer to the above description of "obtaining a set of first coordinate information based on the vehicle's position while driving in the road environment", the difference being that the above first coordinate information is replaced with first sub-coordinate information, which will not be repeated here.

[0090] When the vehicle is driving in this road environment, it can also collect images of the surrounding environment of the vehicle through a second sensor at a second frequency, and / or collect second point cloud data of the surrounding environment of the vehicle through a third sensor at a third frequency. Based on the collected images and / or second point cloud data, a set of second sub-coordinate information corresponding to the road environment is obtained. It should be noted that the specific implementation of the above-mentioned "obtaining a set of second sub-coordinate information corresponding to the road environment based on the collected images and / or second point cloud data" can be referred to the above description of "obtaining a set of first coordinate information based on the images and / or point cloud data of the road environment", the difference being that the above-mentioned first coordinate information is replaced with second sub-coordinate information, which will not be repeated here.

[0091] After obtaining a set of first sub-coordinate information and a set of second sub-coordinate information corresponding to the road environment, the vehicle can fuse the aforementioned first sub-coordinate information and second sub-coordinate information to obtain a set of first coordinate information corresponding to the road environment. In other words, the first coordinates of multiple position points on the lane line included in the first sub-coordinate information and the first coordinates of multiple position points on the lane line included in the second sub-coordinate information are directly added together, so that the first coordinate information includes the first coordinates of more position points.

[0092] In step 301, the execution device can acquire at least one set of coordinate information corresponding to lane lines in the road environment, which can be generated by the vehicle. In one implementation, the execution device can be a cloud server communicatively connected to the vehicle, in which case step 301 can include: the execution device receiving at least one set of first coordinate information corresponding to lane lines in the road environment sent by the vehicle. It should be noted that the aforementioned at least one set of first coordinate information can be obtained by one or more vehicles. In another implementation, the execution device can be a vehicle, in which case step 301 can include: the vehicle acquiring at least one set of first coordinate information corresponding to lane lines in the road environment. It should be noted that the vehicle can drive in the aforementioned road environment at least once to obtain at least one set of first coordinate information.

[0093] Optionally, if at least one set of first coordinate information includes at least two sets of first coordinate information, after obtaining at least two sets of first coordinate information corresponding to the lane lines in the road environment, the execution device can fuse the first coordinates of all the location points included in the at least two sets of first coordinate information. In other words, the first coordinates of all the location points included in the at least two sets of first coordinate information can be added together to obtain a denser number of location points on the lane lines in the road environment.

[0094] Optionally, if the executing device is a server connected to the vehicle and the first coordinate system is not the world coordinate system, after receiving at least one set of first coordinate information, the executing device can also transform the first coordinate of each position point in the at least one set of first coordinate information to the world coordinate system to obtain the updated first coordinate of each position point.

[0095] 302. Obtain the first point of cloud data of the road environment.

[0096] For example, each of the first point cloud data in at least one first point cloud data of the road environment may include the second coordinates of multiple location points within the road environment, and the second coordinates may be in a second coordinate system.

[0097] For example, the second coordinate system may include position parameters along the X-axis, Y-axis, and Z-axis. The X-axis and Y-axis are perpendicular to each other, and the Z-axis is perpendicular to the plane formed by the X-axis and Y-axis. Optionally, the second coordinate system is a world coordinate system, which can also be referred to as the Earth coordinate system. In this case, the X-axis can be aligned with the longitude direction in the Earth coordinate system, the Y-axis can be aligned with the latitude direction in the Earth coordinate system, and the Z-axis can be aligned with the elevation direction in the Earth coordinate system. In this embodiment, if the updated lane lines obtained through subsequent steps are in the world coordinate system, it is convenient for different vehicles to reuse the aforementioned updated lane lines. If both the first coordinate information and the first point cloud data are in the world coordinate system, the execution device no longer needs to perform coordinate system conversion on the first coordinate information and the first point cloud data, which helps to reduce the steps performed by the execution device and reduce the computer resources consumed by the execution device in the process of generating the updated lane lines.

[0098] Alternatively, the second coordinate system can also be the vehicle coordinate system, which can be the vehicle coordinate system of the vehicle that generated the first point cloud data mentioned above; or the second coordinate system can also be other coordinate systems, etc., which can be determined according to the actual application scenario.

[0099] Regarding the specific implementation of obtaining the first point cloud data for the vehicle, for example, when the vehicle is driving in the road environment, it can collect the second point cloud data of the surrounding environment through a third sensor at a third frequency. After the vehicle drives in the road environment once, it can obtain multiple second point cloud data of the road environment. The vehicle can fuse the aforementioned multiple second point cloud data to obtain a first point cloud data of the road environment; for example, the vehicle can accumulate the second coordinates of the position points included in all the second point cloud data in the multiple second point cloud data to obtain a first point cloud data of the road environment, that is, a first point cloud data of the road environment includes the second coordinates of the position points included in all the aforementioned multiple second point cloud data.

[0100] Optionally, when the vehicle is driving in the road environment, it can also collect images of the surrounding environment of the vehicle through the second sensor at the second frequency. After the vehicle drives in the road environment once, it can obtain at least one image of the road environment. The vehicle can also project each image of the at least one image of the road environment into a three-dimensional space based on the intrinsic and extrinsic parameters of the second sensor to obtain at least one third point cloud data corresponding to the aforementioned at least one image. Each third point cloud data includes the third coordinates of the position point in the road environment in the third coordinate system. Optionally, if the third coordinate system is different from the second coordinate system, it is also necessary to align the third point cloud data with the second point cloud data to obtain updated third point cloud data. The updated third point cloud data may include the updated third coordinates of the position point in the road environment in the second coordinate system. A vehicle can fuse all third coordinates included in at least one third point cloud data set (or all updated third coordinates included in the updated third point cloud data set) with the second coordinates included in multiple second point cloud data sets to obtain a first point cloud data set for the road environment. That is, a first point cloud data set can include all second coordinates from the multiple second point cloud data sets and all third coordinates from the at least one third point cloud data set (or all updated third coordinates from the at least one updated third point cloud data set). After driving through the road environment multiple times, multiple first point cloud data sets for the road environment can be obtained.

[0101] In step 302, the execution device can acquire at least one first point cloud data of the road environment, which can be generated by the vehicle. In one implementation, the execution device can be a cloud server communicatively connected to the vehicle, in which case step 301 can include: the execution device receiving at least one first point cloud data of the road environment sent by the vehicle. It should be noted that the at least one first point cloud data can be obtained by one or more vehicles. In another implementation, the execution device can be a vehicle, in which case step 301 can include: the vehicle collecting at least one first point cloud data of the road environment. It should be noted that the vehicle can drive through the road environment at least once to obtain at least one first point cloud data.

[0102] Optionally, if the first coordinate system is not the world coordinate system, after acquiring at least one first point cloud data, the executing device can also transform the second coordinates of each location point in the at least one first point cloud data to the world coordinate system to obtain the updated second coordinates of each location point in the at least one first point cloud data.

[0103] Optionally, before transforming the second coordinates of each location point in at least one first point cloud data to the world coordinate system, the execution device may also align the second coordinate systems corresponding to different first point cloud data based on the position and orientation of the vehicle corresponding to each first point cloud data when collecting the point cloud data. For example, the second coordinates in each of the at least one first point cloud data can be adjusted to translate the origin of the second coordinate system corresponding to each first point cloud data to a preset position, and to translate the X-axis, Y-axis, and Z-axis of the second coordinate system corresponding to each first point cloud data to a preset orientation, thereby achieving alignment between the second coordinate systems corresponding to different first point cloud data in at least one first point cloud data.

[0104] It should be noted that this application does not limit the execution order of steps 301 and 302. Step 301 can be executed first, followed by step 302; or step 302 can be executed first, followed by step 301; or steps 301 and 302 can be executed simultaneously.

[0105] 303. Based on the first coordinate information and the first point cloud data, the updated lane lines are obtained through the first machine learning model.

[0106] For example, the execution device can obtain at least one updated lane line in the road environment based on at least one set of first coordinate information corresponding to the lane lines in the road environment obtained in step 301 and at least one first point cloud data of the road environment obtained in step 302, through a first machine learning model. For example, the first machine learning model can be a convolutional neural network, a fully connected neural network, a residual neural network, an attention-based neural network, or other types of neural networks, which can be determined according to the actual application scenario.

[0107] For example, if the lane lines in this application only include lane center lines (or lane boundary lines), then at least one updated lane line may include at least one set of sequentially arranged coordinates corresponding one-to-one with at least one first updated lane center line (or updated lane boundary line). That is, each updated lane line may include a set of sequentially arranged coordinates corresponding to a first updated lane center line (or updated lane boundary line), and each set of sequentially arranged coordinates includes at least two sequentially arranged fourth coordinates; wherein, the line connecting the aforementioned at least two fourth coordinates may represent a first updated lane center line (or updated lane boundary line), the first coordinate in the aforementioned sequentially arranged at least two fourth coordinates may indicate the starting point of a first updated lane center line (or updated lane boundary line), and the last coordinate in the sequentially arranged at least two fourth coordinates may indicate the ending point of a first updated lane center line (or updated lane boundary line).

[0108] For example, the order of at least two fourth coordinates in each sequence of coordinates can indicate the direction of the updated lane line, which can be from the first coordinate among at least two fourth coordinates to the last coordinate among at least two fourth coordinates.

[0109] For example, at least two fourth coordinates in the above sequence sorting can both be coordinates in the world coordinate system; for example, the aforementioned fourth coordinates can both include position parameters in the X-axis direction and position parameters in the Y-axis direction of the world coordinate system; optionally, the aforementioned fourth coordinates can also include position parameters in the Z-axis direction of the world coordinate system.

[0110] Optionally, each updated lane line may include not only a set of sequentially arranged coordinates, but also a group identifier (ID) corresponding to the aforementioned set of sequentially arranged coordinates. Different group identifiers are used to distinguish between different sets of sequentially arranged coordinates.

[0111] If the lane lines in this application include lane center lines and lane boundary lines, each updated lane line may include not only a set of sequentially arranged coordinates corresponding to the updated lane line, but also the category of the updated lane line. For example, the category of the updated lane line may be lane center line or lane boundary line, etc., which can be determined in combination with the actual application scenario.

[0112] For example, in one implementation, step 303 may include: the execution device performing a projection operation to obtain a first image based on at least one set of first coordinate information corresponding to lane lines in the road environment. In other words, the execution device may project all position points indicated by at least one set of first coordinate information into the first image based on all first coordinates (or updated first coordinates) included in at least one set of first coordinate information. The first image includes multiple position points on lane lines in the road environment.

[0113] The execution device can also perform a projection operation based on at least one first point cloud data of the road environment to obtain a second image. In other words, the execution device can project all the position points indicated by at least one first point cloud data into the second image based on all the second coordinates (or updated second coordinates) included in at least one first point cloud data. The position points in the second image correspond to the position points in the first point cloud data, that is, the position points in the first point cloud data are projected from three dimensions to two dimensions to obtain the position points in the second image.

[0114] The execution device can then input the first image and the second image into the first machine learning model, and process the first image and the second image through the first machine learning model to generate at least one updated lane line in the road environment; the meaning of "at least one updated lane line" can be found in the above description, and will not be repeated here.

[0115] For example, the size of the first image can be the same as or different from the size of the second image.

[0116] Optionally, the first image can be an image of the road environment from a bird's-eye view (BEV), which in this application can also be called a top-down view; or, the first image can also be an image from a perspective view. Optionally, the second image can be an image of the road environment from a bird's-eye view, or, the second image can also be an image from a perspective view, etc. The specific details of the first and second images can be determined in conjunction with the actual application scenario.

[0117] To understand this solution more intuitively, please refer to Figure 4. Figure 4 is a schematic diagram of inputting data into the first machine learning model according to an embodiment of this application. As shown in Figure 4, after the execution device obtains at least one set of first coordinate information corresponding to the lane lines in the road environment and at least one set of first point cloud data of the road environment, it can perform a projection operation based on at least one set of first coordinate information to project the position points on the lane lines in the road environment onto the first image; perform a projection operation based on at least one set of first point cloud data to project the position points in the first point cloud data onto the second image; input the first image and the second image into the first machine learning model, so that the updated lane lines can be obtained through the first machine learning model. It should be understood that the example in Figure 4 is only for the convenience of understanding this solution and is not intended to limit this solution.

[0118] In this embodiment, a projection operation is performed on at least one set of first coordinate information corresponding to lane lines in the road environment to obtain a first image, and a projection operation is performed on at least one first point cloud data of the road environment to obtain a second image. The first image and the second image are then input into a first machine learning model, and the updated lane lines are obtained through the first machine learning model. Since at least one set of first coordinate information and at least one first point cloud data are reduced to the form of images, the computer resources consumed by the processing of the first machine learning model are reduced, and the speed of obtaining the updated lane lines is improved.

[0119] In another implementation, step 303 may include: the execution device inputs at least one set of first coordinate information corresponding to the lane lines in the road environment obtained in step 301 and at least one first point cloud data of the road environment obtained in step 302 into a first machine learning model, and processes the aforementioned at least one set of first coordinate information and at least one first point cloud data through the first machine learning model to generate at least one updated lane line in the road environment.

[0120] In this application, where lane lines include lane center lines, optionally, the execution device inputs a first image and a second image into a first machine learning model. After processing the first image and the second image by the first machine learning model, or the execution device inputs at least one set of first coordinate information corresponding to lane lines in the road environment obtained in step 301 and at least one first point cloud data of the road environment obtained in step 302 into the first machine learning model. After processing the aforementioned at least one set of first coordinate information and at least one first point cloud data by the first machine learning model, the first machine learning model can not only generate at least one updated lane line, but also generate at least one first piece of information. Each piece of first information indicates the area in the road environment where the number of lane center lines changes. The aforementioned at least one updated lane line includes at least one first updated lane center line. Optionally, the at least one updated lane line may also include at least one updated lane boundary line. Furthermore, the execution device can obtain the connection relationship between different lane center lines in at least one lane center line based on the aforementioned at least one piece of first information.

[0121] For example, each piece of first information may include a set of coordinates corresponding to a closed array of location points, which are used to indicate the area in the road environment where the number of lane center lines changes. Thus, at least one piece of first information can indicate the area in the road environment where the number of one or more lane center lines changes. For example, when the number of lanes in the road environment increases, the number of lane center lines in the road environment will change; or, for example, when the number of lanes in the road environment decreases, the number of lane center lines in the road environment will also change. This application does not exhaustively list all scenarios that would cause the number of lane center lines to change.

[0122] Regarding a specific implementation method for an execution device to obtain the connection relationship between different lane center lines in at least one lane center line based on at least one first piece of information, for example, the execution device can update at least one first updated lane center line based on at least one first piece of information to obtain at least one second updated lane center line. The at least one second updated lane center line indicates the connection relationship between different first updated lane center lines in at least one first updated lane center line, or it can be said that the at least one second updated lane center line indicates the connection relationship between different second updated lane center lines in at least one second updated lane center line. The at least one updated lane line includes the updated at least one first updated lane center line. Optionally, the at least one updated lane line may also include at least one updated lane boundary line.

[0123] For ease of description, any one of the at least one pieces of first information is referred to as target information in this application. For example, the execution device can determine, based on all the fourth coordinates included in each of the at least one first updated lane center lines, which of the aforementioned at least one first updated lane center lines (hereinafter referred to as "target lane center lines") intersect within the area where the number of lane center lines indicated by the target information changes. That is, it determines that multiple fourth coordinates within the area where the number of lane center lines indicated by the target information changes are included in at least three target lane center lines. The execution device can update the aforementioned at least three target lane center lines to achieve the connection between the aforementioned at least three target lane center lines, resulting in at least three second updated lane center lines that correspond one-to-one with the at least three target lane center lines.

[0124] Furthermore, in one implementation, after determining at least three target lane centerlines, the execution device can, based on the at least three target lane centerlines, determine the coordinates of the first endpoint of each target lane centerline within the area where the number of lane centerlines indicated by the target information changes. The first endpoint can be the starting point or ending point of the target lane centerline. Based on the coordinates of the first endpoints of the at least three target lane centerlines, the execution device can determine the coordinates of the center points of at least three first endpoints that correspond one-to-one with the at least three target lane centerlines. The execution device can update the coordinates of the first endpoints of each target lane centerline to the coordinates of the aforementioned center points, thereby realizing the update of each target lane centerline among the at least three target lane centerlines, that is, realizing the connection between the at least three target lane centerlines.

[0125] In another implementation, after determining the coordinates of the first endpoint of each target lane centerline within the area where the number of lane centerlines indicated by the target information changes based on at least three target lane centerlines, the execution device can further select the coordinates of a target endpoint from the coordinates of at least three first endpoints corresponding to the at least three first lane centerlines. For example, the coordinates of a target endpoint can be arbitrarily selected from the coordinates of the at least three first endpoints, or the selection can be based on other filtering rules. The execution device updates the coordinates of all other first endpoints besides the target endpoint among the at least three first endpoints to the coordinates of the target endpoint, thereby realizing the update of the at least three target lane centerlines, that is, realizing the connection between the at least three target lane centerlines.

[0126] To understand this solution more intuitively, please refer to Figure 5. Figure 5 is a schematic diagram of updating the lane center line according to an embodiment of this application. Figure 5 is a schematic diagram obtained after visualizing the lane center line and the first information. First, refer to the left sub-schematic diagram of Figure 5, where the ellipse represents the area where the number of lane center lines changes. As shown in the left sub-schematic diagram of Figure 5, the end point of lane center line 1, the end point of lane center line 2, and the starting point of lane center line 3 are located inside the ellipse. After updating the coordinates of the end point of lane center line 1, the end point of lane center line 2, and the starting point of lane center line 3, the right sub-schematic diagram of Figure 5 is obtained. In the right sub-schematic diagram of Figure 5, lane center line 1, lane center line 2, and lane center line 3 are connected. It should be understood that the example in Figure 5 is only for the convenience of understanding this solution and is not intended to limit this solution.

[0127] It should be noted that the execution device can perform the above steps based on each of the first pieces of information in at least one first piece of information, thereby realizing the update operation of at least one first updated lane center line to obtain at least one second updated lane center line. The at least one second updated lane center line includes the updated position information of the target lane center line. Optionally, if there are still first updated lane center lines that have not been updated among the at least one first updated lane center lines, then the at least one second updated lane center line also includes the aforementioned first updated lane center lines that have not been updated.

[0128] In this embodiment, the connection relationship between different first updated lane center lines is obtained by updating the first updated lane center line. At least one second updated lane center line can directly indicate the connection relationship between different lane center lines in at least one lane center line. Therefore, the second updated lane center line is not only more accurate, but also does not require additional information to execute the connection relationship between different lane center lines, which also helps to improve the convenience of subsequent use of the second updated lane center line.

[0129] In another implementation, the execution device can generate at least one second piece of information based on at least one first piece of information. Each second piece of information indicates which first updated lane centerlines are connected within an area where the number of lane centerlines changes in the road environment. For example, each piece of second information may include at least three sets of identifiers, thereby indicating a connection between at least three first updated lane centerlines corresponding to the aforementioned at least three sets of identifiers; or, each piece of second information may include at least three sets of sequentially arranged coordinates, thereby indicating a connection between at least three first updated lane centerlines corresponding to the aforementioned at least three sets of sequentially arranged coordinates, etc. The specific information included in the second information can be determined based on the actual application scenario.

[0130] For ease of description, any one of the first pieces of information in this application is referred to as target information. For example, the execution device can determine which target lane center lines in the area where the number of lane center lines indicated by the target information changes intersect with the aforementioned at least one first updated lane center line based on all the fourth coordinates included in each of the at least one first updated lane center lines. That is, it determines that multiple fourth coordinates in the area where the number of lane center lines indicated by the target information changes are included in at least three target lane center lines, thereby generating a second piece of information. The second piece of information may include at least three sets of identifiers that correspond one-to-one with the at least three target lane center lines, or the second piece of information may include at least three sets of sequentially arranged coordinates that correspond one-to-one with the at least three target lane center lines.

[0131] It should be noted that the execution device can perform the above steps based on each of the first pieces of information to generate the above at least one second piece of information.

[0132] In this embodiment, the first machine learning model not only generates the first updated lane centerline, but also generates additional first information to indicate the area where the number of lane centerlines changes in the road environment. Then, with the help of at least one piece of first information, the connection relationship between different lane centerlines in at least one lane centerline is obtained. Since there may be bifurcations of lane centerlines or different lane centerlines converging into the same lane centerline in the area where the number of lane centerlines changes, the connection relationship of lane centerlines in the area where the number of lane centerlines changes will be relatively complex. After locating the area where the number of lane centerlines changes using the first information, the connection relationship between different lane centerlines in at least one lane centerline is determined, which helps to improve the accuracy of the topology structure of the lane lines obtained in the end.

[0133] For example, the first machine learning model may include a feature extraction module and a feature processing module; wherein, the feature extraction module in the first machine learning model is used to perform feature extraction operations to obtain at least one first feature map; the feature processing module in the first machine learning model is used to process the aforementioned at least one first feature map to generate at least one updated lane line (optionally, it also includes at least one first piece of information); for example, the feature processing module in the first machine learning model is used to obtain at least one second feature map based on the aforementioned at least one first feature map, and then generate the aforementioned at least one updated lane line (optionally, it also includes at least one piece of first information) based on the aforementioned at least one second feature map.

[0134] In one implementation, step 303 may include: the execution device inputs a first image and a second image into a first machine learning model, and performs feature extraction on the first image and the second image through the feature extraction module of the first machine learning model to obtain at least one first feature map, wherein the at least one first feature map includes feature information of the first image and feature information of the second image; the feature processing module in the first machine learning model is used to obtain at least one second feature map based on the at least one first feature map, and then generates the at least one updated lane line based on the at least one second feature map (optionally, it also includes at least one first piece of information).

[0135] In another implementation, step 303 may include: the execution device inputs at least one set of first coordinate information corresponding to lane lines in the road environment obtained in step 301 and at least one first point cloud data of the road environment obtained in step 302 into a first machine learning model; the feature extraction module of the first machine learning model performs feature extraction on the at least one set of first coordinate information and at least one first point cloud data to obtain at least one first feature map, wherein the at least one first feature map includes feature information of at least one set of first coordinate information and feature information of at least one first point cloud data; the feature processing module in the first machine learning model is used to obtain at least one second feature map based on the at least one first feature map, and then generate the at least one updated lane line (optionally, also including at least one first piece of information) based on the at least one second feature map.

[0136] For example, the feature extraction module in the first machine learning model may include at least one of the following: convolutional neural network layer, fully connected neural network layer, residual neural network layer, attention-based neural network layer or other types of neural network layer, etc., without exhaustive list here.

[0137] In one scenario, the size of each of the at least one second feature maps can be equal to the size of the first image, and the size of the feature maps used by the feature processing module in the first machine learning model to generate at least one updated lane line (optionally, also including at least one piece of first information) based on at least one second feature map remains unchanged.

[0138] In this embodiment, the updated lane lines generated by the first machine learning model are described using the size of the first image. Since the size of the second feature map is equal to the size of the first image, and the size of the feature map used in generating the updated lane lines based on the second feature map remains unchanged, the updated lane lines can be predicted based on a feature map with the same size as the first image. If the updated lane lines are predicted based on a feature map with a size smaller than the first image, then after generating the updated lane lines, it is necessary to upsample to the size of the first image. If the updated lane lines are predicted based on a feature map with a size larger than the first image, then it is necessary to downsample to the size of the first image. The aforementioned upsampling and downsampling may further amplify the errors carried in the predicted updated lane lines. However, in this application, the updated lane lines are predicted based on a feature map with the same size as the first image, eliminating the need for upsampling and downsampling operations, which helps to improve the accuracy of the obtained updated lane lines.

[0139] Optionally, the feature processing module of the first machine learning model includes at least one neural network layer (hereinafter referred to as the "first neural network layer" for ease of description), which is used in the process of generating updated lane lines based on at least one second feature map. The aforementioned first neural network layer is used to keep the size of the feature map unchanged and reduce the number of feature maps.

[0140] For example, the feature processing module in the first machine learning model can directly determine at least one first feature map as at least one second feature map; or, the feature processing module in the first machine learning model can also process at least one first feature map to obtain at least one second feature map, etc., which can be determined according to the actual application scenario.

[0141] Optionally, the feature processing module of the first machine learning model further includes at least one second neural network layer, which is used to process at least one first feature map to obtain at least one second feature map.

[0142] To understand this solution more intuitively, please refer to Figure 6. Figure 6 is a schematic diagram of the feature map change process in the feature processing module of the first machine learning model provided in this application embodiment. As shown in Figure 6, after obtaining at least one first feature map through the feature extraction module of the first machine learning model, the first feature map can be upsampled and convolved by the feature processing module of the first machine learning model to obtain at least one feature map 1; the feature map 1 can be further convolved by the feature processing module of the first machine learning model to obtain at least one second feature map; the at least one second feature map can be convolved three times by the feature processing module of the first machine learning model to obtain at least one feature map 2, feature map 3 and feature map 4 in sequence; then the at least one feature map 4 can be processed by the Softmax layer in the feature processing module of the first machine learning model to obtain at least one updated lane line generated by the first machine learning model (optionally, it also includes at least one first information), wherein the Softmax layer can be one of the activation functions. As shown in Figure 6, during the process of evolving from at least one second feature map to at least one feature map 2, feature map 3 and feature map 4, the size of the feature maps remains unchanged, while the number of feature maps decreases continuously. It should be understood that the example in Figure 6 is only for the convenience of understanding this scheme and is not intended to limit this scheme.

[0143] In this embodiment, during the process of generating updated lane lines based on the second feature map, maintaining the size of the feature map unchanged and reducing the number of feature maps provides a specific implementation idea for generating updated lane lines, improving the feasibility of this solution. In addition, reducing the number of feature maps while keeping the size of the feature map unchanged facilitates the continuous extraction of effective information carried in the feature map, which is beneficial for generating updated lane lines based on effective feature information and improving the accuracy of the obtained updated lane lines.

[0144] Alternatively, the size of each second feature map in at least one second feature map may be larger than the size of the first image; or the size of each second feature map in at least one second feature map may be smaller than the size of the first image; or the size of the feature map may change during the process of generating updated lane lines based on at least one second feature map by the feature processing module of the first machine learning model, etc. The specific settings can be combined with the actual application scenario.

[0145] Optionally, after obtaining at least one updated lane line in the road environment, the execution device may also generate map information of the road environment based on the at least one updated lane line, the map information of the road environment may include at least one updated lane line.

[0146] In this embodiment, at least one set of coordinate information corresponding to lane lines in a road environment and point cloud data of the road environment are obtained. The lane lines include lane center lines and / or lane boundary lines. Each set of coordinate information includes the coordinate information of a position point on the lane line. Based on the aforementioned at least one set of coordinate information and point cloud data, an updated lane line is obtained through a machine learning model, providing a scheme for obtaining an updated lane line. Furthermore, in the process of generating an updated lane line in a certain road environment, not only the coordinate information of the position points on the lane line is used, but also the point cloud data of the road environment is used, which helps to improve the accuracy of the obtained updated lane line.

[0147] II. Training Phase

[0148] Please refer to Figure 7, which is a flowchart illustrating a model training method provided in an embodiment of this application. The model training method provided in an embodiment of this application may include:

[0149] 701. Obtain training samples. The training samples include first coordinate information corresponding to lane lines in the road environment and first point cloud data of the road environment. The lane lines include lane center lines and / or lane boundary lines. The first coordinate information includes the coordinates of the position points on the lane lines.

[0150] For example, the training device may be deployed with multiple training data, each training data may include training samples, and each training sample may include at least one set of first coordinate information corresponding to lane lines in the road environment and at least one first point cloud data of the road environment. The specific meaning and acquisition method of "at least one set of first coordinate information corresponding to lane lines in the road environment and at least one first point cloud data of the road environment" can be referred to the description in the embodiment corresponding to Figure 3 above, and will not be repeated here.

[0151] 702. Based on the first coordinate information and the first point cloud data, the updated lane lines are generated through the first machine learning model.

[0152] The specific implementation of step 702 by the training device and the specific meaning of the terms in step 702 can be found in the description of step 303 in the embodiment corresponding to Figure 3 above, and will not be repeated here.

[0153] 703. Train the first machine learning model based on the updated lane lines and the expected updated lane lines.

[0154] For example, each training data may also include a ground truth corresponding to the training sample. The ground truth corresponding to the training sample may include at least one expected updated lane line corresponding to the training sample. The expected updated lane line may also be referred to as the ground truth. Each expected updated lane line in the at least one expected updated lane line can be understood as the correct position information of each lane line. The specific form of the expected updated lane line is similar to the specific form of the updated lane line in the corresponding embodiment of Figure 3, and will not be repeated here.

[0155] Optionally, if the output of the first machine learning model also includes at least one first piece of information, the true value corresponding to the training sample may include at least one expected piece of information corresponding to at least one first piece of information. Each expected piece of information indicates the correct area where the number of lane center lines changes in the road environment. The specific form of the expected information is similar to the specific form of the first information in the corresponding embodiment of Figure 3, and will not be repeated here.

[0156] For example, the training device can determine the value of a loss function based on at least one updated lane line generated by a first machine learning model and at least one desired updated lane line. Then, based on the value of the loss function, it uses a backpropagation algorithm to update the weight parameters of the first machine learning model to complete one training iteration of the first machine learning model. The loss function indicates the similarity between the at least one updated lane line and the desired updated lane line. The objective of training using the loss function includes improving the similarity between the at least one updated lane line and the desired updated lane line.

[0157] Optionally, the training device can determine the value of a loss function based on at least one updated lane line, at least one piece of first information, at least one desired updated lane line, and at least one piece of desired information generated by the first machine learning model. Then, based on the value of the loss function, the weight parameters of the first machine learning model are updated using a backpropagation algorithm to complete one training iteration of the first machine learning model. The loss function indicates not only the similarity between the aforementioned at least one updated lane line and the desired updated lane line, but also the similarity between at least one piece of first information and the desired information. The goal of training using the loss function includes not only improving the similarity between the at least one updated lane line and the desired updated lane line, but also improving the similarity between at least one piece of first information and the desired information.

[0158] The training device repeatedly executes steps 701 to 703 multiple times to iteratively train the first machine learning model until a convergence condition is met, resulting in a first machine learning model that has undergone training. For example, the convergence condition may include: meeting the convergence condition of the loss function and / or performing a preset number of iterative training iterations on the first machine learning model.

[0159] To more intuitively understand the beneficial effects of the method provided in this application, please refer to Figure 8. Figure 8 is a schematic diagram of the beneficial effects provided by the embodiments of this application. Figure 8 includes two sub-schematic diagrams, an upper one and a lower one. The upper sub-schematic diagram of Figure 8 represents that after obtaining the first coordinate information corresponding to the lane line in the road environment, multiple position points on the lane line are directly clustered to fit the updated lane line in the road environment. The lower sub-schematic diagram of Figure 8 represents the updated lane line obtained by the method provided in this application. By comparing the upper and lower sub-schematic diagrams of Figure 8, it can be seen that the clustering-based method is easily affected by noise interference from some position points with large deviations, which can lead to incorrect bifurcation in the topology of the fitted lane line. However, the updated lane line obtained by the method provided in this application has a certain robustness to noise scenarios with some position points with large deviations. These noises can be ignored to obtain a more consistent updated lane line, which is beneficial to obtaining the correct position information of the lane line.

[0160] Based on the embodiments corresponding to Figures 1 to 8, in order to better implement the above-described solutions of the embodiments of this application, related equipment for implementing the above-described solutions is also provided below. Specifically, refer to Figure 9, which is a schematic structural diagram of a lane line acquisition device provided in an embodiment of this application. The lane line acquisition device 900 includes: an acquisition module 901, used to acquire coordinate information corresponding to lane lines in the road environment, wherein the lane lines include lane center lines and / or lane boundary lines, and the coordinate information includes the coordinates of position points on the lane lines; the acquisition module 901 is also used to acquire point cloud data of the road environment; and a processing module 902, used to obtain updated lane lines based on the coordinate information and point cloud data through a machine learning model.

[0161] Optionally, the processing module 902 is specifically used for: performing a projection operation based on coordinate information to obtain a first image, the first image including position points on the lane line; performing a projection operation based on point cloud data to obtain a second image, the position points in the second image corresponding to the point cloud data; inputting the first image and the second image into a machine learning model, and obtaining an updated lane line through the machine learning model.

[0162] Optionally, the machine learning model includes a feature extraction module and a feature processing module. The feature extraction module is used to perform feature extraction operations to obtain a first feature map. The feature processing module is used to obtain a second feature map based on the first feature map and generate an updated lane line based on the second feature map. The size of the second feature map is equal to the size of the first image. The size of the feature map used in the process of the feature processing module generating the updated lane line based on the second feature map remains unchanged.

[0163] Optionally, the feature processing module includes at least one neural network layer used in the process of generating updated lane lines based on the second feature map. The neural network layer is used to keep the size of the feature map unchanged and reduce the number of feature maps.

[0164] Optionally, the lane lines include lane center lines. The processing module 902 is used to generate at least one first updated lane center line and at least one first piece of information through a machine learning model. The first piece of information indicates the area in the road environment where the number of lane center lines changes. The processing module 902 is also used to obtain the connection relationship between different first updated lane center lines in the at least one first updated lane center line based on the at least one piece of first information.

[0165] Optionally, the processing module is specifically configured to update at least one first updated lane centerline based on at least one first piece of information to obtain at least one second updated lane centerline, wherein the at least one second updated lane centerline indicates the connection relationship between different first updated lane centerlines among the at least one first updated lane centerline.

[0166] Optionally, the coordinate information is obtained based on the vehicle's position while driving in a road environment.

[0167] Optionally, both the coordinate information and the point cloud data are in the world coordinate system.

[0168] It should be noted that the information interaction and execution process between the modules / units in the lane line acquisition device 900 are based on the same concept as the various method embodiments corresponding to Figures 1 to 8 in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0169] This application also provides a device, as shown in FIG10, which is a schematic diagram of the structure of the device provided in this application embodiment. Optionally, the device 1000 performs the functions of the execution device or training device in the various method embodiments corresponding to FIG1 to FIG8.

[0170] The device 1000 includes a memory 1002 and at least one processor 1001. Optionally, the processor 1001 implements the methods in the above embodiments by reading instructions stored in the memory 1002, or the processor 1001 may also implement the methods in the above embodiments by internally stored instructions. When the processor 1001 implements the methods in the above embodiments by reading instructions stored in the memory 1002, the memory 1002 stores instructions for implementing the methods provided in the above embodiments of this application.

[0171] Optionally, at least one processor 1001 is one or more CPUs, either a single-core CPU or a multi-core CPU. The memory 1002 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, or optical memory. The memory 1002 stores operating system instructions. After the program instructions stored in the memory 1002 are read by the at least one processor 1001, the device 1000 executes the corresponding operations in the foregoing embodiments.

[0172] Optionally, the device 1000 also includes a network interface 1003, which can be a wired interface or a wireless interface. The network interface 1003 is used to perform data transmission and reception in the various method embodiments corresponding to Figures 1 to 8.

[0173] It should be understood that network interface 1003 has the functions of receiving and sending data. The functions of "receiving data" and "sending data" can be integrated into the same transceiver interface, or the functions of "receiving data" and "sending data" can be implemented in different interfaces, which is not limited here. In other words, network interface 1003 may include one or more interfaces for implementing the functions of "receiving data" and "sending data".

[0174] After the processor 1001 reads the program instructions from the memory 1002, other functions that the device 1000 can perform are described in the preceding method embodiments.

[0175] Optionally, the device 1000 also includes a bus 1004, through which the processor 1001 and memory 1002 are typically interconnected, or in other ways.

[0176] The device 1000 provided in this application embodiment is used to execute the methods executed by the execution device or training device in the above-described method embodiments, and to achieve the corresponding beneficial effects. The specific implementation of the device 1000 shown in Figure 10 can be referred to the descriptions in the foregoing method embodiments, and will not be repeated here.

[0177] This application also provides a computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform the steps executed by the execution device in the methods described in the embodiments shown in Figures 1 to 8, or causes the computer to perform the steps executed by the training device in the methods described in the embodiments shown in Figures 1 to 8.

[0178] This application also provides a computer program product comprising a program that, when run on a computer, causes the computer to perform the steps performed by the execution device in the methods described in the embodiments shown in Figures 1 to 8, or causes the computer to perform the steps performed by the training device in the methods described in the embodiments shown in Figures 1 to 8.

[0179] This application also provides a circuit system including a processing circuit configured to perform the steps executed by the execution device in the method described in the embodiments shown in Figures 1 to 8, or the processing circuit configured to perform the steps executed by the training device in the method described in the embodiments shown in Figures 1 to 8.

[0180] The execution device or training device provided in this application embodiment can specifically be a chip. The chip includes a processing unit, such as a processor. Optionally, the chip also includes a communication unit, such as an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip to execute the methods described in the embodiments shown in Figures 1 to 8. Optionally, the storage unit is a storage unit within the chip, such as a register or cache. The storage unit can also be a storage unit located outside the chip within the execution device or training device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0181] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0182] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CLUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0183] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0184] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

Claims

1. A method for obtaining lane lines, characterized in that, The method includes: Obtain coordinate information corresponding to lane lines in the road environment, wherein the lane lines include lane center lines and / or lane boundary lines, and the coordinate information includes the coordinates of position points on the lane lines; Obtain point cloud data of the road environment; Based on the coordinate information and the point cloud data, the updated lane lines are obtained through a machine learning model.

2. The method according to claim 1, characterized in that, The process of obtaining the updated lane lines based on the coordinate information and the point cloud data using a machine learning model includes: A projection operation is performed based on the coordinate information to obtain a first image, the first image including the position points on the lane line; A second image is obtained by performing a projection operation based on the point cloud data, and the position points in the second image correspond to the point cloud data. The first image and the second image are input into the machine learning model, and the updated lane lines are obtained through the machine learning model.

3. The method according to claim 1 or 2, characterized in that, The machine learning model includes a feature extraction module and a feature processing module. The feature extraction module is used to perform feature extraction operations to obtain a first feature map. The feature processing module is used to obtain a second feature map based on the first feature map, and to generate the updated lane lines based on the second feature map, wherein the size of the second feature map is equal to the size of the first image, and the size of the feature map used in the process of the feature processing module generating the updated lane lines based on the second feature map remains unchanged.

4. The method according to claim 3, characterized in that, The feature processing module includes at least one neural network layer used in the process of generating the updated lane lines based on the second feature map. The neural network layer is used to keep the size of the feature map unchanged and reduce the number of feature maps.

5. The method according to claim 1 or 2, characterized in that, The lane lines include lane center lines, and the updated lane lines obtained through the machine learning model include: At least one first updated lane centerline and at least one first piece of information are generated by a machine learning model, wherein the first piece of information indicates the area in the road environment where the number of lane centerlines changes. The method further includes: obtaining the connection relationship between different first updated lane centerlines in the at least one first updated lane centerline based on the at least one first piece of information.

6. The method according to claim 5, characterized in that, The step of obtaining the connection relationship between different lane centerlines in the at least one lane centerline based on the at least one piece of first information includes: Based on the at least one first piece of information, the at least one first updated lane centerline is updated to obtain at least one second updated lane centerline, and the at least one second updated lane centerline indicates the connection relationship between different first updated lane centerlines among the at least one first updated lane centerlines.

7. The method according to claim 1 or 2, characterized in that, The coordinate information is obtained based on the vehicle's position when it is traveling in the road environment.

8. The method according to claim 1 or 2, characterized in that, Both the coordinate information and the point cloud data are in the world coordinate system.

9. A lane line acquisition device, characterized in that, The device includes: The acquisition module is used to acquire coordinate information corresponding to lane lines in the road environment, wherein the lane lines include lane center lines and / or lane boundary lines, and the coordinate information includes the coordinates of position points on the lane lines; The acquisition module is also used to acquire point cloud data of the road environment; The processing module is used to obtain the updated lane lines based on the coordinate information and the point cloud data through a machine learning model.

10. The apparatus according to claim 9, characterized in that, The processing module is specifically used for: A projection operation is performed based on the coordinate information to obtain a first image, the first image including the position points on the lane line; A second image is obtained by performing a projection operation based on the point cloud data, and the position points in the second image correspond to the point cloud data. The first image and the second image are input into the machine learning model, and the updated lane lines are obtained through the machine learning model.

11. The apparatus according to claim 9 or 10, characterized in that, The machine learning model includes a feature extraction module and a feature processing module. The feature extraction module is used to perform feature extraction operations to obtain a first feature map. The feature processing module is used to obtain a second feature map based on the first feature map, and to generate the updated lane lines based on the second feature map, wherein the size of the second feature map is equal to the size of the first image, and the size of the feature map used in the process of the feature processing module generating the updated lane lines based on the second feature map remains unchanged.

12. The apparatus according to claim 11, characterized in that, The feature processing module includes at least one neural network layer used in the process of generating the updated lane lines based on the second feature map. The neural network layer is used to keep the size of the feature map unchanged and reduce the number of feature maps.

13. The apparatus according to claim 9 or 10, characterized in that, The lane lines include lane center lines. The processing module is configured to generate at least one first updated lane center line and at least one first piece of information through a machine learning model. The first piece of information indicates the area in the road environment where the number of lane center lines changes. The processing module is further configured to obtain the connection relationship between different first updated lane centerlines in the at least one first updated lane centerline based on the at least one first information.

14. The apparatus according to claim 13, characterized in that, The processing module is specifically used to update the at least one first updated lane centerline based on the at least one first information to obtain at least one second updated lane centerline, wherein the at least one second updated lane centerline indicates the connection relationship between different first updated lane centerlines among the at least one first updated lane centerlines.

15. The apparatus according to claim 9 or 10, characterized in that, The coordinate information is obtained based on the vehicle's position when it is traveling in the road environment.

16. The apparatus according to claim 9 or 10, characterized in that, Both the coordinate information and the point cloud data are in the world coordinate system.

17. A device, characterized in that, The method includes a processor coupled to a memory storing program instructions, which, when executed by the processor, implement the method of any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 8.

19. A computer program product, characterized in that, The computer program product includes a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 8.

20. A chip, characterized in that, The chip includes a processor for performing the steps of the method according to any one of claims 1 to 8.

Images