Road topology generation method, model training method, vehicle control method and equipment

Abstract

The embodiment of the invention provides a road topology generation method, a model training method, a vehicle control method and equipment, and relates to the field of automatic driving. The road topology generation method comprises the following steps: acquiring a feature map of a target road area based on environmental perception data of the target road area; inputting the feature map into a pre-generated target topology detection model to obtain a road topology output by the target topology detection model and corresponding to the target road area; wherein the road topology comprises a plurality of lane instances and a connection relation of the lane instances, the target topology detection model comprises a lane instance detection module and a topology generation module, the lane instance detection module is used for detecting elements in the feature map to obtain the lane instances, and the topology generation module is used for generating a topology; the topology generation module is used for generating a connection relationship corresponding to the plurality of lane instances.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and more specifically, to a road topology generation method, a model training method, a vehicle control method, and a device. Background Technology

[0002] In autonomous driving technology, road topology information is crucial for autonomous vehicle navigation and driving planning. Therefore, improving the accuracy of road topology generation has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, the present disclosure proposes a new technical solution for road topology generation.

[0004] According to a first aspect of the present disclosure, a road topology generation method is provided, the method comprising:

[0005] Based on the environmental perception data of the target road area, a feature map of the target road area is obtained;

[0006] The feature map is input into a pre-generated target topology detection model to obtain the road topology corresponding to the target road region output by the target topology detection model.

[0007] The road topology includes multiple lane instances and their connection relationships. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain the lane instances. The topology generation module is used to generate connection relationships corresponding to multiple lane instances. The lane instances include lane line instances and lane centerline instances. The connection relationships include a first connection relationship of multiple lane line instances and a second connection relationship of multiple lane centerline instances. The first connection relationship and the second connection relationship are generated by the same neural network unit in the topology generation module.

[0008] Optionally, the target topology detection model is a model generated after training based on lane line sample data and lane centerline sample data.

[0009] Optionally, the lane instance detection module includes a two-dimensional lane decoder and a three-dimensional lane decoder; the lane instance detection module is used to detect elements in the feature map to obtain the lane instance in the following manner:

[0010] Based on the two-dimensional lane decoder and the pre-set two-dimensional initial query vector, the two-dimensional features of the lane instance on the two-dimensional plane are obtained from the feature map;

[0011] The initial three-dimensional query vector of the lane instance in three-dimensional space is determined based on the two-dimensional features.

[0012] Based on the 3D lane decoder and the 3D initial query vector, the 3D features of the lane instance in 3D space are obtained from the feature map.

[0013] The three-dimensional features are projected into a three-dimensional space to obtain a three-dimensional lane instance;

[0014] The lane instance is determined based on the three-dimensional lane instance, the three-dimensional features, and the two-dimensional features.

[0015] Optionally, determining the initial three-dimensional query vector of the lane instance in three-dimensional space based on the two-dimensional features includes:

[0016] The two-dimensional features are projected onto the two-dimensional plane to obtain two-dimensional lane instances;

[0017] The three-dimensional initial query vector is determined based on the two-dimensional features, the two-dimensional lane instance, and the parameters of the target sensor; wherein the target sensor is a sensor that collects the environmental perception data.

[0018] Optionally, the two-dimensional initial query vector includes a line instance query vector for instance-level feature queries and a point instance query vector for pixel-level feature queries.

[0019] Optionally, the topology generation module is used to generate connection relationships corresponding to multiple lane instances in the following manner:

[0020] The two-dimensional and three-dimensional features of the lane instance are respectively projected onto the topological features to obtain two-dimensional topological features and three-dimensional topological features;

[0021] The three-dimensional lane instance is embedded and encoded to obtain the three-dimensional lane feature;

[0022] The three-dimensional lane features, the three-dimensional topological features, and the two-dimensional topological features are fused to obtain the fused features;

[0023] The fused features are input into the neural network unit in the topology generation module to obtain the connection relationship between multiple lane instances.

[0024] Optionally, the environmental perception data includes image data acquired based on a visual sensor and point cloud data acquired based on radar; the acquisition of a feature map of the target road area based on the environmental perception data of the target road area includes:

[0025] Image features are obtained by performing feature extraction on the image data;

[0026] Point cloud features are obtained by extracting features from the point cloud data;

[0027] The image features and the point cloud features are fused to obtain the feature map of the target road area from an overhead view.

[0028] According to a second aspect of the present disclosure, a model training method is provided, the method comprising:

[0029] Acquire lane line sample data and lane centerline sample data;

[0030] Based on the lane line sample data and the lane center line sample data, the preset topology detection model is trained to obtain the trained target topology detection model.

[0031] The target topology detection model is used to generate a road topology of a target road region based on an input feature map. The feature map is a feature map obtained based on environmental perception data of the target road region. The road topology includes multiple lane instances and the connection relationships of the lane instances. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain the lane instances. The topology generation module is used to generate connection relationships corresponding to multiple lane instances.

[0032] According to a third aspect of the present disclosure, a vehicle control method is provided, the method comprising:

[0033] Based on environmental perception data of the target road area, the road topology of the target road area is determined;

[0034] The vehicle's movement is controlled according to the road topology;

[0035] The road topology is obtained by inputting a feature map acquired based on the environmental perception data into a pre-generated target topology detection model. The road topology includes multiple lane instances and the connection relationships between the lane instances. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain the lane instances. The topology generation module is used to generate connection relationships corresponding to multiple lane instances. The lane instances include lane line instances and lane centerline instances. The connection relationships include a first connection relationship between multiple lane line instances and a second connection relationship between multiple lane centerline instances. The first connection relationship and the second connection relationship are generated by the same neural network unit in the topology generation module.

[0036] According to a fourth aspect of the present disclosure, an electronic device is provided, including a memory and a processor, the memory being configured to store computer instructions, and the processor being configured to invoke the computer instructions from the memory to perform a method as described in any one of the first, second, and / or third aspects.

[0037] According to a fifth aspect of the present disclosure, a vehicle is provided, including a memory and a processor, the memory being configured to store computer instructions, and the processor being configured to invoke the computer instructions from the memory to perform a method as described in any one of the first, second, and / or third aspects.

[0038] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first, second, and / or third aspects.

[0039] According to a seventh aspect of the present disclosure, a chip is provided, including a processing unit configured to perform a method as described in any one of the first, second, and / or third aspects.

[0040] Based on the road topology generation method provided in this disclosure, and based on the structure of the target topology detection model, the same neural network unit is used to generate the connection relationships of lane line instances and lane centerline instances. This method can make full use of the characteristics of lane lines and lane centerlines, improve detection efficiency and accuracy, and enhance the accuracy of the road topology detected by the target topology detection model.

[0041] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the present disclosure.

[0043] Figure 1 This is a schematic diagram of an intelligent connected system to which the methods provided in the embodiments of this disclosure can be applied.

[0044] Figure 2 It is based on Figure 1 The illustrated embodiment provides a schematic diagram of a vehicle.

[0045] Figure 3 This is a schematic diagram of a road topology provided in an embodiment of this disclosure.

[0046] Figure 4This is a schematic flowchart of a road topology generation method provided in an embodiment of this disclosure.

[0047] Figure 5 This is a schematic diagram of the structure of a target topology detection model provided in an embodiment of this disclosure.

[0048] Figure 6 This is a schematic diagram of the structure of a lane instance detection module provided in an embodiment of this disclosure.

[0049] Figure 7 This is a schematic diagram of the structure of a topology generation module provided in an embodiment of this disclosure.

[0050] Figure 8 This is a schematic flowchart of a model training method provided in an embodiment of this disclosure.

[0051] Figure 9 This is a schematic flowchart of a vehicle control method provided in an embodiment of this disclosure.

[0052] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0053] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0054] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0055] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0056] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0057] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0058] First, the application scenarios of the embodiments of this disclosure will be described.

[0059] Figure 1This is a schematic diagram of an intelligent connected system 100 to which the methods provided in the embodiments of this disclosure can be applied. Figure 1 As shown, the intelligent connected system 100 may include: vehicle 101, server 102, and user terminal 103.

[0060] In some examples, vehicle 101 may be a vehicle equipped with autonomous driving capabilities. Autonomous driving, also known as driverless or intelligent driving, refers to a vehicle capable of performing driving tasks such as environmental perception, decision-making, planning, and control execution. The levels of autonomous driving can be classified according to the vehicle intelligence grading standards established by the Society of Automotive Engineers (SAE), for example, L0 is manual driving, L1 is driver assistance, L2 is partial autonomous driving, L3 is conditional autonomous driving, L4 is highly automated driving, and L5 is fully automated driving. The above classification of autonomous driving levels is merely illustrative, and this disclosure does not limit the classification standards and levels of autonomous driving.

[0061] In some examples, server 102 can be a single server or a distributed server cluster consisting of multiple servers, and its deployment method can include local servers or cloud servers. Server 102 can communicate with vehicle 101 and / or user terminal 103 via a communication network, providing various services to vehicle 101 and / or user terminal 103. For example, the server can receive perception data sent by the vehicle and provide services such as high-precision maps, data analysis, and decision planning for the vehicle. Alternatively, the server can receive query commands or control commands sent by the user terminal and provide corresponding services to the user.

[0062] In some examples, the user terminal 103 can be any form of electronic device that provides services to the user, such as a personal computer, laptop, smart tablet, smartphone, smart wearable device, etc. The user can interact with the vehicle or server through the human-machine interface terminal configured in the vehicle 101, or through the user terminal 103. For example, the user can query the vehicle's status and / or parameters, or control the vehicle to perform set tasks and / or modify configuration parameters. The user terminal runs an application based on the intelligent connected system to achieve interaction with the vehicle or server. This application can be a local application, a web application, or a mini-program, etc., without limitation.

[0063] In some examples, the aforementioned application running on the user terminal can provide authentication or authorization services to the user. The user who is successfully authenticated and granted the corresponding permissions can query and / or control the vehicle within the scope of the granted permissions.

[0064] Vehicle 101, server 102, and user terminal 103 can communicate via a communication link provided by communication network 104. This communication network 104 can include one or more networks of any type, such as the Internet, Local Area Network (LAN), Wide Area Network (WAN), Virtual Private Network (VPN), Public Switched Telephone Network (PSTN), satellite communication network, Wi-Fi, 2G, 3G, 4G, 5G, 6G, NB-IoT, eMTC, infrared, Bluetooth, NFC, or a combination of these networks. The communication networks between vehicle 101 and server 102, between user terminal 103 and server 102, and between user terminal 103 and vehicle 101 can be the same or different.

[0065] It should be noted that, Figure 1 The structure of the intelligent connected system 100 shown is merely illustrative. The intelligent connected system in this embodiment is not limited to the above structure and may include more or fewer devices as needed, and the devices may be combined or split. For example, the intelligent connected system may not include user terminals and / or servers; as another example, user terminals and servers may be deployed together.

[0066] Figure 2 It is based on Figure 1 The illustrated embodiment provides a schematic diagram of a vehicle 101. (As shown) Figure 2 As shown, the vehicle 101 may include a sensing component 1011, a computing platform 1012, an execution component 1013, etc. The sensing component 1011, the computing platform 1012, and the execution component 1013 may be connected via a bus or other means.

[0067] In some examples, the perception component 1011 can be used to collect information about the vehicle itself or its external environment. The perception component 1011 may include at least one of a visual sensing unit, radar, a positioning and navigation unit, an inertial measurement unit (IMU), or other sensing units. The visual sensor unit may include one or more cameras, the radar may include at least one of lidar, millimeter-wave radar, ultrasonic radar, or other radars, and the positioning and navigation unit may include at least one of a GPS system, a BeiDou system, or other global positioning systems.

[0068] In some examples, the computing platform 1012 may include a computing-capable device for processing the perception information collected by the perception component 1011 to obtain control information, and sending corresponding control commands to the execution component 1013 to cause the execution component 1013 to perform corresponding actions, thereby achieving control of the vehicle 101. For example, the computing platform 1012 can perform Simultaneous Localization and Mapping (SLAM), path planning, and behavior decision-making on the vehicle, thereby achieving autonomous vehicle control. The computing platform 1012 may include at least one processor and at least one memory, wherein each processor can individually or jointly execute instructions stored in the memory to implement the methods provided in the embodiments of this disclosure. The processor in this disclosure embodiment may include at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), Tensor Processing Unit (TPU), Data Processing Unit (DPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), System on Chip (SOC), Application Specific Integrated Circuit (ASIC), Microcontroller Unit (MCU), or other processors. The memory may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), Magnetic Storage, Flash Memory, Disk, or Optical Disk. In addition to storing instructions, the memory may also store data, such as high-precision maps, route information, vehicle position, direction, speed, etc. Data stored in memory can be accessed and used by the processor.

[0069] In some examples, the vehicle's computing platform can perform computing tasks independently or communicate with a server to complete computing tasks. For example, the vehicle's computing platform can cooperate with a server to complete the corresponding computing tasks.

[0070] The computing platform 1012 can be installed in the vehicle 101. Some or all of the computing platform 1012 can also be installed in the server corresponding to the vehicle. For example, some functions of the computing platform 1012 with high real-time requirements can be installed in the vehicle, while other functions with lower real-time requirements can be installed in the server corresponding to the vehicle.

[0071] In some examples, the execution component 1013 is used to perform corresponding actions based on the control of the computing platform 1012, enabling the vehicle 101 to complete the movement task. The execution component 1013 may include, for example, a power component, a braking component, a transmission component, a steering component, etc.

[0072] It should be noted that, Figure 2 The structure of vehicle 101 shown is merely illustrative. The vehicle in this embodiment is not limited to the above structure and may include more or fewer components as needed. The devices may also be combined or separated. For example, the vehicle may not include the aforementioned computing platform. Furthermore, the vehicle may also include communication components, interface components, multimedia components, input components, output components, etc.

[0073] In autonomous driving technology, road topology information is crucial for the autonomous navigation and driving planning of self-driving vehicles. Road topology generation, also known as road network prediction, requires perception and reasoning about the road environment to extract multiple lane centerlines and their connections. During autonomous driving, the vehicle can control its movement based on the lane centerlines and their connections within the road topology. Figure 3 This is a schematic diagram of a road topology provided in an embodiment of this disclosure. For example... Figure 3 The road topology shown contains two types of elements: lane lines and lane center lines. Solid lines represent lane lines (also known as lane boundary lines or lane dividing lines), while dashed lines represent lane center lines. Lane lines can be painted lines on the road, such as solid white lines, dashed white lines, or double yellow lines. Lane lines are used to separate different driving lanes, ensuring orderly traffic flow. Lane center lines can be virtual or physical lines within a lane used to identify the center position of the lane. For example, a lane center line can be a vectorized representation of a lane in a high-definition map, used to represent the road topology. Lane center lines can help vehicles stay within their lanes and serve as reference points for navigation and autonomous driving systems. In autonomous driving, lane center lines often serve as the carrier of the driving trajectory because this method maximizes the use of map information, thereby reducing real-time computation.

[0074] In some examples, there can be multiple lane centerlines, and the road topology can also include the connection relationships between multiple lane centerlines. These connections can be used to indicate the connectivity between lane centerlines. Furthermore, since lanes are directional, the connection relationship can also include the connection direction between two lane centerlines, distinguishing the sequential order of two connected lane centerlines in the driving direction. For example, this connection relationship can be represented by an arrow.

[0075] It should be noted that, Figure 3 This is merely an example; the elements in a road topology can be more or fewer. For example, a road topology can also include static elements such as traffic lights, traffic signs, and obstacles. Or, for example, a road topology can only contain lane centerlines.

[0076] In related technologies, lane centerlines can be detected based on images captured of a target road area. However, because the image input from a purely visual modality lacks physical location and geometric information about the road structure, the accuracy of lane centerline detection is low, especially in complex road scenarios where inaccurate detection becomes a problem. Current road topology inference methods are often limited by the accuracy of centerline detection; low accuracy in lane centerline detection also leads to low accuracy in road topology generation, failing to meet the needs of autonomous driving.

[0077] To address the problems in related technologies, this disclosure provides a method for generating road topology. Figure 4 This is a schematic flowchart of a road topology generation method provided in an embodiment of this disclosure. The road topology generation method can be... Figure 1 The vehicle and / or server shown are executed. For example... Figure 4 As shown, the road topology generation method of this embodiment may include the following steps S410 to S420.

[0078] Step S410: Based on the environmental perception data of the target road area, obtain the feature map of the target road area.

[0079] The target road area can be a spatial scene area, which may include roads and lanes on those roads, as well as traffic facilities within the spatial scene area. For example, the target road area can be a road with physical lanes, or an intersection area with various traffic elements such as physical lanes, turning waiting areas, traffic lights, and traffic signs. Physical lanes refer to lanes that actually exist on the road and are marked by physical means such as lane lines and road signs. A road typically includes multiple lanes, such as straight-ahead lanes, left-turn lanes, and right-turn lanes.

[0080] In some instances, the target road area can be an area detectable by sensors during vehicle movement. The environmental perception data can include real-world scan data of the target road area. Here, "real-world" in real-world scan data refers to the real world or a scene in the real world; that is, the target road area, as the object of data collection, exists in the real world. The term "scan" in real-world scan data indicates that the sensors have scanned the target road area, but does not limit the specific method by which the sensors collect data. The real-world scan data can include at least one of image data collected by a visual sensor (such as a camera) and point cloud data collected by radar of the target road area.

[0081] For example, the environmental perception data mentioned above may include image data collected based on visual sensors and point cloud data collected based on radar. In this step, image features can be extracted from the image data, and point cloud features can be extracted from the point cloud data. The image features and point cloud features are then fused to obtain a feature map of the target road area.

[0082] In this way, by using multimodal input of image data and point cloud data, more features of the target road area can be extracted, improving the model's ability to detect lane centerlines and enhancing the accuracy of lane centerline detection.

[0083] The image features can be visual image features from a bird's-eye view, and the point cloud features can also be point cloud features from a bird's-eye view. Image features and point cloud features can be fused based on a feature attention mechanism to obtain a feature map from this bird's-eye view. Since the bird's-eye view is a global view, the feature map of the target road area from this view can clearly show the layout and structural features of the entire target road area. Therefore, generating road topology based on the feature map of the target road area from the bird's-eye view improves the efficiency and accuracy of topology generation. It should be noted that the feature map can also be a projection of the target road area from other set views, and its projection method can be orthographic projection or perspective projection, etc., which is not limited in this embodiment.

[0084] In some examples, the aforementioned visual sensor may include one or more cameras mounted on the vehicle, and the aforementioned image data may be images captured by the cameras on the vehicle in the target road area. The vehicle may have multiple cameras positioned at different locations, for example, cameras on the front, left, right, and rear sides of the vehicle, to simultaneously acquire image data of the target road area through multiple cameras. In some examples, the aforementioned radar may include a lidar mounted on the vehicle, and the aforementioned point cloud data may be point cloud data acquired by the vehicle's lidar.

[0085] Step S420: Input the feature map into the pre-generated target topology detection model to obtain the road topology corresponding to the target road area output by the target topology detection model.

[0086] The road topology can include multiple lane instances and the connection relationships between lane instances. The target topology detection model can include a lane instance detection module and a topology generation module. The lane instance detection module can be used to detect elements in the feature map to obtain lane instances, and the topology generation module can be used to generate the connection relationships corresponding to multiple lane instances.

[0087] In some examples, the lane instances detected by the lane instance detection module described above may include lane line instances and lane centerline instances. The connection relationships of these lane instances may include a first connection relationship among multiple lane line instances and a second connection relationship among multiple lane centerline instances. The first and second connection relationships may be generated by the same neural network unit in the topology generation module described above.

[0088] The neural network unit can be a CNN / RNN or a Multilayer Perceptron (MLP), which can be a feedforward neural network model. An MLP can contain at least three layers: an input layer, one or more hidden layers, and an output layer. Except for the input nodes, each node can be a neuron (or processing unit) with a non-linear activation function. MLPs can be trained using backpropagation supervised learning techniques and can be combined with gradient descent algorithms for training.

[0089] It should be noted that the same neural network unit can be the same neural network unit, or it can be two or more neural network units with completely identical structure and parameters. The parameters of these two or more neural network units can be shared and updated in real time.

[0090] Thus, since both the first connection relationship of lane line instances and the second connection relationship of lane centerline instances are generated by the same neural network unit, and the lane centerline is a virtual line that cannot provide real-world visual information, while lane lines are real-world scan data obtained through real-world scanning, which can provide fine-grained visual information such as the precise position, shape, size, color, or texture of the lane lines. Therefore, based on the structure of the target topology detection model described above, using the same neural network unit to generate the connection relationships of lane line instances and lane centerline instances can fully utilize the respective features of lane lines and lane centerlines, improving detection efficiency and accuracy, and enhancing the accuracy of the road topology detected by the target topology detection model.

[0091] In some embodiments of this disclosure, the aforementioned road topology can be stored and displayed using a high-precision map. For example, a high-precision map containing the aforementioned road topology can be displayed via an in-vehicle terminal or other electronic device. This allows users to conveniently view accurate road topologies.

[0092] It should be noted that the output of the target topology detection model can differ between the model training and model inference phases. For example, during model training, it can output lane line instances and lane centerline instances, the first connection relationships of multiple lane line instances, and the second connection relationships of multiple lane centerline instances, enabling joint training based on lane line sample data and lane centerline sample data. During model inference, the output can be tailored to the needs of autonomous driving. For instance, it can output only lane centerline instances and the second connection relationships of multiple lane centerline instances; it can output only lane line instances and the first connection relationships of multiple lane line instances; or it can output all lane line instances, lane centerline instances, the first connection relationships of multiple lane line instances, and the second connection relationships of multiple lane centerline instances.

[0093] In some examples, the target topology detection model described above can be a model generated after joint training based on lane line sample data and lane centerline sample data.

[0094] Both the lane line sample data and the lane centerline sample data can be labeled data, which can be used for supervised training of the target topology detection model. For example, the lane line sample data can label the location, type, and connection relationship of multiple lane lines; the lane line sample data can label the location, direction, type, and connection relationship of multiple lane centerlines.

[0095] In one implementation, the lane line sample data may include lane lines extracted and labeled from environmental perception sample data. This environmental perception sample data may be image data and / or point cloud data obtained by real-world scanning of the target road area using the vehicle's vision sensors or other sensors. The lane centerline in the lane centerline sample data may be a virtual line, such as a virtual line located at the center of the lane, either manually labeled or automatically generated. Since the lane centerline is a virtual line, it cannot provide real-world visual information, and using it alone for training would result in insufficient feature information extracted by the model, leading to underfitting. However, the lane line sample data contains real-world scan data of the lane lines, which can provide fine-grained visual information such as the precise location, shape, size, color, or texture of the lane lines. Thus, after jointly training the target topology detection model based on the lane line sample data and the lane centerline sample data, this fine-grained visual information can be utilized, reducing the underfitting problem caused by the lack of visual information in the lane centerline sample data and improving the detection accuracy of the trained target topology detection model.

[0096] Figure 5 This is a schematic diagram of the structure of a target topology detection model provided in an embodiment of this disclosure. For example... Figure 5 As shown, the target topology detection model 500 in this embodiment may include a lane instance detection module 510 and a topology generation module 520. Wherein:

[0097] The lane instance detection module 510 can receive a feature map input from the feature extraction module 530, detect elements in the feature map, and output lane instances (e.g., lane line instances and / or lane centerline instances).

[0098] The topology generation module 520 can receive lane instances output by the lane instance detection module 510 and generate connection relationships between multiple lane instances.

[0099] In this way, based on multiple lane instances and the connection relationships between multiple lane instances, the road topology 540 of the target road area can be generated.

[0100] In some examples, the feature extraction module 530 may include an image feature extraction module 531, a point cloud feature extraction module 532, and a feature fusion module 533.

[0101] The image feature extraction module 531 can extract image features from the input image data. For example, the image data can be multi-view image data. The image feature extraction module 531 can sequentially input the multi-view image data into a residual network (e.g., ResNet50) and a feature pyramid network (FPN) to obtain multi-view image features. Then, the LSS (Lift, Splat, Shoot) algorithm is used to aggregate the multi-view features to obtain the overhead view image features, which can also be called visual BEV features. ResNet50 is a deep residual network capable of extracting rich feature representations from images. The feature pyramid network (FPN) provides feature fusion technology, which can combine feature maps of different scales to enhance the model's ability to recognize objects of different sizes. The LSS algorithm can lift (Lift) the multi-view image features from the two-dimensional plane to three-dimensional space, then perform voxel pooling (Splat) on these features in three-dimensional space, and finally project (Shoot) these features onto the overhead view (BEV) space to obtain the overhead view image features. It should be noted that the image feature extraction module 531 can obtain the image features in other ways. For example, it can first extract the perspective view (PV) image features of the image data, and then convert the PV image features into the overhead view image features.

[0102] The point cloud feature extraction module 532 can extract features from the input point cloud data to obtain point cloud features. For example, the point cloud data may include point clouds in three-dimensional space. Each point in the three-dimensional space contains positional information (e.g., x, y, z coordinates) and other attributes (e.g., color, intensity). The point cloud feature extraction module 532 can first use a deep learning network (e.g., VoxelNet) to voxelize the point cloud data, that is, convert the point cloud data into voxel form for processing. A voxel is a pixel in three-dimensional space, similar to a pixel in a two-dimensional image, but a voxel is a cube, while a pixel is a square. After voxelization, a three-dimensional sparse convolutional network can further extract richer features from the voxelized data, thus obtaining dense point cloud features. These point cloud features can be point cloud features viewed from an overhead perspective, or they can be called point cloud BEV features.

[0103] The feature fusion module 533 can fuse image features and point cloud features to obtain a feature map of the target road region. For example, the feature fusion module 533 can fuse image features and point cloud features based on a feature attention mechanism (e.g., SEBlock) to obtain a fused feature map. For instance, this feature attention mechanism of SEBlock can be used to process image features and point cloud features separately, learning the importance of each feature channel. Finally, the processed image features and point cloud features are fused to obtain a fused feature map. This fused feature map can simultaneously contain the texture and color information of the image and the spatial geometric information of the point cloud, providing a richer feature representation for subsequent lane instance recognition and improving the accuracy of lane instance recognition. Optionally, this...

[0104] Figure 6 This is a schematic diagram of the structure of a lane instance detection module provided in an embodiment of this disclosure. Figure 6 As shown, the lane instance detection module 510 may include a two-dimensional lane decoder 511 and a three-dimensional lane decoder 512; the lane instance detection module detects elements in the feature map to obtain lane instances through the following steps S11 to S15:

[0105] Step S11: Based on the two-dimensional lane decoder and the pre-set two-dimensional initial query vector, query the two-dimensional features of the lane instance on the two-dimensional plane from the feature map.

[0106] In some examples, the two-dimensional initial query vector may include a line instance query vector (ins query) for instance-level feature querying and a point instance query vector (pts query) for pixel-level feature querying. The line instance query vector (ins query) focuses on instance-level feature extraction, enabling vectorized instance feature extraction, while the point instance query vector (pts query) focuses on image analysis and understanding at the pixel level. Both the line and point instance query vectors can be randomly generated during initialization. Optionally, the line instance query vector (ins query) and the point instance query vector (pts query) can be combined (e.g., added or multiplied) to obtain the two-dimensional initial query vector (e.g., 2D query).

[0107] In some examples, the two-dimensional lane decoder may include a self-attention unit, a deformable attention unit, and a feed-forward network (FFN). After inputting the two-dimensional initial query vector and the feature map into the two-dimensional lane decoder for decoding, the two-dimensional features of the lane instance on the two-dimensional plane can be obtained.

[0108] Step S12: Determine the initial three-dimensional query vector of the lane instance in three-dimensional space based on the two-dimensional features.

[0109] In some examples, two-dimensional features can be directly used as the initial query vector for three dimensions.

[0110] In other examples, two-dimensional lane instances can be obtained by projecting the two-dimensional features onto a two-dimensional plane; and a three-dimensional initial query vector can be determined based on the two-dimensional features, the two-dimensional lane instances, and the parameters of the target sensor. Optionally, this instance feature projection can be implemented using a multilayer perceptron (MLP). This instance feature projection can automatically adjust the region of interest based on the input two-dimensional features to extract two-dimensional region features related to the lane instances, thereby improving the accuracy and effectiveness of feature representation in the two-dimensional lane instances.

[0111] The target sensor may include a sensor that collects environmental perception data, such as a vision sensor. The parameters of the target sensor may include intrinsic and / or extrinsic parameters. The intrinsic parameters can be used to describe the internal properties of the target sensor, such as focal length, optical center coordinates, distortion coefficients, etc. The extrinsic parameters can be used to describe the position and orientation of the target sensor in the world coordinate system, such as the rotation matrix and / or translation vector of the target sensor relative to the external coordinate system.

[0112] Furthermore, in the Cam-Aware Embedding module, two-dimensional features are combined with the intrinsic and extrinsic parameters of the two-dimensional lane instance and the target sensor to perform a position embedding transformation, resulting in a three-dimensional initial query vector. This three-dimensional initial query vector is used for three-dimensional lane information detection. This position embedding transformation method can be applied to multi-view applications. Figure 3 Position Embedding Transformation for Multi-View 3D Object Detection (PETR) can encode the position information of three-dimensional coordinates into two-dimensional features to generate a three-dimensional initial query vector for three-dimensional position perception and querying.

[0113] Step S13: Based on the 3D lane decoder and the 3D initial query vector, obtain the 3D features of the lane instance in the 3D space from the feature map.

[0114] In some examples, the 3D lane decoder may include a self-attention unit, a deformable attention unit, and a feedforward neural network (FFN). After inputting the initial 3D query vector and feature map into the 3D lane decoder for decoding, the 3D features of the lane instance in 3D space can be obtained.

[0115] Step S14: Project the three-dimensional features into the three-dimensional space to obtain a three-dimensional lane instance.

[0116] The instance feature projection can be implemented based on a multilayer perceptron (MLP). This instance feature projection can automatically adjust the region of interest based on the input 3D features to extract 3D spatial features related to lane instances, thereby improving the accuracy and effectiveness of feature representation in 3D lane instances.

[0117] Step S15: Determine lane instances based on 3D lane instances, 3D features, and 2D features.

[0118] For example, the three-dimensional lane instance can be used as the lane instance, or the three-dimensional lane instance, three-dimensional features, and two-dimensional features can all be used as attribute information of the lane instance.

[0119] In this way, the lane instance detection module 510 can detect lane line instances or lane center line instances, which can improve the accuracy of lane instance detection.

[0120] In some examples, the target topology detection model 500 may include two lane instance detection modules 510, such as a first lane instance detection module and a second lane instance detection module. The first lane instance detection module can be used to detect lane line instances, and the second lane instance detection module can be used to detect lane centerline instances. Optionally, the first lane instance detection module and the second lane instance detection module may have the same structure, but different trained parameters.

[0121] In some examples, during the model training phase, the target topology detection model 500 includes a first lane instance detection module and a second lane instance detection module for joint training based on lane line sample data and lane centerline sample data. During the model inference phase, the target topology detection model 500 may include only one or two lane instance detection modules. This reduces the structural complexity of the target topology detection model and lowers the difficulty of model deployment.

[0122] Figure 7 This is a schematic diagram of the structure of a topology generation module provided in an embodiment of this disclosure. For example... Figure 7As shown, the topology generation module 520 may include a neural network unit 521, which may include a multilayer perceptron (MLP).

[0123] In some examples, the topology generation module 520 generates the connection relationships corresponding to multiple lane instances through the following steps S21 to S24:

[0124] Step S21: Project the two-dimensional and three-dimensional features of the lane instance into topological features to obtain two-dimensional and three-dimensional topological features.

[0125] This topological feature projection can be implemented based on a multilayer perceptron (MLP). In this projection, the topological relationships of interest can be automatically adjusted based on the input features to extract features related to the lane instances, thus obtaining two-dimensional topological features corresponding to the two-dimensional features and three-dimensional topological features corresponding to the three-dimensional features.

[0126] Step S22: After embedding and encoding the three-dimensional lane instances, the three-dimensional lane features are obtained.

[0127] For example, 3D lane instances can be embedded and encoded into 3D lane features using a multilayer perceptron (MLP).

[0128] Step S23: Fuse the three-dimensional lane features, three-dimensional topological features and two-dimensional topological features to obtain fused features.

[0129] For example, the fused feature can be obtained by adding the three-dimensional lane features, the three-dimensional topological features, and the two-dimensional topological features. Alternatively, the three-dimensional topological features and the two-dimensional topological features can be added first to obtain the preliminary fused feature, and then the three-dimensional lane features can be embedded into the preliminary fused feature to obtain the fused feature.

[0130] Step S24: Input the fused features into the neural network unit in the topology generation module to obtain the connection relationship between multiple lane instances.

[0131] The neural network unit may include a multilayer perceptron (MLP).

[0132] In some examples, multiple lane instances can be paired up to generate an initial adjacency matrix. Based on this neural network unit, the fusion features are predicted, and the connection relationship between each pair of lane instances in the initial adjacency matrix is ​​assigned a value to obtain the final target adjacency matrix representing the connection relationship, thereby generating a road topology containing lane instances and their connection relationships.

[0133] In some examples, this topology generation module can be used for the first connectivity relationships of multiple lane line instances, and / or, the second connectivity relationships of multiple lane centerline instances. The generation of the first and second connectivity relationships can use the same neural network unit (e.g., MLP).

[0134] It should be noted that, Figure 7 Although two neural network units are shown, these two neural network units can be the same neural network unit, or they can be two or more neural network units with the same structure and parameters. The parameters of these two or more neural network units can be shared and updated in real time, that is, they share weights.

[0135] Figure 8 This is a schematic flowchart of a model training method provided in an embodiment of this disclosure. Figure 8 As shown, the method may include steps S810 to S820.

[0136] Step S810: Obtain lane line sample data and lane centerline sample data.

[0137] Step S820: Based on lane line sample data and lane centerline sample data, train the preset topology detection model to obtain the trained target topology detection model.

[0138] The target topology detection model is used to generate the road topology of the target road area based on the input feature map. The feature map is a feature map obtained based on the environmental perception data of the target road area. The road topology includes multiple lane instances and the connection relationship of the lane instances. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain lane instances, and the topology generation module is used to generate the connection relationship corresponding to multiple lane instances.

[0139] In this way, a target topology detection model is generated through joint training based on lane line sample data and lane centerline sample data. This model can then be used to detect the road topology of the target road region from the input feature map. Because the model training utilizes tangible lane line sample data from visual features to train and optimize the target topology detection model, the accuracy of lane centerline detection is improved, as is the accuracy of the generated road topology.

[0140] In some examples, the structure of the preset topology detection model can be the same as the structure of the target topology detection model, see reference. Figure 5-7 The descriptions in the illustrated embodiments will not be repeated here.

[0141] In other examples, the preset topology detection model may include two lane instance detection modules, such as a first lane instance detection module and a second lane instance detection module, while the target topology detection may include only one lane instance detection module, such as only the first lane instance detection module or the second lane instance detection module.

[0142] In some examples, during the training of the target topology detection model, one or more of the following loss functions (one through three) can be used:

[0143] Loss function 1: Focal loss. This focal loss can be used to supervise the accuracy of the classification of lane instances (such as lane line instances or lane centerline instances) detected by the lane instance detection module.

[0144] Loss function 2: Absolute value loss (L1 loss). This L1 loss can be used to supervise the accuracy of the vector points of lane instances detected by the lane instance detection module.

[0145] The third loss function is the cross-entropy loss function, which can be used to supervise the accuracy of the connections generated by the topology generation module. For example, this cross-entropy loss function can be used to measure the difference between the model's predicted connections and the actual connections.

[0146] Thus, supervised training based on the above loss function can improve training efficiency and the accuracy of model predictions.

[0147] It should be noted that other loss functions from related technologies can be used during the training of the target topology detection model, and this disclosure does not limit this.

[0148] Figure 9 This is a schematic flowchart of a vehicle control method provided in an embodiment of this disclosure. The vehicle control method can be... Figure 1 The vehicle and / or server shown are executed. For example... Figure 9 As shown, the road topology generation method in this embodiment may include:

[0149] Step S910: Determine the road topology of the target road area based on the environmental perception data of the target road area.

[0150] Step S920: Control vehicle movement according to road topology.

[0151] The road topology can be obtained by inputting a feature map acquired based on environmental perception data into a pre-generated target topology detection model. The road topology includes multiple lane instances and the connection relationships between lane instances. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain lane instances, and the topology generation module is used to generate the connection relationships corresponding to multiple lane instances. The target topology detection model is a model generated after joint training based on lane line sample data and lane centerline sample data.

[0152] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. For example... Figure 10 As shown, the electronic device 1000 may include a memory 1010 and a processor 1020. The memory 1010 may be used to store computer instructions, and the processor 1020 may be used to retrieve computer instructions from the memory 1010 to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. The processor may be one or more, and the one or more processors may execute instructions individually or jointly. The memory may also be one or more, and the one or more memories may store the aforementioned computer instructions individually or jointly. Optionally, the electronic device may be... Figure 1 Servers and / or vehicles in the system.

[0153] This disclosure also provides a vehicle that may include a memory and a processor. The memory may be used to store computer instructions, and the processor may be used to retrieve the computer instructions from the memory to perform all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. The processor may be one or more processors, which may execute instructions individually or jointly. Similarly, the memory may be one or more memories, which may store the aforementioned computer instructions individually or jointly.

[0154] The vehicle in the foregoing embodiments of this disclosure may be an electric vehicle, a hybrid vehicle, a fuel cell vehicle, or other types of vehicle. The vehicle may be an autonomous vehicle or a non-autonomous vehicle. For example, the vehicle provided in this embodiment may be... Figure 1 or Figure 2 The vehicle shown.

[0155] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods in the foregoing embodiments of this disclosure. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto; it may also be a temporary storage medium.

[0156] This disclosure also provides a chip that may include a processing unit, which can be used to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure. The chip may be in the form of an Application-Specific Integrated Circuit (ASIC), a System-on-Chip (SOC), a Field-Programmable Gate Array (FPGA), etc., and this embodiment is not limited to this. Optionally, the chip may further include a storage unit, which can be used to store computer instructions. The processing unit can be used to retrieve the computer instructions from the storage unit to execute all or part of the steps of any of the methods in the foregoing embodiments of this disclosure.

[0157] This disclosure also provides a computer program product that may include a computer program that, when executed by a processor, can implement any of the methods described in the foregoing embodiments of this disclosure.

[0158] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0159] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0160] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0161] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0162] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0163] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0164] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0165] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0166] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for generating road topology, characterized in that, The method includes: Based on the environmental perception data of the target road area, a feature map of the target road area is obtained; The feature map is input into a pre-generated target topology detection model to obtain the road topology corresponding to the target road region output by the target topology detection model. The road topology includes multiple lane instances and their connection relationships. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain the lane instances. The topology generation module is used to generate connection relationships corresponding to multiple lane instances. The lane instances include lane line instances and lane centerline instances. The connection relationships include a first connection relationship of multiple lane line instances and a second connection relationship of multiple lane centerline instances. The first connection relationship and the second connection relationship are generated by the same neural network unit in the topology generation module.

2. The method according to claim 1, characterized in that, The target topology detection model is a model generated after training based on lane line sample data and lane centerline sample data.

3. The method according to claim 1, characterized in that, The lane instance detection module includes a two-dimensional lane decoder and a three-dimensional lane decoder; the lane instance detection module is used to detect elements in the feature map to obtain the lane instance in the following manner: Based on the two-dimensional lane decoder and the pre-set two-dimensional initial query vector, the two-dimensional features of the lane instance on the two-dimensional plane are obtained from the feature map; The initial three-dimensional query vector of the lane instance in three-dimensional space is determined based on the two-dimensional features. Based on the 3D lane decoder and the 3D initial query vector, the 3D features of the lane instance in 3D space are obtained from the feature map. The three-dimensional features are projected into a three-dimensional space to obtain a three-dimensional lane instance; The lane instance is determined based on the three-dimensional lane instance, the three-dimensional features, and the two-dimensional features.

4. The method according to claim 3, characterized in that, Determining the initial 3D query vector of the lane instance in 3D space based on the 2D features includes: The two-dimensional features are projected onto the two-dimensional plane to obtain two-dimensional lane instances; The three-dimensional initial query vector is determined based on the two-dimensional features, the two-dimensional lane instance, and the parameters of the target sensor; wherein the target sensor is a sensor that collects the environmental perception data.

5. The method according to claim 3, characterized in that, The two-dimensional initial query vector includes a line instance query vector for instance-level feature queries and a point instance query vector for pixel-level feature queries.

6. The method according to claim 3, characterized in that, The topology generation module is used to generate connection relationships corresponding to multiple lane instances in the following manner: The two-dimensional and three-dimensional features of the lane instance are respectively projected onto the topological features to obtain two-dimensional topological features and three-dimensional topological features; The three-dimensional lane instance is embedded and encoded to obtain the three-dimensional lane feature; The three-dimensional lane features, the three-dimensional topological features, and the two-dimensional topological features are fused to obtain the fused features; The fused features are input into the neural network unit in the topology generation module to obtain the connection relationship between multiple lane instances.

7. The method according to any one of claims 1 to 6, characterized in that, The environmental perception data includes image data acquired based on visual sensors and point cloud data acquired based on radar; the environmental perception data based on the target road area, obtaining the feature map of the target road area, includes: Image features are obtained by performing feature extraction on the image data; Point cloud features are obtained by extracting features from the point cloud data; The image features and the point cloud features are fused to obtain the feature map of the target road area from an overhead view.

8. A model training method, characterized in that, The method includes: Acquire lane line sample data and lane centerline sample data; Based on the lane line sample data and the lane center line sample data, the preset topology detection model is trained to obtain the trained target topology detection model. The target topology detection model is used to generate a road topology of a target road region based on an input feature map. The feature map is a feature map obtained based on environmental perception data of the target road region. The road topology includes multiple lane instances and the connection relationships of the lane instances. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain the lane instances. The topology generation module is used to generate connection relationships corresponding to multiple lane instances.

9. A vehicle control method, characterized in that, The method includes: Based on environmental perception data of the target road area, the road topology of the target road area is determined; The vehicle's movement is controlled according to the road topology; The road topology is obtained by inputting a feature map acquired based on the environmental perception data into a pre-generated target topology detection model. The road topology includes multiple lane instances and the connection relationships between the lane instances. The target topology detection model includes a lane instance detection module and a topology generation module. The lane instance detection module is used to detect elements in the feature map to obtain the lane instances. The topology generation module is used to generate connection relationships corresponding to multiple lane instances. The lane instances include lane line instances and lane centerline instances. The connection relationships include a first connection relationship between multiple lane line instances and a second connection relationship between multiple lane centerline instances. The first connection relationship and the second connection relationship are generated by the same neural network unit in the topology generation module.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method as described in any one of claims 1 to 9.

11. A vehicle, characterized in that, It includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to retrieve the computer instructions from the memory to perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 9.

13. A chip, characterized in that, The chip includes a processing unit for performing the method as described in any one of claims 1 to 9.

Images