Natural language description method for drivable road network based on road points

Abstract

The invention discloses a natural language description method for a drivable road network based on road points, and belongs to the technical field of traffic simulation based on deep learning. The problem that in the prior art, a traditional road network description method is difficult to directly obtain the position of a vehicle and the surrounding real-time road network information from OpenDrive and apply to driving decision making is solved. The method comprises the following steps: S1, acquiring global static information of a map, wherein the global static information comprises all road points and pedestrian crossings; s2, acquiring all road points around the vehicle by using the KD-Tree; and S3, according to the content to be described and the attribute value of the road point, generating a natural language description of the drivable road network. According to the method, the precision and reliability of driving decision making are effectively improved, the complete and reliable natural language description for the driving road network around the vehicle is generated, and the method can be applied to generation of road network information in a natural language form.

Description

Technical Field

[0001] The present invention relates to a natural language description method, and in particular to a natural language description method for a drivable road network based on waypoints, belonging to the technical field of traffic simulation based on deep learning. Background Art

[0002] In recent years, with the rapid development of LLM (Large Language Model), LLM has been widely applied to various task scenarios. In addition to traditional translation, question answering, and text generation, more and more researchers have begun to focus on the language understanding and logical reasoning capabilities of LLM and have tried to apply LLM to tasks that require a relatively high human-like level. Autonomous driving is one of them. Previous autonomous driving algorithms made driving decisions based on image data, lidar point cloud data, vehicle trajectory data, etc., and were called data-driven autonomous driving. After the emergence of LLM, some researchers began to use it for autonomous driving decision-making, that is, providing a driving scenario description for LLM and using the reasoning ability of LLM to imitate human thinking and make decisions to achieve a more human-like and more interpretable autonomous driving system. Since most of the training data of LLM are various forms of human knowledge corpora, and the basis for reasoning is also the human knowledge accumulated during the training process, such an autonomous driving system that uses LLM for driving decision-making is also called knowledge-driven autonomous driving.

[0003] Since the core of knowledge-driven autonomous driving is LLM, there are mainly two directions for its performance optimization: one is to improve the reasoning ability and decision-making accuracy of the model; the other is to perform prompt engineering, that is, to design and optimize the input prompts for LLM to improve the quality and accuracy of the output. Common methods include information supplementation method, explicit instruction method, chain of thought, etc. From the perspective of prompt engineering, the prompts input to LLM in the current mainstream knowledge-driven autonomous driving systems include vehicles and pedestrians around the host vehicle, traffic signals ahead, optional driving instructions, etc. Although it is sufficient to make driving decisions, there are still deficiencies in terms of information completeness.

[0004] In the prior art, the Association for Standardization of Automation and Measuring Systems (ASAM) proposed a road network description standard called OpenDrive. OpenDrive uses Extensible Markup Language (XML) to define detailed road information, including lanes, intersections, traffic signs, road surface types, etc., providing a standard and accurate way to describe road networks, enabling the sharing and exchange of road network data between different systems and platforms. Although OpenDrive can describe the road network completely and accurately, it cannot be directly used as the prompt input for the LLM. First of all, the XML files of OpenDrive are often too long. Just describing the road network of a single small town requires tens of thousands of lines. The large file content will cause difficulties for the LLM to parse. Secondly, since XML is not a natural language, it is also difficult for the LLM to understand the content described by OpenDrive. Finally, OpenDrive is a complete description of the road network within the entire map range and does not contain vehicle position information, which means that the vehicle's position and the real-time road network information around it cannot be directly obtained from OpenDrive, so it cannot be used for the road network description in driving decisions.

[0005] In summary, a natural language description method for a drivable road network based on waypoints is needed. Summary of the Invention

[0006] A brief overview of the present invention is given below to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify the key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is only to present certain concepts in a simplified form as a prelude to the more detailed description to follow.

[0007] In view of this, to solve the problem that the traditional road network description method in the prior art is difficult to directly obtain the vehicle's position and the real-time road network information around it from OpenDrive and apply it to driving decisions, the present invention provides a natural language description method for a drivable road network based on waypoints.

[0008] The technical solution is as follows: A natural language description method for a drivable road network based on waypoints, comprising the following steps:

[0009] S1. Obtain the global static information of the map, which includes all waypoints and crosswalks;

[0010] S2. Use a KD-Tree to obtain all the waypoints around the vehicle;

[0011] S3. Generate a natural language description of the drivable road network according to the content to be described and the attribute values of the waypoints.

[0012] Furthermore, in S1, the waypoints and crosswalk information included in the CARLA map are collected. The specific process is as follows:

[0013] S11. The user obtains all the waypoints in the CARLA map at a set spacing through the API provided by the CARLA map and saves them as a waypoint list. While obtaining the waypoints, the coordinates of each waypoint are extracted to construct a waypoint coordinate array for subsequent initialization of the KD-Tree.

[0014] S12. The user separately obtains all the crosswalks through the API provided by the CARLA map for subsequent description of the crosswalks.

[0015] Furthermore, in S2, the KD-Tree is initialized using the waypoint coordinate array, and the query_ball_point() function is called to obtain the indices of the waypoints within a set radius. Then, all the waypoints around the vehicle are obtained from the waypoint list according to the indices.

[0016] Furthermore, in S3, the content to be described includes the vehicle position, the number of lanes and the lane the vehicle is in, the lane width, the allowed lane-changing behaviors of the lane the vehicle is in, whether there is an intersection ahead and the distance to the intersection, whether there is a crosswalk ahead and the distance to the crosswalk. The specific generation process is as follows:

[0017] S31. According to the CARLA map, the vehicle object coordinates are embedded into a string to describe the vehicle position. The description format is: "You are driving at ({the x-axis coordinate of the vehicle}, {the y-axis coordinate of the vehicle}, {the z-axis coordinate of the vehicle}), and the surrouding road network is illustrated as follows.";

[0018] S32. According to the vehicle position, the lane ID of the lane the vehicle is in is directly described, and the number of lanes in the driving direction of the vehicle is calculated and described through the LLM.

[0019] S33. The lane width of the current road is directly described using the lane width attribute value of the waypoint where the vehicle is located. The description format is: "And the width of lanes of the current road is {the lane width}.";

[0020] S34. According to the possible lane-changing behaviors, the allowed lane-changing behaviors of the lane the vehicle is in are described.

[0021] S35. Starting from the current waypoint where the vehicle is located, continuously iterate forward to obtain waypoints until one of the following two situations occurs: a waypoint is at an intersection or the cumulative iteration distance is the same as the description radius. Describe whether there is an intersection ahead and the distance to the intersection.

[0022] S36. According to the vehicle position information in the CARLA map, describe whether there is a crosswalk ahead and the distance to the crosswalk.

[0023] In S32, the following steps are included:

[0024] S321. Obtain the waypoint corresponding to the vehicle position and check if it exists. If it does not exist, describe an error message and end the description.

[0025] S322. Based on the lane attribute value of the waypoint corresponding to the vehicle position, determine whether the lane where the vehicle is located is drivable. If it is not drivable, describe an error message and end the description.

[0026] S323. Select a waypoint near the vehicle and check if it has the same road ID and the same lane ID symbol as the current waypoint corresponding to the vehicle. Add the lane ID of the waypoint with the same road ID and the same lane ID symbol to the lane ID set.

[0027] S324. Repeat steps S323 and S324 until all waypoints near the vehicle have been traversed.

[0028] S325. Calculate the number of elements in the lane ID set. The result is the number of lanes in the vehicle's driving direction.

[0029] S326. Describe the number of lanes and the lane where the vehicle is located in the following format: "There are {number of lane IDs} lanes in the direction you are driving. You are driving on the lane with ID {lane ID where the vehicle is located} (lane IDs start from 1 or -1. Generally, the smaller the absolute value of the lane ID, the closer the lane is to the road centerline; the larger the absolute value of the lane ID, the farther the lane is from the road centerline)."

[0030] In S34, possible lane-changing behaviors include no lane change allowed, right lane change allowed but left lane change not allowed, left lane change allowed but right lane change not allowed, and left or right lane change allowed;

[0031] The description format for no lane change allowed is expressed as: "In addition, the current lane does not allow turning right or left, only going straight.";

[0032] The description format for right lane change allowed but left lane change not allowed is expressed as: "In addition, the current lane allows turning right, but does not allow turning left.";

[0033] The description format for left lane change allowed but right lane change not allowed is expressed as: "In addition, the current lane allows turning left, but does not allow turning right.";

[0034] The description format for left or right lane change allowed is expressed as: "In addition, the current lane allows turning either right or left.";

[0035] In S35, when a certain waypoint is at an intersection, further calculate the distance between the vehicle's current waypoint and the waypoint, and describe the intersection. The description format is: "Finally, there is a junction ahead and the distance to the junction is {distance to the junction} meters.";

[0036] When the iterated distance is the same as the description radius, it indicates that there is no intersection ahead, and the description is: "Finally, there is no junction ahead.";

[0037] In S36, the following steps are included:

[0038] S361. Use the Point module in the Python.shapely.geometry library to represent the vehicle's current position as a two-dimensional point, initialize the flag indicating whether there is a crosswalk ahead to no, and initialize the distance from the vehicle to the crosswalk to infinity;

[0039] S362. Select a crosswalk and represent the crosswalk as a two-dimensional polygon using the Polygon module in the Python.shapely.geometry library, in combination with the positions of the four vertices of the crosswalk;

[0040] S363. Calculate the distance from the vehicle to the boundary of the crosswalk and determine whether the crosswalk is within the described range;

[0041] S364. If the crosswalk is within the described range, then calculate the dot product of the vehicle direction vector and the crosswalk center point direction vector, and determine whether the crosswalk is in front of the vehicle and within the front described range. If the crosswalk is not within the angle of the front described range, jump to step S362;

[0042] S365. If the crosswalk is within the angle of the front described range, then update the value of the flag indicating whether there is a crosswalk in front to yes, and further compare the existing distance from the vehicle to the crosswalk and the distance from the vehicle to the current crosswalk, and take the expected value of the two to update the distance from the vehicle to the crosswalk, and obtain the distance from the vehicle to the nearest crosswalk;

[0043] S366. Repeat steps S362 to S365 until all crosswalks have been traversed;

[0044] S367. If there is a crosswalk in front, describe the distance from the vehicle to the crosswalk, and the description format is: "Please note that there is a crosswalk ahead of you, the distance to the crosswalk is {the distance from the vehicle to the crosswalk} meters." If there is no crosswalk in front, no additional description is made.

[0045] The beneficial effects of the present invention are as follows: Based on the characteristics of the LLM in the knowledge-driven autonomous driving system imitating the reasoning process of human drivers for driving decisions and the observation process of human drivers during driving, the present invention provides a method for generating a natural language description of the drivable road network that the autonomous driving host vehicle can travel on relying on waypoints in the CARLA traffic simulation platform, which is used to provide road network information in the form of natural language for the knowledge-driven autonomous driving agent running on CARLA to make up for the lack of road network information in the existing knowledge-driven autonomous driving decision-making. The present invention utilizes the waypoint mechanism and combines the range query ability of the KD-Tree to quickly obtain the road network information around the vehicle and quickly retrieve the waypoints around the autonomous driving host vehicle. By designing a template for natural language description of the road network and using the information contained in the waypoints, a complete and reliable natural language description of the drivable road network around the vehicle is generated to supplement the road space information for the LLM, so as to obtain more accurate driving decisions, improve the performance of the knowledge-driven autonomous driving system, improve the accuracy and reliability of the decisions, and achieve more realistic, safer, and more human-like driving decisions. The road network information that the present invention can provide includes the vehicle position, the number of lanes and the lane where the vehicle is located, the lane width, the lane-changing behavior allowed in the lane where the vehicle is located, whether there is an intersection ahead and the distance from the intersection, whether there is a crosswalk ahead and the distance from the crosswalk, which can cover the road network information required for driving decisions and meet the information needs of the LLM that relies on human-like reasoning for driving decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0047] Figure 1 It is a schematic flow chart of a method for natural language description of a drivable road network based on waypoints;

[0048] Figure 2 It is a schematic flow chart of an embodiment of a method for natural language description of a drivable road network based on waypoints. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] In order to make the technical solutions and advantages in the embodiments of the present invention clearer and more understandable, the following further describes the exemplary embodiments of the present invention in detail with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than an exhaustive list of all embodiments. It should be noted that, without conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0050] Refer to Figure 1 and Figure 2Detailed description of this embodiment, a natural language description method for a drivable road network based on waypoints, specifically including the following steps:

[0051] S1. Obtain the global static information of the map, which includes all waypoints and crosswalks;

[0052] S2. According to the global static information of the map, use the KD-Tree to obtain all the waypoints around the vehicle;

[0053] S3. Generate a natural language description of the drivable road network according to the content to be described and the attribute values of the waypoints.

[0054] Furthermore, in S1, in order to generate a natural language description of the drivable road network around the vehicle, collect the waypoint and crosswalk information included in the CARLA map. The specific process is as follows:

[0055] S11. The waypoints in the CARLA map are predefined in the road network. The user uses the API (Application Programming Interface) provided by the CARLA map to obtain all the waypoints in the CARLA map at a set spacing and save them as a waypoint list. While obtaining the waypoints, extract the coordinates of each waypoint and construct a Python.numpy array of waypoint coordinates for subsequent initialization of the KD-Tree (K-dimensional tree);

[0056] S12. Since the waypoints do not contain information related to crosswalks, the user needs to separately use the API provided by the CARLA map to obtain all the crosswalks for subsequent description of the crosswalks.

[0057] Furthermore, in S2, initialize the KD-Tree with the waypoint coordinate array, call the query_ball_point() function to obtain the indices of the waypoints within a set radius range (referred to as the description radius), and then obtain all the waypoints within the set range around the vehicle from the waypoint list according to the indices.

[0058] Specifically, perform a range query by calling the KD-Tree and the query_ball_point() function (used to find all points within a specified radius) through the Python.scipy.spatial library (a module dedicated to spatial calculations and computational geometry-related operations).

[0059] Furthermore, in S3, the content to be described includes the vehicle position, the number of lanes and the lane in which the vehicle is located, the lane width, the allowed lane-changing behavior of the lane in which the vehicle is located, whether there is an intersection ahead and the distance to the intersection, whether there is a crosswalk ahead and the distance to the crosswalk. The specific generation process is as follows:

[0060] S31. According to the CARLA map, since the vehicle objects in the CARLA map already have three-dimensional spatial coordinate values, the vehicle object coordinates are directly embedded into a string to describe the vehicle's position. The description format is: "You are driving at ({the x-axis coordinate of the vehicle}, {the y-axis coordinate of the vehicle}, {the z-axis coordinate of the vehicle}), and the surrounding road network is illustrated as follows.";

[0061] S32. Based on the vehicle's position, since each waypoint has road ID and lane ID attributes, the lane ID where the vehicle is located can be directly described. (To enable the LLM (Large Language Model) to understand the relative position of the lanes, an additional note is needed: the lane ID starts from 1 or -1. Generally, the smaller the absolute value of the lane ID, the closer it is to the center line of the road, and vice versa.) Calculate and describe the number of lanes in the driving direction (i.e., the drivable direction) of the vehicle through the LLM;

[0062] S33. Directly use the lane width attribute value of the waypoint where the vehicle is located to describe the lane width of the current road (assuming that the widths of all lanes are the same). The description format is: "And the width of lanes of the current road is {lane width}.";

[0063] S34. The lane-changing attribute of the waypoint defines the allowable lane-changing behavior at the location of the waypoint. Essentially, it describes whether the left and right sides of the lane are solid lines (no lane-changing allowed) or dashed lines (lane-changing allowed). Describe the allowable lane-changing behavior of the lane where the vehicle is located according to the possible lane-changing behavior;

[0064] S35. To determine whether there is an intersection in front of the vehicle, only need to start from the waypoint where the vehicle is currently located, continuously iterate forward to obtain waypoints until one of the situations where a certain waypoint is at an intersection or the iteration distance and the description radius are the same is encountered. Describe whether there is an intersection in front and the distance to the intersection;

[0065] S36. Since the crosswalks in the CARLA map are represented by the positions of their four vertices, the positioning and detection of crosswalks are relatively complex. According to the vehicle position information in the CARLA map, describe whether there is a crosswalk in front and the distance to the crosswalk;

[0066] In the above S32, the following steps are included:

[0067] S321. Obtain the waypoint corresponding to the vehicle's position and determine whether it exists. If it does not exist, describe the error message and end the description;

[0068] S322. Determine whether the lane where the vehicle is located is drivable according to the lane attribute value of the waypoint corresponding to the vehicle position. If it is not drivable, describe the error message and end the description;

[0069] S323. Select a waypoint around the vehicle and determine whether it has the same road ID as the current waypoint corresponding to the vehicle and the same lane ID symbol (the same symbol means the same direction, otherwise it means the opposite direction); add the lane ID of the waypoint with the same road ID and the same lane ID symbol to the Python lane ID set (the elements of the set are unique);

[0070] S324. Repeat steps S323 and S324 until all the waypoints around the vehicle are traversed;

[0071] S325. Calculate the number of elements in the lane ID set, and the result is the number of lanes in the driving direction of the vehicle;

[0072] S326. Describe the number of lanes and the lane where the vehicle is located in the format: "There are {number of lane IDs} lanes in the direction you are driving. You are driving on the lane with ID {lane ID where the vehicle is located} (lane IDs start from 1 or -1. Generally, the smaller the absolute value of the lane ID, the closer the lane is to the road centerline; the larger the absolute value of the lane ID, the farther the lane is from the road centerline).";

[0073] In S34, the possible lane change behaviors include not allowing lane change, allowing right lane change but not left lane change, allowing left lane change but not right lane change, and allowing left or right lane change;

[0074] The description format for not allowing lane change is: "In addition, the current lane does not allow turning right or left, only going straight.";

[0075] The description format for allowing lane change to the right but not to the left is expressed as: "In addition, currentlane allows turning right, but does not allow turning left.";

[0076] The description format for allowing lane change to the left but not to the right is expressed as: "In addition, currentlane allows turning left, but does not allow turning right.";

[0077] The description format for allowing lane change to the left or right is expressed as: "In addition, current lane allowsturning either right or left.";

[0078] In S35, when a certain waypoint is at an intersection, further calculate the distance between the vehicle's current waypoint and the waypoint, and describe the intersection. The description format is: "Finally, there is a junction aheadand the distance to the junction is {distance to the intersection} meters.";

[0079] When the iterated distance is the same as the description radius, it indicates that there is no intersection ahead, and the description is: "Finally, there is no junction ahead.";

[0080] In S36, the following steps are included:

[0081] S361. Use the Point module (used to represent points in a two-dimensional plane) in the Python.shapely.geometry library (a library for processing two-dimensional planar geometric objects) to represent the vehicle's current position as a two-dimensional point, initialize the flag indicating whether there is a crosswalk ahead to no, and initialize the distance from the vehicle to the crosswalk to infinity;

[0082] S362. Select a crosswalk and use the Polygon module (the core class for representing two-dimensional polygons) in the Python.shapely.geometry library to represent the crosswalk as a two-dimensional polygon in combination with the positions of the four vertices of the crosswalk (for simplicity of processing, the z-axis coordinate is ignored, that is, it is assumed that the vehicle and the crosswalk are on a two-dimensional plane);

[0083] S363. Calculate the distance from the vehicle to the boundary of the crosswalk and determine whether the crosswalk is within the described range;

[0084] S364. If the crosswalk is within the described range, calculate the dot product of the vehicle direction vector and the center point direction vector of the crosswalk, and determine whether the crosswalk is in front of the vehicle and within the front described range. If the crosswalk is not within the included angle of the front described range, jump to step S362; [[ID=~5]]

[0085] S365. If the crosswalk is within the included angle of the front described range, update the value of the flag indicating whether there is a crosswalk in front to yes, and further compare the existing distance from the vehicle to the crosswalk and the distance from the vehicle to the current crosswalk, and take the expected value (i.e., the smaller value) of the two to update the distance from the vehicle to the crosswalk, and obtain the distance from the vehicle to the nearest crosswalk;

[0086] S366. Repeat steps S362 to S365 until all crosswalks have been traversed;

[0087] S367. If there is a crosswalk in front, describe the distance from the vehicle to the crosswalk in the following format: "Please note that there is a crosswalk ahead of you, the distance to the crosswalk is {the distance from the vehicle to the crosswalk} meters." If there is no crosswalk in front, do not make any additional description.

[0088] Specifically, the CARLA (Car Learning to Act) map is an open-source autonomous driving simulation platform designed to provide a highly realistic virtual environment for autonomous driving research, development, and testing. CARLA offers a variety of scenarios, weather conditions, traffic, road networks, and complex environment settings for researchers to test autonomous driving systems and algorithms;

[0089] The road network refers to a traffic network composed of a series of roads, intersections, lanes, etc. It not only covers basic road connections but also includes elements such as traffic signals, signs, and road conditions. In autonomous driving and intelligent transportation systems, the road network can provide detailed geometric information, lane divisions, traffic signs, traffic flow directions, etc., which helps to improve the vehicle's positioning accuracy and path planning ability;

[0090] A waypoint is a node in the CARLA road network with specific positions and attributes, used to describe the topological structure of the road network. Each road network contains geometric information about the roads it belongs to, such as 3D coordinates, orientations, IDs of roads and lanes, lane attributes, road segment types, etc. Through a series of ordered waypoints, CARLA can create complete road paths for the autonomous driving simulation system to perform path planning, localization, and navigation.

[0091] A K-Dimensional Tree (KD-Tree) is a tree data structure for multi-dimensional spatial data used for fast retrieval, often used to handle problems such as space partitioning, nearest neighbor search, and range queries. The KD-Tree divides data points into sub-regions according to dimensions, thus improving the efficiency of performing search operations in high-dimensional spaces.

[0092] A Large Language Model (LLM) is a deep learning model capable of processing natural language data, able to complete various tasks according to task descriptions in natural language form, such as text generation, question answering, translation, and summarization, etc. Large language models are considered to have human-like language understanding and logical reasoning abilities, and can think, reason, and make decisions by imitating humans based on the knowledge accumulated during training.

[0093] A prompt is a text description or instruction input to the LLM to guide the LLM to generate specific outputs and complete tasks. The prompt can be a question, a description, a command, or any form of input. The LLM can understand the intention of the task based on the input and generate corresponding answers or content.

[0094] Although the present invention has been described based on a limited number of embodiments, those skilled in the art in this technical field will understand, based on the above description, that other embodiments can be envisioned within the scope of the present invention thus described. In addition, it should be noted that the language used in this specification is mainly selected for readability and teaching purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention. Therefore, many modifications and changes are obvious to those of ordinary skill in this technical field without departing from the scope and spirit of the appended claims. For the scope of the present invention, the disclosure of the present invention is illustrative rather than restrictive, and the scope of the present invention is defined by the appended claims.

Claims

1. A natural language description method for a drivable road network based on waypoints, characterized in that, It includes the following steps: S1. Obtain the global static information of the map, which includes all waypoints and crosswalks; S2. Use the KD-Tree to obtain all the waypoints around the vehicle; S3. Generate a natural language description of the drivable road network according to the content to be described and the attribute values of the waypoints.

2. The natural language description method of a drivable road network based on waypoints according to claim 1, characterized in that, In S1, collect the waypoint and crosswalk information included in the CARLA map. The specific process is as follows: S11. The user obtains all the waypoints in the CARLA map at a set interval through the API provided by the CARLA map and saves them as a waypoint list. While obtaining the waypoints, extract the coordinates of each waypoint and construct a waypoint coordinate array for subsequent initialization of the KD-Tree; S12. The user separately obtains all the crosswalks through the API provided by the CARLA map for subsequent description of the crosswalks.

3. The natural language description method of a drivable road network based on waypoints according to claim 2, wherein, In S2, initialize the KD-Tree with the waypoint coordinate array, call the query_ball_point() function to obtain the indices of the waypoints within the set radius, and then obtain all the waypoints around the vehicle from the waypoint list according to the indices.

4. A natural language description method for a drivable road network based on waypoints according to claim 3, characterized in that, In S3, the content to be described includes the vehicle position, the number of lanes, the lane where the vehicle is located, the lane width, the allowed lane-changing behavior of the lane where the vehicle is located, whether there is an intersection ahead and the distance to the intersection, whether there is a crosswalk ahead and the distance to the crosswalk. The specific generation process is as follows: S31. Embed the vehicle object coordinates into a string according to the CARLA map to describe the vehicle position. The description format is: "You are driving at ({x-axis coordinate of the vehicle}, {y-axis coordinate of the vehicle}, {z-axis coordinate of the vehicle}), and the surrounding road network is illustrated as follows.”; S32. Directly describe the lane ID where the vehicle is located according to the vehicle position, and calculate and describe the number of lanes in the driving direction of the vehicle through the LLM; S33. Directly use the lane width attribute value of the waypoint where the vehicle is located to describe the lane width of the current road. The description format is: "And the width of lanes of the current road is {lane width}.”; S34. Describe the allowed lane-changing behavior of the lane where the vehicle is located according to the possible lane-changing behavior; S35. Starting from the waypoint where the vehicle is currently located, continuously iterate forward to obtain waypoints until one of the situations where a certain waypoint is at an intersection or the iteration distance and the description radius are the same is encountered, and describe whether there is an intersection ahead and the distance to the intersection; S36. Describe whether there is a crosswalk ahead and the distance to the crosswalk according to the vehicle position information in the CARLA map; In S32, it includes the following steps: S321. Obtain the waypoint corresponding to the vehicle position and determine whether it exists. If it does not exist, describe the error message and end the description; S322. Determine whether the lane where the vehicle is located is drivable according to the lane attribute value of the waypoint corresponding to the vehicle position. If it is not drivable, describe the error message and end the description; S323. Select a waypoint around the vehicle and determine whether it has the same road ID and the same lane ID symbol as the current waypoint corresponding to the vehicle; add the lane ID of the waypoint with the same road ID and the same lane ID symbol to the lane ID set; S324. Repeat steps S323 and S324 until all the waypoints around the vehicle are traversed; S325. Calculate the number of elements in the lane ID set, and the result is the number of lanes in the driving direction of the vehicle; S326. Describe the number of lanes and the lane where the vehicle is located in the following format: "There are {number of lane IDs} lanes in the direction you are driving. You are driving on the lane with ID {lane ID where the vehicle is located} (lane IDs start from 1 or -1. Generally, the smaller the absolute value of the lane ID, the closer the lane is to the road centerline; the larger the absolute value of the lane ID, the farther the lane is from the road centerline).”; In S34, the possible lane-changing behaviors include not allowing lane changes, allowing right lane changes but not left lane changes, allowing left lane changes but not right lane changes, and allowing left or right lane changes; The description format for not allowing lane changes is: "In addition, the current lane does not allow turning right or left, only going straight.”; The description format for allowing right lane changes but not left lane changes is: "In addition, the current lane allows turning right, but does not allow turning left.”; The description format for allowing left lane changes but not right lane changes is: "In addition, the current lane allows turning left, but does not allow turning right.”; The description format for allowing lane changes to the left or right is expressed as: "In addition, current lane allows turning either right or left.”; In S35, when a certain waypoint is at an intersection, further calculate the distance between the vehicle's current waypoint and the waypoint, and describe the intersection. The description format is: "Finally, there is a junction ahead and the distance to the junction is {distance to the intersection} meters.”; When the iterated distance is the same as the description radius, it indicates that there is no intersection ahead, and the description is: "Finally, there is no junction ahead.”; In S36, the following steps are included: S361. Use the Point module in the Python.shapely.geometry library to represent the vehicle's current position as a two-dimensional point, initialize the flag indicating whether there is a crosswalk ahead to no, and initialize the distance from the vehicle to the crosswalk to infinity; S362. Select a crosswalk and use the Polygon module in the Python.shapely.geometry library to represent the crosswalk as a two-dimensional polygon in combination with the positions of the four vertices of the crosswalk; S363. Calculate the distance from the vehicle to the boundary of the crosswalk and determine whether the crosswalk is within the description range; S364. If the crosswalk is within the description range, calculate the dot product of the vehicle's direction vector and the direction vector of the center point of the crosswalk, and determine whether the crosswalk is in front of the vehicle and within the front description range. If the crosswalk is not within the included angle of the front description range, jump to step S362; S365. If the crosswalk is within the included angle of the front description range, update the value of the flag indicating whether there is a crosswalk ahead to yes, and further compare the existing distance from the vehicle to the crosswalk and the distance from the vehicle to the current crosswalk, and take the expected value of the two to update the distance from the vehicle to the crosswalk to obtain the distance from the vehicle to the nearest crosswalk; S366. Repeat steps S362 to S365 until all crosswalks have been traversed; S367. If there is a crosswalk ahead, describe the distance from the vehicle to the crosswalk. The description format is: "Please note that there is a crosswalk ahead of you, the distance to the crosswalk is {distance from the vehicle to the crosswalk} meters.” If there is no crosswalk ahead, no additional description is made.

Images