Intelligent vehicle autonomous navigation method based on large language model

Abstract

The application discloses an intelligent vehicle autonomous navigation method based on a large language model, and belongs to the technical field of automatic driving. Mainly includes the following steps: S1, obtaining the input target instruction information, analyzing the user intention through the large language model, and obtaining the navigation point sequence; S2, obtaining the vehicle navigation initialization state; S3, generating the drivable direction set of the current time according to the navigation initialization state; S4, generating the space observation result of the current vehicle relative to the target position by using a visual recognition method; S5, obtaining a plurality of reasoning results by calculation; S6, saving the plurality of reasoning results coupled with time stamps to the historical semantic memory information base; S7, analyzing the change trend and identifying the abnormal result by using the large language model; S8, using the large language model to combine the prompt template to evaluate the consistency of each drivable direction and the target direction; The application achieves low dependence on high-precision maps, low computational complexity, high robustness navigation; It does not need the target accurate position.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to an intelligent vehicle autonomous navigation method based on a large language model. Background Technology

[0002] The automotive industry is rapidly developing towards intelligence and connectivity, and the public's demands for travel safety, efficiency, and driving experience are constantly increasing. Autonomous navigation of intelligent vehicles in complex environments has become a key requirement and an urgent problem to be solved for intelligent driving systems.

[0003] Currently, intelligent vehicle navigation mainly relies on high-precision maps, real-time traffic information, and traditional map search or optimization algorithms. However, high-precision maps are costly to produce and maintain, and are not updated in a timely manner. Furthermore, traditional methods lack human-like decision-making capabilities and cannot flexibly respond to unexpected road situations, affecting driving efficiency and safety.

[0004] In recent years, with the development of large language models (LLMs), which possess multimodal information understanding and logical reasoning capabilities, a new path has been provided for autonomous navigation of intelligent vehicles in complex environments. Related research has verified its feasibility. However, existing LLM-based navigation methods rely too heavily on high-precision maps and suffer from problems such as high pressure in processing high-dimensional multimodal data, high training costs, and long processing times. Therefore, it is necessary to provide an autonomous navigation method for intelligent vehicles based on large language models to solve the above problems.

[0005] It should be noted that the information disclosed in this background section is only for understanding the background technology of this application concept, and therefore may include information that does not constitute prior art. Summary of the Invention

[0006] According to one aspect of this application, an intelligent vehicle autonomous navigation method based on a large language model is provided, which achieves navigation with low dependence on high-precision maps, low computational complexity, and high robustness; it does not require the precise location of the target.

[0007] The technical solution adopted by this application to solve its technical problem is: an intelligent vehicle autonomous navigation method based on a large language model, the method comprising the following steps: S1. Obtain the target instruction information input by the user, analyze the user's intent through a large language model, and generate multiple consecutive navigation points in combination with a custom navigation prompt template to obtain a navigation point sequence. S2. Obtain the vehicle's current position and orientation information and use it as the navigation initialization state; S3. Based on the navigation initialization state, extract several sets of drivable directions from the navigation point sequence, and in combination with preset road driving constraints, remove impassable directions from the extracted sets of drivable directions to generate the set of drivable directions at the current moment. S4. Using visual recognition methods or pre-stored landmark coordinate space vector calculation methods, obtain all vehicle visible landmark information of the current drivable direction set, and generate spatial observation results of the current vehicle relative to the target position. S5. Obtain key spatial information between the target location and each of the vehicle's visible landmarks from the spatial observation results, construct a vector relationship model between the current vehicle and the target location and each of the vehicle's visible landmarks, calculate the direction and distance of the current vehicle to reach the target location by passing through any one or more of the vehicle's visible landmarks, and obtain multiple sets of inference results; S6. Couple multiple sets of reasoning results with timestamps and save them to the historical semantic memory information database; S7. Read all reasoning results between the current vehicle and the target location within a preset time node from the historical semantic memory information database to analyze the consistency of the results of the same vehicle's drivable direction, and remove abnormal or deviating results to dynamically update the historical semantic memory information database. S8. Based on preset navigation rules, retrieve the optimal reasoning result of the current vehicle's arrival at the target location from the historical semantic memory information database, and generate a navigation route.

[0008] Optionally, S2 specifically includes: S21. Obtain the vehicle's current pose state information, which includes at least the vehicle's two-dimensional position coordinates and heading angle information in the global coordinate system. S22. Represent the vehicle pose information as a state vector in a global coordinate system. ,in Indicates the vehicle's location. Indicates the vehicle's heading direction; S23. Based on the current pose state, initialize the current position node of this macro-path planning as a reference benchmark for road topology and direction planning.

[0009] Optionally, the two-dimensional position coordinates are represented using a two-dimensional discrete space model, specifically including: The vehicle operating environment is divided into regular grid units according to a preset scale, and each grid unit corresponds to a fixed area in the actual road space.

[0010] Optionally, S3 specifically includes: S31. Based on the connection relationship of the current node of the vehicle in the road topology, extract all road directions connected to the current node, wherein the road directions are represented as discrete directions; S32. Map each road connection direction to a standardized direction representation, wherein the standardized direction representation includes azimuth angle or direction category; S33. Combining road attributes and traffic rules, directions that do not meet the traffic conditions are eliminated to obtain the set of permissible directions for vehicles at the current moment. .

[0011] Optionally, the set of discrete directions is set to eight directions.

[0012] Optionally, S4 specifically includes: S41. Acquire current vehicle visual data, perform target recognition of landmarks using a visual language model, and output the orientation angle of visible landmarks relative to the vehicle's current position. The system obtains the set of landmarks currently observable by the vehicle, with each landmark corresponding to pre-stored global coordinate information for subsequent spatial calculations. S42. For each visible landmark, estimate the spatial distance between the vehicle and the landmark based on the vehicle positioning accuracy. ; S43. Combine the azimuth and distance of each landmark to form spatial observation results. ,in This indicates the number of the landmark building.

[0013] Optionally, the spatial distance The distance is predicted by geometric distance estimation. The geometric distance estimation input includes the target landmark category information output by the visual language model, the pixel height or pixel scale information of the target in the image, and the prior information of the true height of the corresponding landmark. The distance estimation model is as follows: ; in For scale calibration coefficients, The equivalent focal length of the camera. The actual height of the landmark building The height of the projected pixels of the target in the image. This is the distance offset term.

[0014] Optionally, S5 specifically includes: S51. Obtain spatial observation results, as well as the spatial relationship between the target location and any landmark and the spatial relationship between any two landmarks; S52. Construct spatial vectors between vehicles and landmarks, between targets and landmarks, and between landmarks; S53. Infer the spatial vector from the vehicle to the target by using the vector superposition relationship; S54. Perform statistical fusion on multiple landmark inference results, remove outliers, and obtain stable target direction and distance estimation results. .

[0015] Optionally, in step S6, it is suitable to use a large language model to read the most recent structured historical trajectory information from the historical semantic memory information database, convert it into a natural language description form, input it into the large language model for analysis and summarization, and store it as semantic memory; In step S7, multiple target direction reasoning results are read from the historical semantic memory information database to form a historical reasoning sequence. The changing trend is analyzed and abnormal results are identified through a large language model, and the most likely target direction and position estimate are output.

[0016] Optionally, in step S8, the set of feasible directions, the prediction results of the target direction, and the historical trajectory information are constructed into a set of decision features. A large language model is used in combination with prompt templates to evaluate the consistency between each feasible direction and the target direction, output the priority score or ranking result of each direction, and select the direction with the highest priority as the next planning direction.

[0017] The beneficial effects that this application can produce include: This application provides an intelligent vehicle autonomous navigation method based on a large language model. This invention upgrades node path planning to direction-level macro-planning, combining a set of drivable directions, landmark target direction reasoning, and long-term spatial memory and reflection mechanisms. By training the large language model with zero samples, it achieves navigation with low dependence on high-precision maps, low computational complexity, and high robustness. It does not require the precise location of the target, infers the direction through vector relationships, and reduces perception errors by fusing the reasoning results. It has temporal consistency and self-correction capabilities, outputs only macro-direction, and can be decoupled and integrated with different vehicle control systems. It is suitable for complex scenarios with incomplete information such as weak maps, and has high engineering practicality and promotion value. Attached Figure Description

[0018] Figure 1 This is a flowchart of an intelligent vehicle autonomous navigation method based on a large language model proposed in this invention; Figure 2 This is a schematic diagram of the structure of an intelligent vehicle autonomous navigation system based on a large language model. Figure 3 This is a schematic diagram illustrating the process of obtaining the relative position of a landmark based on a visual language model. Figure 4 This is a schematic diagram illustrating the process of summarizing historical trajectories based on a large language model. Figure 5 A schematic diagram of the reflection process based on a large language model; Figure 6 This is a schematic diagram of the long-term and short-term planning process based on a large language model. Detailed Implementation

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0021] like Figure 1 As shown, this application provides an intelligent vehicle autonomous navigation method based on a large language model, including the following steps: S1. Obtain the user's voice commands, combine them with the navigation prompt template, analyze the user's intent using a large language model, and output each navigation point in sequence to obtain the navigation point sequence; S2. Obtain the vehicle's current pose, including global coordinates and heading information, as the navigation initialization state. It is uniformly represented in the global coordinate system, providing basic input for subsequent direction reasoning and feasible direction extraction. S3. Extract the possible directions of the vehicle, combine road constraints, intersection connections and traffic rules, eliminate the impassable directions, and output the set of possible directions at the current time to provide a candidate set for direction-level planning; S4. Obtain information about visible landmarks of the vehicle, including direction and distance, and calculate spatial vector representation using visual recognition or pre-stored landmark coordinates; output the spatial observation results of the vehicle relative to the landmarks. S5. Obtain spatial relationship information between the target and landmarks, construct a vector relationship model between the vehicle, target and landmarks; infer the direction and rough distance from the vehicle to the target, and fuse multiple landmark results to obtain a stable estimate, thus obtaining the target direction reasoning result based on the spatial relationship of landmarks, providing a basis for macro-path planning; S6. Use a large language model to summarize the historical trajectory and save it as semantic memory; S7. Read the historical target reasoning results from spatial memory; analyze the consistency of multiple reasoning results, eliminate abnormal or trend-deviation results; output the most likely target location and direction at present. S8. Integrate the set of feasible directions, target direction prediction, semantic memory, and long-term memory information; input them into the large language model, the large language model evaluates the priority of each direction, selects the direction closest to the target as the next movement direction, outputs short-term decisions, and realizes macro-planning at the direction level; the short-term decisions include the driving direction and the corresponding node. S9. After the vehicle moves along the planned direction and travels a fixed distance, update the current position node. Record the historical driving path, selected direction, and target reasoning results, update the long-term spatial memory, and retain the most recent trajectories and target reasoning information to achieve continuous planning. Repeat the S2–S9 loop until the target position is reached or the planning conditions are met.

[0022] In this embodiment, S2 specifically includes: S21. Obtain the vehicle's pose state information at the current moment. The pose state includes at least the vehicle's two-dimensional position coordinates and heading angle information in the global coordinate system. The position coordinates are used to describe the vehicle's absolute position in the global space, and the heading angle is used to describe the vehicle's current orientation. This pose state serves as the starting state of the current planning cycle. S22. Represent the vehicle pose information as a state vector in a global coordinate system. ;in Indicates the vehicle's location. This indicates the vehicle's heading direction; the unified representation is used for subsequent direction calculation, spatial vector construction, and direction determination. S23. Based on the current pose state, initialize the current position node for this macro-path planning; this node serves as a reference point for road topology and direction planning, and is used for subsequent direction-level movement state updates. The two-dimensional coordinates described here do not use real meter-level continuous coordinates. Instead, the vehicle operating environment is abstracted into a two-dimensional discrete spatial model. The actual road environment is divided into regular grid cells according to a preset scale, with each grid cell corresponding to a fixed area in the actual road space. In this embodiment, the side length of a single grid cell is preferably set to 50 meters, which reduces the computational complexity of the navigation inference process while ensuring spatial representation accuracy. The two-dimensional coordinates also significantly reduce the impact of errors smaller than one grid scale on the planning results, avoiding the infinite amplification of visual distance estimation errors in continuous space, thereby improving the robustness of macroscopic navigation decisions.

[0023] In this embodiment, S3 specifically includes: S31. Based on the connection relationship of the current node of the vehicle in the road topology, extract all road directions connected to the node; the directions can be represented by discrete direction labels, and each direction corresponds to a passable road connection. S32. Map each road connection direction to a standardized direction representation, which may include, but is not limited to, azimuth angle or direction category; through this mapping, different road directions can be compared in a unified direction space; S33. Combining road attributes and traffic rules, directions that do not meet the traffic conditions are eliminated; after filtering, the set of permissible directions for vehicles at the current moment is obtained. This output is used for subsequent macroeconomic direction decisions; This invention discretizes continuous directions into a finite set of directions, transforming direction decision-making from a continuous optimization problem into a finite classification problem. This significantly reduces the search space for direction reasoning, facilitating stable and interpretable direction reasoning in large language models. In this embodiment, the discrete direction set is preferably set to eight directions: North (N) → Northeast (NE) → East (E) → Southeast (SE) → South (S) → Southwest (SW) → West (W) → Northwest (NW), to achieve direction-level navigation decision-making. This effectively solves the problem of excessively high dimensionality in the decision space caused by continuous and precise direction values ​​in a continuous direction space.

[0024] like Figure 1 and Figure 3 As shown, in this embodiment, S4 specifically includes: S41. Acquire the current vehicle's visual data, use a visual language model to identify landmarks, and output the relative offset angle of the visible landmark from the image centerline. Combine this with the difference between the image and the current vehicle's heading angle to give the direction angle of the visible landmark relative to the vehicle's current position. ; To obtain the set of landmarks currently observable by the vehicle; Each landmark corresponds to pre-stored global coordinate information for subsequent spatial calculations; S42. For each visible landmark: If the vehicle has good positioning accuracy in the city and can obtain its precise location, then the spatial distance between the vehicle and the landmark can be estimated directly using the converted two-dimensional coordinates of the vehicle. The distance mentioned is a coarse-grained estimate used for macroscopic directional reasoning. If the vehicle enters urban canyons, semi-enclosed roads, overpasses, or other sections where positioning accuracy is insufficient, geometric distance estimation is used to directly predict the spatial distance between the vehicle and landmarks. ; S43. Combine the azimuth and distance of each landmark to form spatial observation results. ,in Indicates the number of the landmark building; In this embodiment, the visual language model is mainly used to perform semantic recognition and target confirmation of landmark buildings from a single frame, determining whether there is a building entity in the current field of view that matches the navigation target or a pre-stored landmark. When the visual language model outputs a recognition result and further spatial distance information between the target and the vehicle is required, this invention does not directly rely on the visual language model for distance regression, but instead calls an independent geometric distance estimate to predict the target distance, thereby avoiding the instability problem of large language models in numerical scale regression tasks.

[0025] The geometric distance estimation input includes: 1) Target landmark building category information output by the visual language model; 2) Pixel height or pixel scale information of the target in the image; 3) Prior information on the actual height of the corresponding landmark building; Geometric distance estimation is based on constructing target distance estimation relationships using a monocular imaging geometric model. It is assumed that the true vertical height of the landmark building is... Its projected pixel height in the image is The equivalent focal length of the camera is In the ideal imaging model, the geometric distance between the target and the camera is inversely proportional to the parameters mentioned above.

[0026] In this embodiment, the geometric distance estimation adopts the following distance estimation model: ; in, Indicates the estimated distance to the target landmark; These are scale calibration coefficients used to correct scale deviations between the actual imaging environment and the ideal model; This is the distance offset term, used to compensate for system errors and installation errors.

[0027] This invention introduces a scale calibration mechanism based on real distance data. By collecting real distance data between vehicles and multiple known landmarks offline, a training sample set is constructed, and the scale calibration coefficients in the aforementioned distance estimation model are adjusted accordingly. With bias term Fitting and optimization are performed. The scale-calibrated geometric distance estimate can maintain stable estimation accuracy over a large distance range, significantly reducing systematic errors in long-distance scenarios, and ensuring good continuity and consistency in the distance estimation results of landmark buildings within the range of 50 meters to 1500 meters.

[0028] In this embodiment, S5 specifically includes: S51. Obtain spatial observation results, and obtain the spatial relationship between the target position and landmarks, as well as the spatial relationship between landmarks, from the known information of the navigation points; the relationship includes the direction and distance from the target to the landmarks and between landmarks; S52. Construct spatial vectors from vehicles to landmark 1 respectively. Spatial vector from target to landmark 2 And the spatial vector from landmark 1 to landmark 2 ; S53. Infer the spatial vector from the vehicle to the target by using vector superposition relationships. ; S54. When multiple landmark inference results exist, statistical fusion is performed on multiple target directions to remove outlier results with large differences; stable target direction and distance estimation results are obtained by averaging the direction angle and the distance. As a basis for macro-planning; This invention constructs spatial vector relationships between vehicles and landmarks, targets and landmarks, and between landmarks. It then uses vector superposition and subtraction to calculate the relative direction and distance from the vehicle to the target. Simultaneously, it statistically fuses multiple landmark inference results to eliminate outliers, achieving reliable target direction inference even when the vehicle cannot directly observe the target or global positioning information is incomplete. This method fully utilizes the prior spatial relationships between perceptible landmark information and navigation points, without relying on direct target distance measurement, improving the stability and consistency of target direction and distance estimation. This provides continuous, interpretable, and robust direction-level decision-making support for macro-path planning, enabling vehicles to navigate efficiently in complex environments or long-distance scenarios.

[0029] like Figure 1 and Figure 4 As shown, in this embodiment, S6 specifically includes: S61. Retrieves several recent structured historical trajectory information from long-term spatial memory; converts the historical trajectory information into natural language description and inputs it into a large language model for analysis. The large language model, combined with template prompts, summarizes the historical trajectories and generates semantic description information reflecting the vehicle's recent navigation behavior; S62. The trajectory semantic summary results generated by the large language model are stored as semantic memory, and a correlation is established with the corresponding structured trajectory segments. The semantic memory serves as an important input for the reflection mechanism and direction-level path planning, and is used to assist in subsequent target direction prediction and macro-direction decision-making. This invention converts a vehicle's recent structured historical trajectory information into natural language descriptions and inputs them into a large language model for analysis and summarization. This generates semantic trajectory information reflecting the vehicle's recent navigation behavior. The system prompts templates to guide the large language model in gradually summarizing historical trajectories, outputting information such as the vehicle's past movement trends, main directions, visited and unvisited areas, and potential intersections. In this embodiment, the preferred number of historical trajectories stored is 10. For each trajectory sequence, the system converts the current position coordinates, feasible directions, actions taken, and next position into natural language descriptions, such as "The vehicle's current position is (0,0), feasible directions include north and east, choose to move north, arrive at coordinates (0,1)", and concatenates them in chronological order to form a complete trajectory text. After establishing a association between the semantic information and the corresponding structured trajectory segments, it is stored as long-term semantic memory, forming the basis for a reflection mechanism. This allows the system to comprehensively analyze and optimize the target direction based on historical behavior and current location environment information, effectively solving the problem of large language models forgetting historical trajectories in long dialogues.

[0030] like Figure 1 and Figure 5 As shown, in this embodiment, S7 specifically includes: S71. Read the results of multiple target direction reasoning from long-term spatial memory to form a historical reasoning sequence; S72. Use large language models to analyze the changing trends of historical inference results in terms of direction and distance, and identify abnormal results that deviate significantly from the mainstream trend. S73. Based on the consistency analysis results, the large language model outputs the most likely target direction and location estimate. When the number of historical target direction reasoning results reaches a preset threshold (preferably 5 in this embodiment), the text and system prompt information are input into the large language model for analysis. When the number of historical target direction reasoning results is short, only the initial information or empty text is used as input. A target direction memory template is dynamically generated based on the number of historical target reasoning results: if there is no historical reasoning, the system prompts "The large language model currently has no target reasoning"; if there are 1 to 3 historical target reasoning results, the system outputs "The coordinates of the previous / first / second / third target reasoning are (x,y)" sequentially. The large language model analyzes the direction and distance change trends in the historical target direction sequence, identifies abnormal reasoning results that deviate from the mainstream trend, and outputs the most likely target direction and position estimate based on consistency analysis. This provides support for the next direction-level decision-making step. Through the above method, this invention can effectively integrate historical reasoning information with real-time perception results, improving the stability and reliability of target direction prediction, thereby supporting continuous, interpretable, and robust macro-path planning for vehicles in complex or dynamic environments.

[0031] like Figure 1 and Figure 6 As shown, in this embodiment, S8 specifically includes: S81. Construct a set of decision features from the set of drivable directions, the predicted target location, historical trajectory information, and reflection results. S82. Use a large language model to evaluate the consistency between each feasible direction and the target direction; output the priority score or ranking result of each direction. S83. The direction with the highest priority output by the large language model is used as the next planning direction; the direction is used for macro-navigation control and does not involve specific control quantities. This invention constructs a decision feature set by combining the set of drivable directions of the current node, the target direction and position estimation results output by semantic analysis, and the historical trajectory information stored in long-term spatial memory. In the specific implementation of this embodiment, the set of drivable directions is generated from the current road topology; the target direction prediction result comes from the consistency analysis of the historical direction reasoning sequence; the historical trajectory information includes visited directions, corresponding spatial coordinate changes, and whether there is round-trip or cyclical behavior.

[0032] Reasoning based on a large language model involves a prompt template that includes at least the following: the reachability and access status of the current coordinates and adjacent directions, the historical target location reasoning results or their statistical description, the currently executing macro-navigation plan and candidate update instructions, and direction selection constraints (such as the requirement to choose from drivable directions, avoiding unnecessary backtracking, and prioritizing breaking loops when loop trends occur). Under these prompt constraints, the large language model comprehensively evaluates each candidate direction and target direction in terms of spatial consistency, historical decision coherence, and exploration effectiveness. The output includes the current plan execution stage, whether a plan update is needed, the reason for direction selection, and the priority ranking of candidate directions or the optimal direction decision. Ultimately, the direction with the highest priority is selected as the vehicle's next macro-planning direction. This output only represents a direction-level decision and does not involve specific control variables. By introducing structured prompts and historical memory constraints, this invention achieves interpretable reasoning in the direction decision-making process, effectively avoiding local oscillations and blind exploration, and improving the stability and robustness of macro-navigation decisions in complex environments.

[0033] The content of the prompt template is merely illustrative and is not intended to limit the scope of protection of this invention. Any reasonable changes, equivalent substitutions, or optimizations made based on the design concept of this invention shall fall within the scope of protection of this invention's patent rights.

[0034] In this embodiment, S9 specifically includes: S91. Based on the selected direction, after the vehicle has traveled a fixed distance, advance the current position node in the road topology and update the vehicle's planning status. S92. Write the updated node position and selection direction into long-term memory; S93. Determine whether the target node has been reached or the termination condition has been met; if not, return to step S2 and proceed to the next round of macro-path planning.

[0035] See Figure 2 , Figure 2 This diagram illustrates the structure of an intelligent vehicle autonomous navigation system based on a large language model, according to an embodiment of the present invention. The system may include: The feasible space extraction module is used to extract the road directions that the vehicle can travel at the current moment based on the connection relationship of the current node in the road topology; and combined with road attributes, traffic rules and traffic constraints, it eliminates the directions that cannot be traveled, and outputs the set of the vehicle's travel directions at the current moment as the candidate direction set for direction-level planning.

[0036] In this embodiment, the vehicle's operating space is abstracted as a two-dimensional grid space divided by a fixed scale. Preferably, the grid is divided in units of 50 meters, and each grid node corresponds to a candidate position of the vehicle at the macro-navigation level. This approach effectively reduces the size of the state space, decreases computational complexity, and improves the stability and interpretability of subsequent directional reasoning and decision-making processes while ensuring the integrity of the spatial representation. The set of drivable directions is generated from the current grid node based on road connectivity, avoiding the introduction of impassable or traffic-violating directions.

[0037] The perception module is used to acquire the vehicle's current environmental perception information, including image data collected by the visual sensor; to identify landmarks in the image using a visual language model, and to extract the direction angle and distance information of visible landmarks relative to the vehicle; and to construct the spatial observation results of the vehicle relative to multiple landmarks by combining the pre-stored global coordinates of the landmarks, providing perceptual input for target direction reasoning.

[0038] In this embodiment, the visual information can be acquired periodically by an onboard camera, with the acquisition interval determined by the time taken for the vehicle to travel a fixed step of 50 meters. The perception module utilizes a pre-trained visual language model to perform semantic parsing on single or multiple frames of images, identifying landmark objects with stable spatial semantics in the images, such as buildings, intersection structures, or significant man-made facilities. It then outputs the relative orientation and distance range between the "vehicle and landmark" in natural language or structured form, providing semantic observation input for subsequent target direction reasoning.

[0039] The target direction reasoning module receives the landmark spatial observation results output by the perception module, combines the known spatial relationship between the target and landmarks in the navigation point, and constructs a vector relationship model between vehicle, landmark and target; through vector combination and statistical fusion, it infers the direction and rough distance of the vehicle pointing to the target; when there are multiple landmark reasoning results, it removes outliers and outputs stable target direction and distance estimation results, which serve as an important basis for macro-path planning.

[0040] In this embodiment, the direction reasoning does not directly rely on high-precision coordinate calculation, but is based on the combination of relative direction and distance, and further maps the continuous angle space into a discrete direction space. Preferably, the direction space is constrained to an eight-direction representation, thereby transforming the complex continuous decision-making problem into a finite direction selection problem, which helps to reduce decision noise, improve the robustness of macroscopic navigation, and enhance the interpretability of decision results.

[0041] The memory module is used to store and manage the structured trajectory information, direction reasoning results, and decision results during the vehicle's historical navigation process. The memory module includes a long-term spatial memory unit and a semantic memory unit. The long-term spatial memory is used to save node sequences, driving directions, and target reasoning records, while the semantic memory is used to save high-level trajectory semantic information summarized by a large language model, providing historical data support for the reflection and decision-making module.

[0042] In this embodiment, the memory module maintains both short-term operational status records and long-term semantic memory structures. Historical trajectories are stored as a sequence of grid nodes, and directional reasoning results are recorded as direction-distance pairs. The large language model periodically performs semantic compression and summarization on the above information, such as abstracting high-level descriptions like "the vehicle is generally approaching the target in a northeast direction but there are local detours," to support subsequent reflection and decision-making.

[0043] The reflection module is used to read multiple target direction and location reasoning results from long-term spatial memory to form a historical reasoning sequence; and to use a large language model to analyze the changing trends of historical reasoning results in terms of direction and distance, and identify abnormal reasoning results that deviate significantly from the mainstream trend; based on consistency analysis, it outputs the most likely target direction and target location estimate to improve the stability and robustness of target prediction.

[0044] In this embodiment, the reflection module performs semantic analysis on historical direction and distance change trends based on a large language model. It identifies abnormal inference results caused by perceptual noise or temporary occlusion through consistency judgment, thereby avoiding interference from single perceptual errors on the overall navigation strategy. The target direction estimate output by this module serves as an important prior input for subsequent short- and long-term decisions.

[0045] The short-term and long-term decision-making module integrates the set of drivable directions, the target direction prediction results output by the reflection module, semantic memory, and historical trajectory information to construct a set of decision features. The decision features are then input into a large language model to comprehensively evaluate the consistency between each candidate direction and the target direction, and output the priority score or ranking result of each direction. Finally, the direction with the highest priority is selected as the next macro-planning direction for the vehicle, realizing short-term decision-making at the direction level.

[0046] In this embodiment, the inputs to the decision-making module include: the current set of feasible directions, the target direction prediction results output by the reflection module, and the historical driving direction sequence. The large language model, based on a preset prompt template, jointly evaluates the long-term goal consistency and short-term feasibility of each direction, ultimately outputting the direction with the highest priority as the next macro-planning direction. This planned direction is only used for macro-navigation decision-making and does not involve specific steering angles or control variables.

[0047] The path execution and state update module is used to detect the vehicle's travel distance along the selected direction based on the planning direction output by the long-term and short-term decision modules, and advance the current position node in the road topology structure. At the same time, it records the vehicle's travel path, direction selection and target inference results, and writes the relevant information into the long-term spatial memory module. It determines whether the target position has been reached or the termination condition has been met. If not, it triggers a new round of macro-path planning.

[0048] This invention upgrades node path planning to direction-level macro-planning, combining a set of drivable directions, landmark target direction reasoning, and long-term spatial memory and reflection mechanisms. Through zero-shot training of a large language model, it achieves navigation with low dependence on high-precision maps, low computational complexity, and high robustness. It does not require the precise location of the target, infers the direction through vector relationships, and reduces perception errors by fusing the reasoning results. It has temporal consistency and self-correction capabilities, outputs only macro-direction, and can be decoupled and integrated with different vehicle control systems. It is suitable for complex scenarios with incomplete information such as weak maps, and has high engineering practicality and promotional value.

[0049] The above description is merely a few embodiments of this application and is not intended to limit this application in any way. Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any changes or modifications made by those skilled in the art without departing from the scope of the technical solution of this application using the disclosed technical content are equivalent to equivalent implementation cases and fall within the scope of the technical solution.

Claims

1. An autonomous navigation method for intelligent vehicles based on a large language model, characterized in that: The method includes the following steps: S1. Obtain the target instruction information input by the user, analyze the user's intent through a large language model, and generate multiple consecutive navigation points in combination with a custom navigation prompt template to obtain a navigation point sequence. S2. Obtain the vehicle's current position and orientation information and use it as the navigation initialization state; S3. Based on the navigation initialization state, extract several sets of drivable directions from the navigation point sequence, and in combination with preset road driving constraints, remove impassable directions from the extracted sets of drivable directions to generate the set of drivable directions at the current moment. S4. Using visual recognition methods or pre-stored landmark coordinate space vector calculation methods, obtain all vehicle visible landmark information of the current drivable direction set, and generate spatial observation results of the current vehicle relative to the target position. S5. Obtain key spatial information between the target location and each of the vehicle's visible landmarks from the spatial observation results, construct a vector relationship model between the current vehicle and the target location and each of the vehicle's visible landmarks, calculate the direction and distance of the current vehicle to reach the target location by passing through any one or more of the vehicle's visible landmarks, and obtain multiple sets of inference results; S6. Couple multiple sets of reasoning results with timestamps and save them to the historical semantic memory information database; S7. Read all reasoning results between the current vehicle and the target location within a preset time node from the historical semantic memory information database to analyze the consistency of the results of the same vehicle's drivable direction, and remove abnormal or deviating results to dynamically update the historical semantic memory information database. S8. Based on preset navigation rules, retrieve the optimal reasoning result of the current vehicle's arrival at the target location from the historical semantic memory information database, and generate a navigation route.

2. The intelligent vehicle autonomous navigation method based on a large language model according to claim 1, characterized in that: S2 specifically includes: S21. Obtain the vehicle's current pose state information, which includes at least the vehicle's two-dimensional position coordinates and heading angle information in the global coordinate system. S22. Represent the vehicle pose information as a state vector in a global coordinate system. ,in Indicates the vehicle's location. Indicates the vehicle's heading direction; S23. Based on the current pose state, initialize the current position node of this macro-path planning as a reference benchmark for road topology and direction planning.

3. The intelligent vehicle autonomous navigation method based on a large language model according to claim 2, characterized in that: The two-dimensional position coordinates are represented using a two-dimensional discrete space model, specifically including: The vehicle operating environment is divided into regular grid units according to a preset scale, and each grid unit corresponds to a fixed area in the actual road space.

4. The intelligent vehicle autonomous navigation method based on a large language model according to claim 1, characterized in that: S3 specifically includes: S31. Based on the connection relationship of the current node of the vehicle in the road topology, extract all road directions connected to the current node, wherein the road directions are represented as discrete directions; S32. Map each road connection direction to a standardized direction representation, wherein the standardized direction representation includes azimuth angle or direction category; S33. Combining road attributes and traffic rules, directions that do not meet the traffic conditions are eliminated to obtain the set of permissible directions for vehicles at the current moment. .

5. The intelligent vehicle autonomous navigation method based on a large language model according to claim 4, characterized in that: The set of discrete directions is set to eight directions.

6. The intelligent vehicle autonomous navigation method based on a large language model according to claim 1, characterized in that: S4 specifically includes: S41. Acquire current vehicle visual data, perform target recognition of landmarks using a visual language model, and output the orientation angle of visible landmarks relative to the vehicle's current position. The system obtains the set of landmarks currently observable by the vehicle, with each landmark corresponding to pre-stored global coordinate information for subsequent spatial calculations. S42. For each visible landmark, estimate the spatial distance between the vehicle and the landmark based on the vehicle positioning accuracy. ; S43. Combine the azimuth and distance of each landmark to form spatial observation results. ,in This indicates the number of the landmark building.

7. The intelligent vehicle autonomous navigation method based on a large language model according to claim 6, characterized in that: The spatial distance The distance is predicted by geometric distance estimation. The geometric distance estimation input includes the target landmark category information output by the visual language model, the pixel height or pixel scale information of the target in the image, and the prior information of the true height of the corresponding landmark. The distance estimation model is as follows: ； in For scale calibration coefficients, The equivalent focal length of the camera. The actual height of the landmark building The height of the projected pixels of the target in the image. This is the distance offset term.

8. The intelligent vehicle autonomous navigation method based on a large language model according to claim 7, characterized in that: S5 specifically includes: S51. Obtain spatial observation results, as well as the spatial relationship between the target location and any landmark and the spatial relationship between any two landmarks; S52. Construct spatial vectors between vehicles and landmarks, between targets and landmarks, and between landmarks; S53. Infer the spatial vector from the vehicle to the target by using the vector superposition relationship; S54. Perform statistical fusion on multiple landmark inference results, remove outliers, and obtain stable target direction and distance estimation results. .

9. The intelligent vehicle autonomous navigation method based on a large language model according to claim 1, characterized in that: In step S6, it is suitable to use a large language model to read the most recent structured historical trajectory information from the historical semantic memory information database, convert it into a natural language description form, input it into the large language model for analysis and summarization, and store it as semantic memory; In step S7, multiple target direction reasoning results are read from the historical semantic memory information database to form a historical reasoning sequence. The changing trend is analyzed and abnormal results are identified through a large language model, and the most likely target direction and position estimate are output.

10. The intelligent vehicle autonomous navigation method based on a large language model according to claim 9, characterized in that: In step S8, the set of feasible directions, the prediction results of the target direction, and the historical trajectory information are constructed into a set of decision features. A large language model combined with prompt templates is used to evaluate the consistency between each feasible direction and the target direction, output the priority score or ranking result of each direction, and select the direction with the highest priority as the next planning direction.

Images