An adaptive preview control method based on experience learning and large language model

Abstract

The present invention discloses an adaptive preview control method based on experience learning of a large language model. The standard sentence structure, few-shot experience and insights, and self-correction mechanism based on Reflexion used in the GPT4 reasoning prompt process are of great significance to the improvement of GPT4 parameter adjustment performance. The standard sentence structure can help GPT4 focus on the information of a specific position and enhance its understanding of the tracking task scene; the few-shot experience provides a reference for similar cases for the reasoning parameter adjustment process of GPT4, which helps it capture the correlation between tasks; the insights stored in the long-term memory library can provide macro guidance for related tasks and improve the generalization of preview parameter adjustment; the self-correction mechanism based on Reflexion enables the algorithm to autonomously solve failed tasks again, saving manpower, and the content of the reflective reasoning can be intercepted to generate insights.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an experience-based adaptive preview control method based on a large language model. Background Art

[0002] In modern warehouse logistics management, forklifts are an indispensable and important means of transportation. Pre-aiming in tracking control, that is, the advance response to the path, directly affects its operating efficiency and safety in autonomous driving. Traditional forklift preview algorithms often rely on empirical formulas with fixed parameters, which have limited generalization and are difficult to cope with complex and changeable storage and handling tasks. There are problems such as low control accuracy and huge calibration workload. Therefore, a preview control method that can autonomously learn from human experience to cope with a variety of handling tasks is crucial to the tracking performance of forklifts when loading and unloading goods.

[0003] In tracking control, both the traditional pure tracking algorithm and the Stanley algorithm use the empirical formula of the preview distance to select the appropriate preview point. The lateral error between the preview point coordinates and the vehicle position coordinates is used to calculate the front wheel steering angle for control. However, due to the accuracy requirements of the forklift to align with the shelf pallet socket and the diversity of the handling trajectory, the fixed parameter preview distance empirical formula is difficult to maintain the best control quality under different conditions. Based on this, in order to better adapt to the control accuracy requirements and complex handling tasks, industrial applications manually calibrate the preview parameters according to different handling tasks, and use a MAP table to improve tracking performance. However, this method consumes a lot of time and human resources. The industry urgently needs intelligent control methods to learn from human experience in driving forklifts and replace the tedious manual calibration work.

[0004] In recent years, large language models (LLMs) with emergent capabilities have played a key role in helping specific artificial intelligence systems understand and perceive the surrounding environment, interpret complex task descriptions, formulate task plans, seamlessly collaborate with other system modules, adapt to dynamic environments, and promote social interactions with humans through natural language communication. LLMs have excellent cognitive, learning, and reasoning capabilities, and they lay the foundation for adaptive preview control methods based on experience learning.

[0005] In summary, for the actual engineering problems of forklift tracking control, using LLM to learn human experience knowledge for adaptive preview is a method worth exploring. Summary of the invention

[0006] The purpose of the present invention is to provide an adaptive preview control method based on experience learning of a large language model in order to address the technical defects in the prior art.

[0007] The technical solution adopted to achieve the purpose of the present invention is:

[0008] A forklift preview control method based on experience learning and adaptive large language model includes the following steps:

[0009] Step 1: Calculate the distance error e of the autonomous forklift in real time based on the target driving trajectory of the autonomous forklift d and heading error e θ , initialize the empirical formula for preview distance;

[0010] Step 2, simplify the autonomous forklift into a bicycle with 2 degrees of freedom, and obtain its bicycle kinematic model for vehicle state update;

[0011] Step 3, a pure tracking algorithm with preview distance is used to control the heading angle of the forklift, and PID is used to control the acceleration of the forklift;

[0012] Step 4: The large language model GPT4 generates a text description of the tracking scene based on the driving environment, and retrieves similar scenes-preview parameters-reward trajectories from the short-term memory based on the text description of the tracking scene, retrieves guiding insights from the long-term memory, and inputs them into the large language model GPT4 in a few-shot manner;

[0013] Step 5: The large language model GPT4 infers and selects the preview distance parameter in the preview distance empirical formula based on the few-shot scenario, preview parameters, reward trajectory and insights;

[0014] Step 6: Assign the selected preview distance parameter to the preview distance empirical formula, and calculate the distance error e obtained in step 1. d , heading error e θ Conduct rule judgment and manual semantic judgment on the tracking control effect, wherein the rule judgment is whether the tracking control effect meets the accuracy requirements, and the manual semantic judgment is whether the forklift can be inserted into the shelf pallet;

[0015] Step 7: If the tracking control effect meets the accuracy requirement and the shelf tray can be inserted, the scene-preview parameter-return trajectory is written into the short-term memory bank, thereby updating the short-term memory bank;

[0016] Step 8. If the tracking control effect does not meet the accuracy requirements of step 7 and the execution conditions of the shelf pallet, use Reflexion to reflect and correct the loop step 6 until the accuracy requirements are met. Write the corrected scene-preview parameters-reward trajectory into the short-term memory bank, select the reasoning guidance content in Reflexion as insights, and write it into the long-term memory bank.

[0017] In the above technical solution, in step 1, the point on the target driving trajectory closest to the current actual position is found by using the dichotomy method, and the distance error e of the autonomous forklift is calculated.d and heading error e θ , and at the same time, with the positioning center as the center of the circle and the preview distance as the radius, find the point closest to the target driving trajectory in the forward direction of the autonomous forklift as the preview point. The empirical formula for the preview distance is expressed as:

[0018] d = av 2 +bv+c

[0019] Wherein, d is the preview distance, v is the vehicle speed, and a, b, and c are the preview distance parameters to be adjusted.

[0020] In the above technical solution, in step 2, the bicycle kinematics model is:

[0021]

[0022] Among them, L is the forklift wheelbase, δ is the tire turning angle, (x, y) is the vehicle coordinate, and θ is the vehicle heading angle.

[0023] In the above technical solution, in step 3, the heading angle control law of the pure tracking algorithm is:

[0024]

[0025] Among them, α is the angle between the vehicle body and the preview point.

[0026] In the above technical solution, in step 4, in order to facilitate GPT4's understanding of the tracking task, the tracking scene text description follows a standard sentence structure, and the description contains static planning curve details and dynamic information about the forklift and surrounding obstacles in the scene;

[0027] In the above technical solution, in step 4, during the retrieval process in the memory library, the tracking scene text description is also embedded in the vector, and the vector is used to perform similarity query in the short-term memory library and the long-term memory library to search for the top k similar situations. Thus, the queried scene-preview parameter-reward trajectory and guiding insights are assembled into a few-shot experience prompt, which is input into GPT4 in the form of a thought chain to select the preview distance parameters a, b, c.

[0028] In the above technical solution, in step 6, the accumulated distance error e d and heading error e θ A rule judgment based on the limit threshold will be made, and whether the forklift is finally inserted into the shelf pallet (i.e. the success or failure of the task) will be manually fed back in natural language. The human prompts and guidance sentences in the process will also be recorded to assist in the generation of insights in the long-term memory bank.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention constructs an architecture of experience learning based on a large language model, involving an interactive environment, an autonomous vehicle, and a memory library. Through manual feedback on the performance of the autonomous vehicle in different tracking tasks, the present invention realizes a reinforcement learning process with natural language as the carrier. As the number of tasks experienced by the forklift increases, the scene-preview parameters-reward trajectory and insights will gradually increase, and the performance of adaptive preview will continue to improve, realizing continuous learning. This not only improves the generalization of the preview algorithm, but also replaces the traditional complex and time-consuming manual calibration process.

[0031] 2. The standard sentence structure, few-shot experience and insights, and the self-correction mechanism based on Reflexion used in the GPT4 reasoning prompt process in the present invention are of great significance to the improvement of GPT4's parameter adjustment performance. The standard sentence structure can help GPT4 focus on the information at a specific location and strengthen its understanding of the tracking task scenario; the few-shot experience provides a reference for similar cases for the GPT4 reasoning parameter adjustment process, which helps it capture the correlation between tasks; the insights stored in the long-term memory library can provide macro guidance for related tasks and improve the generalization of pre-aiming parameter adjustment; the self-correction mechanism based on Reflexion enables the algorithm to autonomously solve failed tasks again, saving manpower, and the content of the reflective reasoning can be intercepted to generate insights.

[0032] 3. The adaptive preview control method based on experience learning of a large language model proposed in the present invention has good comprehensive control performance, which solves the problems of low control accuracy and poor generalization ability when autonomous forklifts face complex and diverse tracking tasks in warehouse environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Shown is the overall architecture diagram of the control method of the present invention. DETAILED DESCRIPTION

[0034] The present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] like Figure 1 As shown, a forklift preview control method based on experience learning and adaptive large language model includes the following steps:

[0036] Step 1: Based on the target driving trajectory of the self-driving forklift, use the dichotomy method to find the point on the target trajectory closest to the current actual position, and calculate the distance error e of the self-driving forklift at the current moment. d and heading error e θ, and at the same time, with the positioning center as the center of the circle and the preview distance as the radius, find the point in the vehicle's driving trajectory that is closest to the target driving trajectory as the preview point, and then obtain the preview point coordinates (X p ,Y p ), the empirical formula for preview distance is expressed as:

[0037] d = av 2 +bv+c

[0038] Wherein, d is the preview distance, v is the vehicle speed, and a, b, and c are the preview distance parameters to be adjusted.

[0039] Step 2: Simplify the self-driving forklift into a bicycle with 2 degrees of freedom, and obtain its bicycle kinematic model for vehicle state update. The bicycle kinematic model is:

[0040]

[0041] Among them, v is the vehicle speed, L is the forklift wheelbase, δ is the tire turning angle, (x, y) is the vehicle coordinate, and θ is the vehicle heading angle.

[0042] Step 3: Use the heading angle control law of the pure tracking algorithm with preview distance to control the forklift heading angle:

[0043]

[0044] Among them, α is the angle between the vehicle body and the preview point, and the vehicle acceleration is controlled based on PID using the error between the vehicle speed and the given target speed.

[0045] Step 4: The large language model GPT4 generates a text description of the tracking scene based on the driving environment. The tasks of the tracking scene are described in a standardized manner, including the coordinate sequence of the planned trajectory, the coordinates of the obstacles, and the future motion trajectory of the obstacles. The Word2Vec network is used to transform it into a retrieval vector, which is used to retrieve similar scenes-pre-aiming parameters-reward trajectories from the short-term memory library, and to retrieve guiding insights from the long-term memory library, searching for the first k similar situations. If k≤3, all scenes-pre-aiming parameters-reward trajectories are used to construct few-shot experience. If k≥3, 3 are randomly selected from the k similar situations for few-shot experience construction. At the same time, the insights in the long-term memory library with the most appearances in similar scenes are queried to guide GPT4 reasoning and adjust the preview distance parameters a, b, and c.

[0046] In step 5, the large language model GPT4 first receives system prompts to inform it of the task content and the standardized description of subsequent input. Through this instruction, the performance of its parameter adjustment is activated and improved. Then the scene description in step 4, the few-shot scene-pre-aiming parameters-reward trajectory and insights are input to the large language model GPT4, and the prompt method of the thinking chain is used to make the large language model GPT4 reason step by step to select the pre-aiming distance parameters in the pre-aiming distance empirical formula.

[0047] Step 6: Assign the selected preview distance parameter to the preview distance empirical formula, and calculate the distance error e obtained in step 1. d , heading error e θ Conduct rule judgment and manual semantic judgment on the tracking control effect. The rule judgment is whether the tracking control effect meets the accuracy requirement, and the manual semantic judgment is whether the forklift can be inserted into the shelf pallet.

[0048] Step 7: If the tracking control effect meets the accuracy requirements and the shelf pallet can be inserted, the scene-preview parameter-report trajectory is written into the short-term memory bank, thereby updating the short-term memory bank.

[0049] Step 8: If the tracking control effect does not meet the accuracy requirements of step 7 and the execution conditions of the shelf pallet, use Reflexion for reflection correction, so that GPT4 can generate reasoning and solutions for the reasons for task failure in reflection, and give the adjustment results after reflection. For the results after reflection correction, loop step 6 until the accuracy requirements are met, write the correction scene-preview parameters-return trajectory into the short-term memory bank, and select the reasoning guidance content in Reflexion as insights and write them into the long-term memory bank.

[0050] The above is only a preferred embodiment of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A forklift preview control method based on experience learning and adaptive large language model, characterized in that: The following steps are involved: Step 1: Calculate the distance error e of the autonomous forklift in real time based on the target driving trajectory of the autonomous forklift d and heading error e θ , initialize the empirical formula for preview distance; Step 2, simplify the autonomous forklift into a bicycle with 2 degrees of freedom, and obtain its bicycle kinematic model for vehicle state update; Step 3, a pure tracking algorithm with preview distance is used to control the heading angle of the forklift, and PID is used to control the acceleration of the forklift; Step 4: The large language model GPT4 generates a text description of the tracking scene based on the driving environment, and standardizes the description according to the task of the tracking scene, including the coordinate sequence of the planned trajectory, the coordinates of the obstacle, and the future motion trajectory of the obstacle. The description is converted into a retrieval vector using the Word2Vec network, and the retrieval vector is used to retrieve similar scenes-pre-aiming parameters-reward trajectories from the short-term memory library, and to retrieve guiding insights from the long-term memory library, and input into the large language model GPT4 using a few-shot method. Step 5: The large language model GPT4 infers and selects the preview distance parameter in the preview distance empirical formula based on the few-shot scenario, preview parameters, reward trajectory and insights; Step 6: Assign the selected preview distance parameter to the preview distance empirical formula, and calculate the distance error e obtained in step 1. d , heading error e θ Conduct rule judgment and manual semantic judgment on the tracking control effect, wherein the rule judgment is whether the tracking control effect meets the accuracy requirements, and the manual semantic judgment is whether the forklift can be inserted into the shelf pallet; Step 7: If the tracking control effect meets the accuracy requirement and the shelf tray can be inserted, the scene-preview parameter-return trajectory is written into the short-term memory bank, thereby updating the short-term memory bank; Step 8: If the tracking control effect does not meet the accuracy requirements of step 7 and the execution conditions of the shelf pallet, use Reflexion to reflect and correct the loop step 6 until the accuracy requirements are met, write the corrected scene-preview parameters-return trajectory into the short-term memory bank, and select the guiding content of the reasoning in Reflexion as insights and write them into the long-term memory bank.

2. The control method according to claim 1, characterized in that: In step 1, the binary search method is used to find the point on the target driving trajectory closest to the current actual position, and the distance error e of the autonomous forklift is calculated. d and heading error e θ , and at the same time, with the positioning center as the center of the circle and the preview distance as the radius, find the point closest to the target driving trajectory in the forward direction of the autonomous forklift as the preview point.

3. The control method according to claim 2, characterized in that: In step 1, the empirical formula for the preview distance is expressed as: d=off 2 +bv+c Wherein, d is the preview distance, v is the vehicle speed, and a, b, and c are the preview distance parameters to be adjusted.

4. The control method according to claim 1, characterized in that: In step 2, the bicycle kinematics model is: Among them, L is the forklift wheelbase, δ is the tire turning angle, (x, y) is the vehicle coordinate, and θ is the vehicle heading angle.

5. The control method according to claim 1, characterized in that: In step 3, the heading angle control law of the pure tracking algorithm is: Among them, α is the angle between the vehicle body and the preview point.

6. The control method according to claim 3, characterized in that: In step 4, in order to facilitate GPT4's understanding of the tracking task, the tracking scene text description follows a standard sentence structure, which contains static planning curve details and dynamic information about the forklift and surrounding obstacles in the scene.

7. The control method according to claim 6, characterized in that: In step 4, during the retrieval process in the memory library, the text description of the tracking scene is embedded into a vector, and the vector is used to perform a similarity query in the short-term memory library and the long-term memory library to search for the top k similar situations.

8. The control method according to claim 7, characterized in that: In step 4, the queried scene-preview parameter-reward trajectory and guiding insights are assembled into a few-shot experience prompt, which is input into GPT4 in the form of a thought chain to select the preview distance parameters a, b, and c.

9. The control method according to claim 1, characterized in that: In step 6, the accumulated distance error e d and heading error e θ A rule judgment based on the limit threshold will be performed.

10. The control method according to claim 9, characterized in that: In step 6, whether the forklift is finally inserted into the shelf pallet is manually fed back in natural language, and the human prompts and guidance statements during the process will also be recorded to assist in the generation of insights in the long-term memory bank.

Images