AUV motion control method and system based on LLM human-computer interaction framework

Abstract

The present invention belongs to the technical field of robot motion control and relates to an AUV motion control method and system based on an LLM human-computer interaction framework. The method comprises: acquiring a target AUV and constructing a target AUV simulation platform based on ROS2; integrating LLM and NMPC in ROS2 to form an LLM human-computer interaction framework; acquiring user instructions, and the LLM human-computer interaction framework converting the user instructions into control instructions, and transmitting the control instructions to the target AUV simulation platform; the AUV in the AUV simulation platform executes the control instructions, generates trajectory tracking and displays it. The present invention applies a large language model to the field of underwater robots, combines the human-computer interaction framework with the traditional NMPC control method, utilizes high-level decisions from LLM to realize control of the underlying NMPC, and effectively improves the control flexibility of the AUV in complex experimental scenarios through the interactive display interface of Rviz2.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot motion control, and relates to an AUV motion control method and system based on an LLM human-computer interaction framework. Background Art

[0002] In recent years, various types of robots have been continuously developed and widely used on land, at sea, and in the air. Autonomous underwater vehicles (AUVs) have garnered widespread attention due to their ability to assist humans in performing dangerous and challenging tasks such as seafloor mapping, pipeline inspection, and pollution source location. Motion control is crucial for improving AUV performance in these tasks. However, due to the complexity of ocean experiments, enhancing the interaction between AUVs and users to achieve flexible motion control is becoming a significant challenge.

[0003] Large Language Models (LLMs) have recently demonstrated significant potential for understanding and processing natural language, attracting significant attention, particularly in the field of interactive robotic control. Traditional robotic control methods often rely on predefined rules and mathematical models. While reliable for specific tasks, these approaches lack flexibility in dynamic and changing environments, are slow to adapt to new tasks and environments, and struggle to rapidly adjust. Furthermore, the complex programming logic, parameter tuning, and debugging required in traditional control systems are not only time-consuming and labor-intensive, but also significantly increase development and maintenance costs. These issues are particularly acute in technically demanding and complex environments, such as marine experiments involving autonomous underwater vehicles (AUVs). The introduction and development of LLMs has ushered in a significant breakthrough in this field. Through natural language interaction, operators can issue task instructions to robots in an intuitive and flexible manner, avoiding lengthy and complex programming processes. This advancement not only lowers the barrier to entry for system users but also makes robotic control more adaptable and scalable. LLMs can significantly enhance the user experience, particularly in scenarios where humans and robots collaborate, by allowing users to describe tasks in a language they are familiar with, rather than relying on technically proficient programming knowledge.

[0004] However, while LLMs have demonstrated exceptional performance in high-level tasks such as natural language understanding and mission planning, they face significant challenges in low-level control tasks. Low-level control tasks, such as precise attitude adjustment, path planning, and real-time trajectory tracking, require robots to possess fast response and high-precision operational capabilities. These tasks typically involve continuous sensor feedback and real-time computation, but LLMs, due to their computational complexity and resource requirements, struggle to provide sufficient processing speed and control accuracy within these time constraints. Therefore, despite their revolutionary potential for high-level task decision-making and human-robot interaction, LLMs remain limited in practical low-level robotic control tasks. For these tasks, traditional control methods, such as PID control and model predictive control (MPC), remain dominant due to their lightweight and efficient computational architectures, necessitating real-time and high-precision control. However, traditional control methods suffer from poor flexibility in AUV control in complex sea trial scenarios. Summary of the Invention

[0005] The purpose of the present invention is to provide an AUV motion control method and system based on the LLM human-computer interaction framework to solve the problem of poor control flexibility of AUVs by traditional control methods in complex sea trial scenarios in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention discloses an AUV motion control method based on the LLM human-computer interaction framework, comprising:

[0008] Acquire the target AUV and build a target AUV simulation platform based on ROS2; integrate LLM and NMPC in ROS2 to form the LLM human-computer interaction framework;

[0009] Obtain user instructions, the LLM human-computer interaction framework converts user instructions into control instructions, and transmits the control instructions to the target AUV simulation platform;

[0010] The AUV in the target AUV simulation platform executes control instructions, generates trajectory tracking and displays it.

[0011] Preferably, the target AUV simulation platform is constructed based on ROS2, specifically: setting a URDF robot description file in ROS2 and constructing a simulation scene, such as an ocean, island, etc. scene; and obtaining the target AUV simulation platform.

[0012] Preferably, the LLM human-computer interaction framework includes:

[0013] The LLM high-level decision module is used to understand and analyze user instructions, generate high-level decisions, and send the high-level decisions to the NMPC low-level control module;

[0014] The NMPC bottom-level control module is used to receive high-level decisions, solve high-level decisions to obtain control instructions, and transmit the control instructions to the target AUV simulation platform.

[0015] Preferably, the high-level decision making includes: generating a trajectory, modifying a desired position, modifying a desired speed, and modifying an NMPC weight matrix.

[0016] Preferably, the integration of LLM and NMPC in ROS2 to form an LLM human-computer interaction framework specifically includes:

[0017] In the startup file of the LLM high-level decision module, add ROS2 publisher and subscriber nodes and define data types related to user instructions;

[0018] In the startup file of the NMPC underlying control module, add the ROS2 publisher and subscriber nodes;

[0019] In Rviz2 of ROS2, the AUV model, coordinate system, and sensor are added to the toolbar to obtain the LLM human-computer interaction framework.

[0020] Preferably, the data types related to the user instruction include: position, speed and posture information.

[0021] Preferably, the trajectory tracking is displayed in an LLM human-computer interaction framework.

[0022] In a second aspect, the present invention discloses an AUV motion control system based on the LLM human-computer interaction framework, comprising:

[0023] Build a unit to acquire the target AUV and construct a target AUV simulation platform based on ROS2; integrate LLM into ROS2 to form an LLM human-computer interaction framework;

[0024] The instruction generation unit is used to obtain user instructions. The LLM human-computer interaction framework converts the user instructions into control instructions and transmits the control instructions to the target AUV simulation platform;

[0025] The instruction execution unit executes the control instructions of the AUV in the target AUV simulation platform, generates trajectory tracking and displays it.

[0026] In a third aspect, the present invention discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the AUV motion control method based on the LLM human-computer interaction framework described above are implemented.

[0027] In a fourth aspect, the present invention discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the AUV motion control method based on the LLM human-computer interaction framework described above.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention applies the large language model to the field of underwater robots, combines the human-computer interaction framework with the traditional NMPC control method, uses high-level decisions from LLM to realize the control of the underlying NMPC, and effectively improves the control flexibility of AUV in complex experimental scenarios through the interactive display interface of Rviz2. The present invention establishes a six-degree-of-freedom model of the underwater robot in a highly realistic Gazebo physical environment, adopts nonlinear model predictive control as the underlying motion controller, fully considers the nonlinear coupling terms of the underwater robot, and adopts the new Acados solver to effectively improve the control accuracy and efficiency of the AUV. The present invention is completed in the robot operating system ROS2, and has made significant improvements compared to ROS1 in terms of distributed, multi-platform and real-time support. By writing URDF robot description files in ROS2, different robot models can be easily added, and different scenes can be rendered through plug-ins to meet the robot control, planning and recognition and other task requirements in different fields, and has broad application prospects. The present invention combines LLMs with traditional control methods, using the language understanding capabilities of LLMs to handle high-level task decisions, while leaving low-level control tasks to traditional methods or hybrid architectures to ensure the robot's accuracy and response speed, ultimately achieving an efficient and flexible control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 is a flow chart of the method of the present invention;

[0032] Figure 2is a block diagram of a human-computer interaction framework in an embodiment of the present invention;

[0033] Figure 3 This is a case diagram of LLM prompts in an embodiment of the present invention;

[0034] Figure 4 This is a block diagram of ROS2 message transmission in an embodiment of the present invention;

[0035] Figure 5 This is a diagram of AUV experimental results in an embodiment of the present invention;

[0036] Figure 6 Schematic diagram of an LLM-based human-computer interaction framework for AUV motion control provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0038] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0039] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0040] In the description of the embodiments of the present invention, it should be noted that if the terms "upper," "lower," "horizontal," "inner," etc. appear, the orientation or positional relationship indicated is based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the inventive product is typically placed when in use. These terms are merely for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or component referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. In addition, the terms "first," "second," etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0041] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0042] In the description of the embodiments of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0043] The present invention is described in further detail below with reference to the accompanying drawings:

[0044] This paper proposes a human-computer interaction framework based on LLM. In complex sea trial scenarios, natural language instructions provide researchers with a familiar and flexible way to describe tasks, eliminating the need for complex and repetitive programming, thereby reducing the frequency of AUV recovery and debugging, and significantly reducing unnecessary sea trial workload. This paper addresses the poor flexibility and interactivity of the above-mentioned traditional control methods by combining a large language model with traditional optimal control methods. The specific method is as follows:

[0045] See also Figure 1 , an AUV motion control method based on the LLM human-computer interaction framework, including:

[0046] S1: Acquire the target AUV and build a target AUV simulation platform based on ROS2; integrate LLM and NMPC in ROS2 to form the LLM human-computer interaction framework;

[0047] S2: Obtain user instructions, the LLM human-computer interaction framework converts the user instructions into control instructions, and transmits the control instructions to the target AUV simulation platform;

[0048] S3: The AUV in the target AUV simulation platform executes the control instructions, generates trajectory tracking and displays it.

[0049] In some embodiments, the target AUV simulation platform is constructed based on ROS2, specifically: setting a URDF robot description file in ROS2 and constructing a simulation scene, such as an ocean, island, etc. scene; and obtaining the target AUV simulation platform.

[0050] In some embodiments, the LLM human-computer interaction framework includes:

[0051] The LLM high-level decision module is used to understand and analyze user instructions, generate high-level decisions, and send the high-level decisions to the NMPC low-level control module;

[0052] The NMPC bottom-level control module is used to receive high-level decisions, solve high-level decisions to obtain control instructions, and transmit the control instructions to the target AUV simulation platform.

[0053] In some embodiments, the high-level decision making includes generating a trajectory, modifying a desired position, modifying a desired velocity, and modifying an NMPC weight matrix.

[0054] In some embodiments, the integration of LLM and NMPC in ROS2 to form an LLM human-computer interaction framework specifically includes:

[0055] In the startup file of the LLM high-level decision module, add ROS2 publisher and subscriber nodes and define data types related to user instructions;

[0056] In the startup file of the NMPC underlying control module, add the ROS2 publisher and subscriber nodes;

[0057] In the Rviz2 interface of ROS2, the AUV model, coordinate system, and sensor are added to the toolbar to obtain the LLM human-computer interaction framework.

[0058] In some embodiments, the data types related to the user instructions include: position, speed and posture information.

[0059] In some embodiments, the trajectory tracking is displayed in an LLM human-computer interaction framework.

[0060] In some embodiments, an AUV motion control method based on the LLM human-computer interaction framework may include the following steps:

[0061] Step 1: Design the target AUV simulation platform and write the URDF robot model file of the AUV used; integrate LLM and NMPC in ROS2 to form the LLM human-computer interaction framework;

[0062] Step 2: Get user instructions, write high-level decisions based on LLM according to user instructions, and complete them in combination with ROS2 mechanism;

[0063] Step 3: Write the driver file based on the underlying NMPC controller according to the high-level decision, and transfer it to the target AUV simulation platform in combination with ROS2;

[0064] Step 4: The AUV in the AUV simulation platform executes the control instructions, generates and displays the trajectory, and realizes the motion control of the AUV.

[0065] See also Figure 2 In an embodiment of the present invention, a human-computer interaction framework based on LLM is used for AUV motion control, specifically including:

[0066] Step 1: Design the target AUV simulation platform and compile the URDF robot model file for the AUV to be used. LLM and NMPC are integrated into ROS2 to form the LLM human-computer interaction framework. Baidu's ENIRE Bot is used as the LLM to understand and analyze user control commands and connect to the AUV simulation platform through ROS2 mechanisms.

[0067] Step 2: The LLM high-level decision module designs navigation and optimization functions based on the predefined functions of the prompt engineering design, including generating trajectories, modifying the desired position and velocity, and modifying the NMPC weight matrix;

[0068] In step 3, the NMPC underlying control module calculates the thruster control input through iterative optimization and sends it to the AUV in the Gazebo simulation platform (AUV simulation platform) to complete the trajectory tracking task.

[0069] See also Figure 3 This figure describes an embodiment of analyzing user speed instructions based on the LLM human-computer interaction framework. This embodiment may specifically include the following steps:

[0070] Step 1: Design a prompt function in the ROS2 system to modify the current desired speed of the AUV;

[0071] Step 2: Use JSON format to design the information input to the AUV simulation platform;

[0072] Step 3: Design the subscriber of the ROS2 node to subscribe to the speed status of the AUV and lay the foundation for the callback function;

[0073] Step 4: Design a callback function in the ROS2 system to automatically detect the AUV status information in the simulation system and set the expected speed according to user instructions.

[0074] See also Figure 4 In the present invention, the LLM high-level decision module and the NMPC low-level control module are completed through the ROS2 node message mechanism, which specifically includes the following steps:

[0075] Step 1: In the LLM startup file, add ROS2 publisher and subscriber nodes to detect the AUV status and publish the expected command message;

[0076] Step 2: Define the data types related to user commands, including position, speed, and posture information, for analysis of command messages;

[0077] Step 3: In the NMPC startup file, define the ROS2 publisher and subscriber nodes, and implement communication with the AUV in the Gazebo environment through bridging.

[0078] Step 4: Start the Rviz2 tool, add the AUV model, coordinate system, and sensor to the toolbar, and display the current AUV status information and trajectory tracking effect on the visual interface;

[0079] Step 5: In the launched Gazebo and Rviz2 interfaces, you can observe the motion status of the AUV. In case of emergency, human intervention can be performed to simulate the reality and prevent AUV abnormal accidents.

[0080] Test Example

[0081] In the following specific embodiments of the present invention, the open-source BlueROV2 from BlueRobotics was selected as the research object of the invention. This AUV is a fully driven underwater robot equipped with six thrusters as control actuators, driving the AUV's six degrees of freedom. The present invention example is based on the Humble version of the ROS2 system running on Linux Ubuntu 22.04. The simulation framework is implemented in Ignition Gazebo, and the real-time position and motion trajectory are displayed using Rviz2. The entire invention example was completed on a computer equipped with an i5-13500HX @ 2.70GHz CPU, 16GB of memory, and an NVIDIA GeForce RTX4060 GPU.

[0082] The specific implementation steps include: setting the AUV's initial position, with both the x and y coordinates set to 0. Due to the buoyancy effect of the physical properties of seawater in the Gazebo simulation environment, the initial z coordinate is approximately -0.1. Secondly, the initial prediction and control steps of the NMPC controller are set to 10, the control input weight matrix of the AUV thruster is set to diag(0.1, 0.1, 0.1, 0.1, 0.1), and the state weight matrix of the AUV model state equation is set to diag(10, 10, 10, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1). This matrix shows that this method focuses on the AUV's position tracking accuracy. The sampling time is set to 0.1s, which meets the transmission requirements of actual AUV sensors.

[0083] First, use the designed Launch file to start the AUV simulation platform, including the Gazebo sim and Rviz2 interfaces, and set the display coordinate system, sensors, etc. in the interface. Secondly, start the LLM and NMPC control main program files, and expand them through the following two examples:

[0084] (1) Navigation command test

[0085] To test the accuracy of the human-computer interaction framework proposed in this invention in analyzing user navigation commands, this embodiment inputs the text "Set the speeds of the x-axis and z-axis to 0.7 m / s and -0.2 m / s, respectively, and the y-coordinate follows a sine wave with a period of 20 seconds and an amplitude of 5 meters" into the AUV's control interface. After analyzing the text using Baidu's ERINE Bot 4.0 large language model, the prompt engineering set by this invention correctly understands the user's command; generates reasonable ROS2 code and publishes it to the AUV's terminal, and observes the rationality of the published message through the terminal; after receiving the ROS2 command, the AUV in the Gazebo environment transmits it to the specific ROS2 interface of this invention, allowing the AUV to move along the desired trajectory under NMPC control, such as Figure 5 shown.

[0086] (2) Bottom-level optimization control instruction test

[0087] In addition to testing navigation commands, users can enter low-level NMPC control-related commands through the interactive interface to change the controller's weight matrix and other parameters. In this example, the command "Increase weights, set the control input weight matrix to diag(0.2, 0.2, 0.2, 0.2, 0.2)" is entered. The LLM analyzes this command and publishes it as a ROS2 message to a specifically designed acceptor function in the NMPC controller, thereby modifying the NMPC weight matrix. To ensure that user commands do not affect the AUV's normal operation, the present invention sets a waiting time of 5 seconds. This means that if the system does not receive the correct command from the user within 5 seconds, the AUV will operate according to the default parameters.

[0088] The framework also allows for modification of other NMPC parameters, such as the prediction step size. By inputting the command "Increase prediction step size to 15," the prediction step size Np is set to 15. The interactive interface returns a message from ROS2, allowing the user to confirm that the LLM correctly understood the user's message and completed the prediction step size modification. After the modification is complete, the default trajectory tracking task automatically begins. A comparison of the trajectory tracking results before and after the experiment further demonstrates the effectiveness of the framework.

[0089] In summary, the embodiments of the present invention disclose a human-computer interaction framework based on LLM for AUV motion control. The framework effectively improves the flexibility of AUV sea trial program debugging through LLM; the NMPC underlying control ensures the motion control accuracy of the AUV by establishing a nonlinear six-degree-of-freedom motion model; the visual interaction method improves the flexibility and interactivity of the user's control of the AUV; this method introduces the large prediction model into the field of underwater robots, which can effectively improve the convenience of researchers in deploying underwater robot tasks, and each component unit is simple to implement in ROS2, which is suitable for robot experimental testing in other fields and has strong applicability.

[0090] The following are device embodiments of the present invention, which can be used to perform the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0091] See also Figure 6 This application also discloses an AUV motion control system based on the LLM human-computer interaction framework, including:

[0092] A construction unit is used to acquire the target AUV and build a target AUV simulation platform based on the ROS2 system; LLM is integrated into ROS2 to form an LLM human-computer interaction framework;

[0093] The instruction generation unit is used to obtain user instructions. The LLM human-computer interaction framework converts the user instructions into control instructions and transmits the control instructions to the target AUV simulation platform;

[0094] The instruction execution unit,the AUV in the AUV simulation platform executes control instructions,generates trajectory tracking and displays it.

[0095] The present application also discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the AUV motion control method based on the LLM human-computer interaction framework described above are implemented.

[0096] The present application also discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the AUV motion control method based on the LLM human-computer interaction framework are implemented.

[0097] In some embodiments, a computer-readable storage medium includes a processor and a memory, the memory being configured to store a computer program, the computer program including program instructions, and the processor being configured to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal and is suitable for implementing one or more instructions, specifically loading and executing one or more instructions in the computer storage medium to implement corresponding method processes or corresponding functions. The processor described in this embodiment of the present invention can be used in an LLM-based human-computer interaction framework for motion control operations of an AUV.

[0098] Large Language Model (LLM): In the fields of artificial intelligence and natural language processing, LLMs typically refer to large neural network models capable of understanding and generating human language. These models are trained on large amounts of text data and are capable of performing a variety of tasks, such as text generation, question answering, and translation. For example, OpenAI's GPT series of models (such as GPT-3 and GPT-4) and Google's BERT model are both well-known LLMs.

[0099] AUV stands for "Autonomous Underwater Vehicle." It's an unmanned submersible capable of autonomously navigating and performing tasks underwater without human intervention. AUVs are capable of performing a variety of tasks, including deep-sea exploration, topographic mapping, underwater archaeology, and environmental monitoring. They are crucial tools in modern marine scientific research and technological applications.

[0100] ROS2, the full name of Robot Operating System 2, is an open source operating system that provides robots with hardware abstraction, device drivers, function libraries, visualization tools, message communication, and package management.

[0101] NMPC, short for Nonlinear Model Predictive Control, is an advanced control strategy particularly well-suited for nonlinear systems with complex dynamics and constraints. NMPC effectively controls the system by predicting its future behavior and optimizing current control inputs based on these predictions.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A method for AUV motion control based on LLM human-computer interaction framework, characterized in that: include: Acquire the target AUV and build a target AUV simulation platform based on ROS2; Integrate LLM and NMPC in ROS2 to form the LLM human-computer interaction framework; Obtain user instructions, the LLM human-computer interaction framework converts user instructions into control instructions, and transmits the control instructions to the target AUV simulation platform; The AUV in the target AUV simulation platform executes control instructions, generates trajectory tracking and displays it.

2. The AUV motion control method based on the LLM human-computer interaction framework according to claim 1 is characterized in that: The target AUV simulation platform is constructed based on ROS2, specifically: setting the URDF robot description file in ROS2 and constructing a simulation scene to obtain the target AUV simulation platform.

3. The AUV motion control method based on the LLM human-computer interaction framework according to claim 1 is characterized in that: The LLM human-computer interaction framework includes: The LLM high-level decision module is used to understand and analyze user instructions, generate high-level decisions, and send the high-level decisions to the NMPC low-level control module; The NMPC bottom-level control module is used to receive high-level decisions, solve high-level decisions to obtain control instructions, and transmit the control instructions to the target AUV simulation platform.

4. The AUV motion control method based on the LLM human-computer interaction framework according to claim 3 is characterized in that: The high-level decisions include: generating a trajectory, modifying a desired position, modifying a desired velocity, and modifying an NMPC weight matrix.

5. The AUV motion control method based on the LLM human-computer interaction framework according to claim 3 is characterized in that: The integration of LLM and NMPC in ROS2 to form the LLM human-computer interaction framework specifically includes: In the startup file of the LLM high-level decision module, add ROS2 publisher and subscriber nodes and define data types related to user instructions; In the startup file of the NMPC underlying control module, add the ROS2 publisher and subscriber nodes; In the Rviz2 interface of ROS2, the AUV model, coordinate system, and sensor are added to the toolbar to obtain the LLM human-computer interaction framework.

6. The AUV motion control method based on the LLM human-computer interaction framework according to claim 5 is characterized in that: The data types related to the user instructions include: position, speed and posture information.

7. The AUV motion control method based on the LLM human-computer interaction framework according to claim 1 is characterized in that: The trajectory tracking is displayed in the LLM human-computer interaction framework.

8. An AUV motion control system based on the LLM human-computer interaction framework, characterized in that: include: Construction unit, used to acquire the target AUV and build the target AUV simulation platform based on ROS2; Integrate LLM into ROS2 to form the LLM human-computer interaction framework; The instruction generation unit is used to obtain user instructions. The LLM human-computer interaction framework converts the user instructions into control instructions and transmits the control instructions to the target AUV simulation platform; The instruction execution unit executes the control instructions of the AUV in the target AUV simulation platform, generates trajectory tracking and displays it.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the AUV motion control method based on the LLM human-computer interaction framework according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the AUV motion control method based on the LLM human-computer interaction framework according to any one of claims 1 to 7.

Images