Design method and control system and method for large command and control model of unmanned aerial vehicle

Abstract

The invention relates to a design method and a control system and method of an unmanned aerial vehicle command and control large model, and belongs to the technical field of command and control. In order to solve the problem that an existing command and control system depends on a traditional rule-based or manual programming method and is limited when processing complex tasks, the method for controlling the unmanned aerial vehicle through the unmanned aerial vehicle command and control system comprises the following steps that task texts are input into an unmanned aerial vehicle command and control large model, and a MavLink format instruction is generated; according to the method, unsafe instructions are corrected, executable unmanned aerial vehicle tasks are planned into a behavior tree, flight and task execution of the unmanned aerial vehicle are controlled, the powerful language understanding and generation capability of LLama3 is utilized, the intelligent level and performance of an unmanned aerial vehicle command and control system are improved, and the task execution efficiency and success rate of the unmanned aerial vehicle are improved.

Description

Technical Field

[0001] The present invention relates to a design method and a control system of a large model of unmanned aerial vehicle command and control, and in particular to a unmanned aerial vehicle command and control system with safety constraints based on a local large model and a control method thereof, and belongs to the technical field of command and control. Background Art

[0002] Drones are increasingly used in military and civilian fields, and drone command and control systems are a key component of drone applications. With the advancement of drone technology, the complexity of command and control systems has also increased, placing higher demands on the intelligence level of the system. Artificial intelligence (AI) technology, especially large language models, has brought revolutionary changes to drone command and control systems. Among them, large pre-trained language models such as LLama3 have demonstrated impressive language understanding and generation capabilities, and have attracted much attention in the AI ​​field.

[0003] LLama3 is an open source large language model (LLM) that can be deployed locally offline. It can handle complex natural language tasks such as text classification, question answering, and generation, and has great application potential.

[0004] Consider that although large language models perform well in idea generation, they often lack safety guarantees for the dynamic constraints of drones.

[0005] At present, UAV command and control systems generally face challenges such as increasing complexity, higher requirements for real-time data processing and analysis, and the need for natural language interaction with operators. Traditional methods often rely on manual programming and rule setting, which are difficult to adapt to changing environments and mission requirements. The instructions directly generated by LLM lack security guarantees and are difficult to apply directly.

[0006] Although the LLama3 model demonstrates powerful language processing capabilities, it has not yet been applied to UAV command and control systems.

[0007] Among existing technical solutions, most command and control systems still rely on traditional rule-based or manually programmed methods, which face limitations in handling complex tasks. Summary of the invention

[0008] The purpose of the present invention is to solve the problems in the prior art and provide a design method and control system of a large model of unmanned aerial vehicle command and control. The method applies the LLama3 model to the unmanned aerial vehicle command and control system, and utilizes the powerful language understanding and generation capabilities of LLama3 to improve the intelligence level and performance of the unmanned aerial vehicle command and control system, and improve the efficiency and success rate of the unmanned aerial vehicle's mission execution.

[0009] In order to solve the above problems, the present application is implemented through the following technical solutions: A design method for a large model of UAV command and control, which is special in that it includes the following steps: Step 1: Fine-tune the Llama3 model according to the specific needs of the UAV command and control system; Step 2: Set appropriate training parameters and fine-tune the LLama3 model in step 1. Step 3: Evaluate the performance of the LLama3 model after fine-tuning in step 2 to ensure that the model’s effect in the UAV command and control mission meets expectations; Step 4: Integrate the qualified LLama3 model evaluated in step 3 into the UAV command and control system; Furthermore, the step 1 specifically includes the following steps: Step 1.1: Collect and prepare professional data in the field of drones and pre-process the collected data; Step 1.2: Use the professional large language model fine-tuning project to load and fine-tune the LLama3 model; Furthermore, the professional data in step 1.1 includes UAV technical documents, flight mission text descriptions, and operator's text records of operation tasks, etc.; Wherein, the drone technical document is a drone control instruction manual; The flight mission text description is the flight plan, flight log, etc. issued for the UAV flight mission; The written record of the operator's operating tasks is a flight report, log or other record written by the drone operator.

[0010] Furthermore, the collected data in step 1.1 is preprocessed, including: Split text data into subwords or words and standardize the data format to ensure consistency.

[0011] Furthermore, the large language model fine-tuning project in step 1.2 adopts Meta's LLama-recipes project to load the pre-trained LLama3 model; The LLama-recipes project provides a set of tools and scripts for fine-tuning LLama3 models on specific tasks and datasets; Furthermore, in step 1.2, the drone data preprocessed in step 1.1 is loaded into the fine-tuning project, and the LLama3 model is configured to be trained using the drone data.

[0012] Furthermore, the training parameters in step 2 include learning rate, batch size, and number of training rounds; Wherein, the learning rate is set to 5e-5: The batch size is set to 32: The batch size is the number of training examples used in each iteration; The number of training rounds is set to 10: the number of times the model sees the training data, and a smaller number of training rounds is chosen to avoid overfitting.

[0013] Furthermore, the fine-tuning training in step 2 is specifically as follows: obtaining the source code and pre-trained weights of the LLama3 model from an open source platform or community, and adjusting the weights and biases of the LLama3 model based on the pre-training to better adapt to the drone data.

[0014] Furthermore, the step 3 adopts manual evaluation to evaluate the performance of the LLama3 model from the accuracy of the task described by the operator to the generation of the corresponding MavLink format instruction sequence.

[0015] Furthermore, the UAV command and control system of step 4 includes hardware resources and software environment, and the software environment is Ubuntu operating system, LLama-3 model data, and Ollama large model operating framework.

[0016] The UAV command and control system composed of the UAV command and control large model designed by the above method has the following special features: it includes a task input module: used for inputting UAV control tasks; UAV command and control large model: receives tasks input by the task input module and is used to generate MavLink format instructions; Safety filtering and control execution module: Receives MavLink format instructions generated by the UAV command and control large model, including safety filtering rules, which are used to filter and alarm dangerous action instructions, instructions beyond the maneuvering range, or instructions based on other set rules, generate executable UAV mission planning behavior trees, and control the flight of the UAV and the execution of tasks.

[0017] The method for controlling a UAV using the UAV command and control system described above is special in that it includes the following steps: Step S1, input the task text into the UAV command and control large model, and generate MavLink format instructions; Step S2, correct the unsafe instructions generated in step S1, plan the executable UAV mission behavior tree, and control the flight of the UAV and the execution of the mission.

[0018] Furthermore, the step S1 specifically includes the following steps: Step S1.1, natural language instruction conversion and prompt word design, namely, defining prompt words and phrases: Define a set of key cues and phrases related to drone operations, such as “takeoff”, “hover”, “scan area”, “return to base”, etc. These cues are designed to activate the LLama-3 model and make it focus on outputs related to drone commands; The prompt words are designed with various scenarios and tasks of drone operation in mind to ensure that common drone operation instructions are covered. These prompt words can be customized and expanded according to specific needs. Step S1.2 Natural semantic instruction conversion: The LLama-3 model is triggered using prompt words. When the operator inputs a natural language instruction, the system uses the prompt words to identify and classify the intent of the instruction. For example, when the operator inputs "Ask the drone to check the target area and return", the system will recognize the two key phrases "check the target area" and "return".

[0019] Step S1.3, the LLama-3 model generates the corresponding MavLink format instructions and the action sequence that the drone needs to perform.

[0020] Furthermore, the step S2 adopts a computer algorithm of a safety filter, specifically comprising the following steps: Step S2.1, check the MavLink format command outputted from step S1 and the corresponding flight control operation to ensure that the distance between the waypoints of the drone and the obstacles is less than a certain threshold. If such a situation exists, eliminate the potential collision by a preset offset; Step S2.2: Use the alternating minimization method to process the waypoints generated by the language model LLM, as follows: Interpolate between waypoints, satisfy physical constraints and minimize collisions, redefine constraints in polar coordinate form and apply the alternating minimization algorithm to solve the optimization problem while maintaining the quadratic programming QP structure. The specific steps are as follows: Step S2.2.1: Determine a single drone The initial and target positions of: For a group size of A single drone , the initial and target positions are represented by the three-dimensional position vector p = [ x , y , z ] T Description, and given by the language model LLM; Step S2.2.2: Determine the constraints for each UAV and optimize its position: Initialization constraint: Each drone is initialized at the initial moment The position, velocity and acceleration of are set to their actual initial values, and the trajectory position and high-order derivatives of each drone are initialized with the following constraints: p i ( q ) [ 0 ] = p i , a ( q ) , ∀ q = 0 , 1 , 2 , Indicates drones at a time time Trajectory of the derivatives; Indicates The actual initial value of the drone, express Derivatives; when When indicates the position, Indicates speed, represents acceleration; Feasibility constraints: enforce position, velocity, and acceleration limits as follows: , the drone position must be between the given lower and upper limits, i.e. p ¯ ≤ p i [ k ] ≤ p ¯ , ∀ k ; Where: p i [ k ] Indicates UAVs in time step The position at represents the lower limit constraint of the position, represents the upper limit constraint of the position, For all time steps , the speed of the drone always remains within the given upper limit constraint, i.e. ‖ p i ( 1 ) [ k ] ‖ 2 ≤ v ¯ 2 , ∀ k , Where: p i ( 1 ) [ k ] Indicates UAVs in time step The speed of represents the Euclidean norm, i.e., the Euclidean distance, represents the upper limit constraint of the speed, For all time steps , the acceleration of the drone (taking into account the influence of gravity) is always kept within the given upper limit constraint, that is, ‖ p i ( 2 ) [ k ] ‖ 2 + ‖ g ‖ 2 ≤ f ¯ 2 , ∀ k , Where: p i ( 2 ) [ k ] Indicates UAVs in time step The acceleration at represents the Euclidean norm, i.e., the Euclidean distance, represents the gravity vector, represents the upper limit constraint of acceleration, Obstacle avoidance constraint: avoid The collision of adjacent drones or obstacles is constrained as follows: , Where: diagonal matrix and the gravitational acceleration vector Used to describe the surrounding environment and the effects of gravity; Barrier Function: Adding barrier function (BF) constraints to enhance collision avoidance introduces different levels of safety behavior, as follows: [ h i j [ k ] − h i j [ k − 1 ] ≥ − c h i j [ k − 1 ] , ∀ k , j ] , Where: h i j [ k ] Indicates The drone and The distance or obstacle function value between drones or obstacles in the time step The value of is a control parameter ranging from 0 to 1, which controls the movement speed of each drone on the constraint boundary. Obstacle Avoidance Constraints [ h i j [ k ] − h i j [ k − 1 ] ≥ − c h i j [ k − 1 ] , ∀ k , j ] To ensure that the distance between adjacent drones does not decrease suddenly and avoid collisions, it stipulates that the rate of change of the distance function value is limited; Step S2.2.3: Establish a cost function based on the constraints of step S2.2.2: A. Establish target cost: oh g ∑ k = K − k K − 1 ‖ p i [ k ] − p i , g ‖ 2 , Where: p i [ k ] Indicates UAVs in time step The actual position at Represents the specified target position generated by LLama-3. This item penalizes the deviation between the drone position and the target position. is the weight; B. Establish smoothness cost: oh s ∑ k = 0 K − 1 ‖ p i ( q ) [ k ] ‖ 2 , Where: p i ( q ) [ k ] Indicates UAVs in time step of This term penalizes high-order derivatives of the position trajectory and encourages smooth motion. is the weight; C. Establish cost function: m i n p i oh g ∑ k = K − k K − 1 ‖ p i [ k ] − p i , g ‖ 2 + oh s ∑ k = 0 K − 1 ‖ p i ( q ) [ k ] ‖ 2 , Step S2.2.4, solve the cost function in step S2.2.3: Non-Linear Programming (NLP): Take the cost function formula as the objective function and use the Sequential Quadratic Programming algorithm to find a set of input variables to minimize the objective function.

[0021] Through the application of the fine-tuned LLama3 model, the UAV command and control system has achieved significant advantages in many aspects, as follows: 1. Improve decision-making speed: After fine-tuning, the LLama3 model has strong language understanding and generation capabilities, and can quickly understand and process the real-time data of the UAV command and control system, so that the UAV command and control system can quickly generate decision suggestions and improve the decision-making speed, so that the UAV can respond quickly and perform tasks; 2. Improve the efficiency of task execution: After fine-tuning, the LLama3 model can generate efficient task planning based on the generated task description and instructions, which helps the UAV command and control system to automatically generate efficient task planning, improve the efficiency of UAV task execution, and reduce the time and resource consumption of task execution; 3. Improve the user experience of operators: After fine-tuning, the LLama3 model can understand and generate natural language, allowing the UAV command and control system to communicate with operators more intuitively, improving the user experience of operators, allowing operators to interact with the UAV command and control system more easily, and improving operational efficiency and satisfaction.

[0022] Through this application, the UAV command and control system can utilize the powerful language understanding and generation capabilities of the open source large model LLama3 to achieve more intelligent and automated mission planning, decision support and natural language interaction, thereby improving the efficiency and success rate of UAV mission execution, and providing an efficient and intelligent solution for the field of UAV command and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of Embodiment 2 of the present invention; Figure 3 This is a flow chart of Example 3 of the present invention. DETAILED DESCRIPTION

[0024] The following is a specific implementation of the present invention with reference to the accompanying drawings to further illustrate the composition of the present invention.

[0025] Example 1. A design method for a large model of UAV command and control, the specific process is as follows Figure 1 As shown, the following steps are included: Step 1: Fine-tune the Llama3 model according to the specific needs of the UAV command and control system; Step 2: Set appropriate training parameters and fine-tune the LLama3 model in step 1. Step 3: Evaluate the performance of the LLama3 model after fine-tuning in step 2 to ensure that the model’s effect in the UAV command and control mission meets expectations; Step 4: Integrate the qualified LLama3 model evaluated in step 3 into the UAV command and control system; Furthermore, the step 1 specifically includes the following steps: Step 1.1: Collect and prepare professional data in the field of drones and pre-process the collected data; Step 1.2: Use the professional large language model fine-tuning project to load and fine-tune the LLama3 model; Furthermore, the professional data in step 1.1 includes UAV technical documents, flight mission text descriptions, and operator's text records of operation tasks, etc.; Wherein, the drone technical document is a drone control instruction manual; The flight mission text description is the flight plan, flight log, etc. issued for the UAV flight mission; The written record of the operator's operating tasks is a flight report, log or other record written by the drone operator.

[0026] Furthermore, the collected data in step 1.1 is preprocessed, including: segmenting the text data into subwords or words, and standardizing the data format to ensure consistency.

[0027] Furthermore, the large language model fine-tuning project in step 1.2 adopts Meta's LLama-recipes project to load the pre-trained LLama3 model; The LLama-recipes project provides a set of tools and scripts for fine-tuning LLama3 models on specific tasks and datasets; Furthermore, in step 1.2, the drone data preprocessed in step 1.1 is loaded into the fine-tuning project, and the LLama3 model is configured to be trained using the drone data.

[0028] Furthermore, the training parameters in step 2 include learning rate, batch size, and number of training rounds; Wherein, the learning rate is set to 5e-5: The batch size is set to 32: The batch size is the number of training examples used in each iteration; The number of training rounds is set to 10: the number of times the model sees the training data, and a smaller number of training rounds is chosen to avoid overfitting.

[0029] Furthermore, the fine-tuning training in step 2 is specifically as follows: obtaining the source code and pre-trained weights of the LLama3 model from an open source platform or community, and adjusting the weights and biases of the LLama3 model based on the pre-training to better adapt to the drone data.

[0030] Furthermore, the step 3 adopts manual evaluation to evaluate the performance of the LLama3 model from the accuracy of the task described by the operator to the generation of the corresponding MavLink format instruction sequence.

[0031] Furthermore, the UAV command and control system of step 4 includes hardware resources and software environment, and the software environment is Ubuntu operating system, LLama-3 model data, and Ollama large model operating framework.

[0032] Example 2. A UAV command and control system composed of the UAV command and control large model designed in Example 1, such as Figure 2 As shown, it includes a task input module: used to input the UAV control task; UAV command and control large model: receives tasks input by the task input module and is used to generate MavLink format instructions; Safety filtering and control execution module: Receives MavLink format instructions generated by the UAV command and control large model, including safety filtering rules, which are used to filter and alarm dangerous action instructions, instructions beyond the maneuvering range, or instructions based on other set rules, generate executable UAV mission planning behavior trees, and control the flight of the UAV and the execution of tasks.

[0033] Example 3. A method for controlling a drone using the drone command and control system described in Example 2, the process of which is as follows: Figure 3 As shown, the following steps are included: Step S1, input the task text into the UAV command and control large model, and generate MavLink format instructions; Step S2, correct the unsafe instructions generated in step S1, plan the executable UAV mission behavior tree, and control the flight of the UAV and the execution of the mission.

[0034] Furthermore, the step S1 specifically includes the following steps: Step S1.1, natural language command conversion and prompt word design, i.e., defining prompt words and phrases: defining a set of key prompt words and phrases related to drone operations, such as “take-off”, “hover”, “scan area”, “return to base”, etc. These prompt words are designed to activate the LLama-3 model and make it focus on the output related to drone commands; The prompt words are designed with various scenarios and tasks of drone operation in mind to ensure that common drone operation instructions are covered. These prompt words can be customized and expanded according to specific needs. Step S1.2 Natural semantic instruction conversion: Use prompt words to trigger the LLama-3 model. When the operator inputs a natural language instruction, the system uses the prompt words to identify and classify the intention of the instruction; For example, when the operator inputs "Ask the drone to check the target area and return", the system will recognize the two key phrases "check the target area" and "return".

[0035] Step S1.3, the LLama-3 model generates the corresponding MavLink format instructions and the action sequence that the drone needs to perform.

[0036] Furthermore, the step S2 adopts a computer algorithm of a safety filter, specifically comprising the following steps: Step S2.1, check the MavLink format command outputted from step S1 and the corresponding flight control operation to ensure that the distance between the waypoints of the drone and the obstacles is less than a certain threshold. If such a situation exists, eliminate the potential collision by a preset offset; Step S2.2, the waypoints generated by the language model LLM are processed using the alternating minimization method, as follows: interpolation is performed between the waypoints to satisfy the physical constraints and minimize collisions, the constraints are redefined in polar coordinate form and the alternating minimization algorithm is applied, and the optimization problem is solved while maintaining the quadratic programming (QP) structure. The specific steps are as follows: Step S2.2.1: Determine a single drone The initial and target positions of: For a group size of A single drone , the initial and target positions are represented by the three-dimensional position vector p = [ x , y , z ] T Description, and given by the language model LLM; Step S2.2.2: Determine the constraints for each UAV and optimize its position: Initialization constraint: Each drone is initialized at the initial moment The position, velocity and acceleration of are set to their actual initial values, and the trajectory position and high-order derivatives of each drone are initialized with the following constraints: p i ( q ) [ 0 ] = p i , a ( q ) , ∀ q = 0 , 1 , 2 , Indicates drones at a time time Trajectory of the derivatives; Indicates The actual initial value of the drone, express Derivative; when When indicates the position, Indicates speed, represents acceleration; Feasibility constraints: enforce position, velocity, and acceleration limits as follows: , the drone position must be between the given lower and upper limits, i.e. p ¯ ≤ p i [ k ] ≤ p ¯ , ∀ k ; Where: p i [ k ] Indicates UAVs in time step The position at represents the lower limit constraint of the position, represents the upper limit constraint of the position, For all time steps , the speed of the human-machine always remains within the given upper limit constraint, that is ‖ p i ( 1 ) [ k ] ‖ 2 ≤ v ¯ 2 , ∀ k , Where: p i ( 1 ) [ k ] Indicates UAVs in time step The speed of represents the Euclidean norm, i.e., the Euclidean distance, represents the upper limit constraint of the speed, For all time steps , the acceleration of the drone (taking into account the influence of gravity) is always kept within the given upper limit constraint, that is, ‖ p i ( 2 ) [ k ] ‖ 2 + ‖ g ‖ 2 ≤ f ¯ 2 , ∀ k , Where: p i ( 2 ) [ k ] Indicates UAVs in time step The acceleration at represents the Euclidean norm, i.e., the Euclidean distance, represents the gravity vector, represents the upper limit constraint of acceleration, Obstacle avoidance constraint: avoid The collision of adjacent drones or obstacles is constrained as follows: , Where: diagonal matrix and the gravitational acceleration vector Used to describe the surrounding environment and the effects of gravity; Barrier Function: Adding barrier function (BF) constraints to enhance collision avoidance introduces different levels of safety behavior, as follows: [ h i j [ k ] − h i j [ k − 1 ] ≥ − c h i j [ k − 1 ] , ∀ k , j ] , Where: h i j [ k ] Indicates The drone and The distance or obstacle function value between UAVs or obstacles in the time step The value of is a control parameter ranging from 0 to 1, which controls the movement speed of each drone on the constraint boundary. Obstacle Avoidance Constraints [ h i j [ k ] − h i j [ k − 1 ] ≥ − c h i j [ k − 1 ] , ∀ k , j ] To ensure that the distance between adjacent drones does not decrease suddenly and avoid collisions, it stipulates that the rate of change of the distance function value is limited; Step S2.2.3: Establish a cost function based on the constraints of step S2.2.2: A. Establish target cost: oh g ∑ k = K − k K − 1 ‖ p i [ k ] − p i , g ‖ 2 , Where: p i [ k ] Indicates UAVs in time step The actual position at Represents the specified target position generated by LLama-3. This item penalizes the deviation between the drone position and the target position. is the weight; B. Establish smoothness cost: oh s ∑ k = 0 K − 1 ‖ p i ( q ) [ k ] ‖ 2 , Where: p i ( q ) [ k ] Indicates UAVs in time step of This term penalizes high-order derivatives of the position trajectory and encourages smooth motion. is the weight; C. Establish cost function: m i n p i oh g ∑ k = K − k K − 1 ‖ p i [ k ] − p i , g ‖ 2 + oh s ∑ k = 0 K − 1 ‖ p i ( q ) [ k ] ‖ 2 ; Step S2.2.4, solve the cost function in step S2.2.3: Non-Linear Programming (NLP): Take the cost function formula as the objective function and use the Sequential Quadratic Programming algorithm to find a set of input variables to minimize the objective function.

[0037] By iteratively solving the obstacle function constraints and optimizing the trajectory parameters, this method ensures that various constraints such as obstacle avoidance, position constraints and motion feasibility are met.

[0038] The algorithm switches between different aspects of the optimization process, ensuring that the imposed constraints are followed at each step. The resulting trajectory strikes a balance between safety and efficiency.

[0039] The algorithm can show consistent and reliable performance in complex and dynamic environments and has good scalability.

[0040] Security filters enable robust operation of the system in densely interactive environments.

[0041] The algorithm also interfaces well with the waypoints output by LLM.

[0042] Compared with other methods, it does not require additional motion information, which makes LLMs easier to generate the desired output.

[0043] Embodiment 4: This embodiment is a specific application example of Embodiment 3.

[0044] Tips: Mission description: Fly to point A. If you encounter an enemy drone, avoid it and continue to conduct reconnaissance for 20 minutes before returning. Requirements: Decompose into more detailed executable steps, including flight parameters such as flight speed and turning radius, and generate a behavior tree.

[0045] The executable drone behavior tree instructions generated by the large model are as follows: - Mission Start: -take off: - Check the drone status, - Send takeoff command, -Go to point A: - Navigate to point A (flight speed: 25m / s), - Arrive at point A, - Detect and detect enemy drones: -Perform reconnaissance missions (flight speed: 15m / s), -Detect enemy drones, - If you encounter an enemy drone, avoid it: - Plan a detour route (turning radius: 50m), -Send bypass instructions (custom MavLink instructions), -Continuous reconnaissance: - Set a detection timer, -Perform reconnaissance missions (flight speed: 15m / s), - Check the detection time, -Return flight: - Navigate to the take-off point (flight speed: 25m / s), - Arrive at the take-off point, -landing: - Navigate to the landing point, - Send landing command (MAV_CMD_NAV_LAND), - Mission accomplished.

[0046] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the defined scope.

Claims

1. A design method for a large model of UAV command and control, characterized by: The following steps are involved: Step 1: Fine-tune the Llama3 model according to the specific needs of the UAV command and control system; Step 2: Set appropriate training parameters and fine-tune the LLama3 model in step 1. Step 3: Evaluate the performance of the LLama3 model after fine-tuning in step 2 to ensure that the model’s effect in the UAV command and control mission meets expectations; Step 4: Integrate the qualified LLama3 model evaluated in step 3 into the UAV command and control system.

2. The design method of a large model of UAV command and control according to claim 1 is characterized by: The step 1 specifically comprises the following steps: Step 1.1: Collect and prepare professional data in the field of drones and pre-process the collected data; Step 1.2: Use the professional large language model fine-tuning project to load and fine-tune the LLama3 model.

3. The design method of a large model of UAV command and control according to claim 2 is characterized by: The professional data in step 1.1 includes UAV technical documents, flight mission text descriptions, and operator's text records of the mission; Wherein, the drone technical document is a drone control instruction manual; The flight mission text description is the flight plan and flight log issued for the UAV flight mission; The written record of the operator's operation task is the flight report and log written by the UAV operator; In the step 1.1, the collected data is preprocessed, including: dividing the text data into subwords or words, and standardizing the data format to ensure consistency.

4. The design method of a large model of UAV command and control according to claim 2 or 3, characterized in that: The large language model fine-tuning project in step 1.2 uses Meta's LLama-recipes project to load the pre-trained LLama3 model; The LLama-recipes project provides a set of tools and scripts for fine-tuning LLama3 models on specific tasks and datasets; In step 1.2, the drone data preprocessed in step 1.1 is loaded into the fine-tuning project, and the LLama3 model is configured to be trained using the drone data.

5. The design method of a large model of UAV command and control according to claim 4 is characterized by: The training parameters in step 2 include learning rate, batch size, and number of training rounds; The learning rate is set to 5e-5, the batch size is set to 32, and the number of training rounds is set to 10; The fine-tuning training in step 2 is specifically as follows: obtaining the source code and pre-trained weights of the LLama3 model from an open source platform or community, and adjusting the weights and biases of the LLama3 model based on the pre-training.

6. The design method of a large model of UAV command and control according to claim 4 is characterized by: The step 3 uses manual evaluation to evaluate the performance of the LLama3 model from the accuracy of the task described by the operator to the generation of the corresponding MavLink format instruction sequence; The UAV command and control system of step 4 includes hardware resources and software environment, and the software environment is Ubuntu operating system, LLama-3 model data, and Ollama large model operating framework.

7. An unmanned aerial vehicle command and control system, characterized in that: Including task input module: used to input UAV control tasks; UAV command and control large model: receives tasks input by the task input module and is used to generate MavLink format instructions; Safety filtering and control execution module: Receives MavLink format instructions generated by the UAV command and control large model, including safety filtering rules, which are used to filter and alarm dangerous action instructions, instructions beyond the maneuvering range, or instructions based on other set rules, generate executable UAV mission planning behavior trees, and control the flight of the UAV and the execution of tasks.

8. A method for controlling a drone using the drone command and control system of claim 7, characterized in that: The following steps are involved: Step S1, input the task text into the UAV command and control large model, and generate MavLink format instructions; Step S2, correct the unsafe instructions generated in step S1, plan the executable UAV mission behavior tree, and control the flight of the UAV and the execution of the mission.

9. A UAV command and control method according to claim 8, characterized in that: The step S1 specifically includes the following steps: Step S1.1, natural language command conversion and prompt word design, namely, defining prompt words and phrases: defining a set of key prompt words and phrases related to drone operations, activating the LLama-3 model to focus on outputs related to drone commands; Step S1.2, natural semantic instruction conversion: Use prompt words to trigger the LLama-3 model. When the operator inputs a natural language instruction, the system uses the prompt words to identify and classify the intention of the instruction; Step S1.3, the LLama-3 model generates the corresponding MavLink format instructions and the action sequence that the drone needs to perform.

10. A UAV command and control method according to claim 8 or 9, characterized in that: The step S2 adopts a computer algorithm of a safety filter, and specifically comprises the following steps: Step S2.1, check the MavLink format command outputted from step S1 and the corresponding flight control operation to ensure that the distance between the waypoints of the drone and the obstacles is less than a certain threshold. If such a situation exists, eliminate the potential collision by a preset offset; Step S2.2, use the alternating minimization method to process the waypoints generated by the language model LLM, as follows: interpolate between waypoints, satisfy physical constraints and minimize collisions, redefine constraints in polar coordinate form and apply the alternating minimization algorithm, maintain the quadratic programming QP structure, the specific steps are as follows: Step S2.2.1: Determine a single drone The initial and target positions of: For a group size of A single drone , the initial and target positions are represented by the three-dimensional position vector Description, and given by the language model LLM; Step S2.2.2: Determine the constraints for each UAV and optimize its position: Initialize the constraints: Set each UAV at the initial time The position, velocity and acceleration of are set to their actual initial values, and the trajectory position and high-order derivatives of each drone are initialized with the following constraints: , Indicates drones at a time time Trajectory of the derivatives; Indicates The actual initial value of the drone, express Derivative; when When indicates the position, Indicates speed, represents acceleration; Feasibility constraints: enforce position, velocity, and acceleration limits as follows: , the drone position must be between the given lower and upper limits, i.e. ; Where: Indicates UAVs in the time step The position at represents the lower limit constraint of the position, represents the upper limit constraint of the position, For all time steps , the speed of the drone always remains within the given upper limit constraint, i.e. , Where: Indicates UAVs in the time step The speed of represents the Euclidean norm, i.e., the Euclidean distance, represents the upper limit constraint of the speed, For all time steps , the acceleration of the drone always remains within the given upper limit constraint, i.e. , Where: Indicates UAVs in the time step The acceleration at represents the Euclidean norm, i.e., the Euclidean distance, represents the gravity vector, represents the upper limit constraint of acceleration, Obstacle avoidance constraint: avoid The collision of adjacent drones or obstacles is constrained as follows: , Where: diagonal matrix and the gravitational acceleration vector Describe the surrounding environment and the effects of gravity; Barrier Function: Add barrier function (BF) constraints to enhance collision avoidance and introduce different levels of safety behaviors, as follows: , Where: Indicates The drone and The distance or obstacle function value between UAVs or obstacles in the time step The value of is a control parameter ranging from 0 to 1, which controls the movement speed of each drone on the constraint boundary. Obstacle Avoidance Constraints To ensure that the distance between adjacent drones does not decrease suddenly and avoid collisions, the rate of change of the distance function value is limited; Step S2.2.3: Establish a cost function based on the constraints of step S2.2.2: A. Establish target cost: , Where: Indicates UAVs in the time step The actual position at Indicates the specified target location generated by LLama-3, is the weight; B. Establish smoothness cost: , Where: Indicates UAVs in the time step Speed ​​or acceleration Derivatives, is the weight; C. Establish cost function: ; Step S2.2.4, solve the cost function in step S2.2.3: Nonlinear Programming: Take the cost function formula as the objective function and use the Sequential QuadraticProgramming algorithm to find a set of input variables to minimize the objective function.

Images