Unmanned aerial vehicle intelligent adaptive scheduling method and system based on large model

Abstract

The invention provides an unmanned aerial vehicle intelligent adaptive scheduling method and system based on a large model, and the method comprises the steps: carrying out the type selection of a large model according to the specific task demands and computing resources of an unmanned aerial vehicle system, and carrying out the fine adjustment of a specific task field of the selected large model; integrating a plurality of professional models for executing different task requirements in the unmanned aerial vehicle system; for different task scenes, large model analysis judges whether a specific professional model needs to be activated or not, and the professional model is selected for task processing when needed; the large model confirms a professional model needing to be called, and automatically extracts and transmits required parameters to the professional model based on the collected environment information and task instructions; and the professional model receives the parameter information transmitted by the large model, executes the corresponding subtask, and feeds back an execution result to the large model, so that the large model performs continuous learning optimization. According to the invention, the unmanned aerial vehicle can complete the task more quickly, and the accuracy and reliability of task completion are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of large models, and specifically relates to an intelligent adaptive scheduling method and system for unmanned aerial vehicles based on large models. Background Art

[0002] In recent years, with the rapid development of global science and technology, drone technology has experienced a significant leap. This technology has been widely used in agriculture, forestry, animal husbandry, environmental monitoring, geographic surveying and mapping, disaster response, scientific research and other fields, which not only greatly improves work efficiency, but also expands the scope of application, injecting new vitality into industrial innovation. In the agricultural field, drones are used for crop monitoring, fertilization, spraying pesticides, etc., which improves agricultural production efficiency and crop quality; in environmental monitoring, drones are equipped with sensors for air quality detection and water quality monitoring, etc., which provides strong support for environmental protection and climate change research; in geographic surveying and mapping, drones provide high-precision spatial data, which promotes the sustainable development of urban planning and land use. Looking to the future, drone technology will be more deeply applied in a wider range of fields. With the continuous advancement of intelligent technology, drone systems will be further intelligently empowered and have higher autonomy. The development trend of drone systems is characterized by autonomous and accurate environmental perception, autonomous and efficient mission design, autonomous and scientific mission planning, autonomous and flexible mission execution, and autonomous and fast mission evaluation, which will greatly improve the work efficiency and execution capabilities of drones. It can be foreseen that in the future, drones will play a more important role in various fields and bring more convenience and innovation to the development of human society.

[0003] At present, UAV systems have made significant technological progress and integrated many advanced intelligent algorithms, such as route planning, target recognition, flight control, etc. These algorithms have greatly improved the operational efficiency and accuracy of UAVs, enabling them to perform tasks flexibly and freely in complex environments. However, despite the improvement in the level of intelligence, the task execution process of UAV systems still cannot be separated from human intervention. Operators need to schedule algorithms according to real-time scenarios to ensure that UAVs can properly respond to various emergencies. This means that UAVs have not yet achieved the goal of full intelligence and autonomy, and still need to rely on human judgment and decision-making.

[0004] At the current stage, large model technology has attracted widespread attention from all parties. This technology has already possessed strong language understanding and text generation capabilities, and its potential and application prospects are highly anticipated. Although large general models have demonstrated strong capabilities in language understanding and generation, they often perform worse than dedicated models when dealing with specific tasks. For example, in the absence of professional tool support, large models are difficult to directly solve complex problems such as path planning, and can only provide reference suggestions. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for intelligent adaptive scheduling of unmanned aerial vehicles based on large models, so as to realize fully autonomous task execution of the unmanned aerial vehicle system, improve the scheduling and management capabilities of small models, and enhance the level of intelligence.

[0006] The technical solution to achieve the purpose of the present invention is: a method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model, comprising the steps of:

[0007] S1), select a large model based on the specific mission requirements and computing resources of the UAV system, and fine-tune the selected large model in specific mission areas;

[0008] S2), integrating multiple professional models to perform different mission requirements in the UAV system;

[0009] S3), in the face of different task scenarios, the large model analyzes and determines whether a specific professional model needs to be activated, and selects a professional model to process the task when necessary;

[0010] S4), the big model confirms the professional model to be called, and automatically extracts and transfers the required parameters to the professional model based on the collected environmental information and task instructions;

[0011] S5), the professional model receives the parameter information transmitted by the large model, executes the corresponding subtasks, and feeds back the execution results to the large model, so that the large model can perform continuous learning and optimization.

[0012] Furthermore, the selection of large models based on the specific mission requirements and computing resources of the drone system includes: selecting large models with 6 billion and 13 billion parameters for 1 A800 computing card, and selecting large models with 65 billion and 72 billion parameters for more than 2 A800 computing cards.

[0013] Furthermore, in step S1), fine-tuning the selected large model in a specific task field specifically includes: using an efficient parameter fine-tuning method based on low-rank matrix decomposition to perform vertical field fine-tuning of the large model, low-rank matrix decomposition to freeze the pre-trained large model weights, and injecting the trainable rank decomposition matrix into each weight of the Transformer layer of the large model.

[0014] Furthermore, the weight is:

[0015] W h =W h-1 +ΔW

[0016] Furthermore, the professional model includes target recognition, route planning and game confrontation algorithm.

[0017] Furthermore, the step S3) specifically includes:

[0018] Step S31), environmental information and task evaluation, by analyzing environmental information, clarifying the tasks that the UAV system needs to perform;

[0019] Step S32), use the trained large model to analyze and determine whether a specific professional model needs to be activated, and select the professional model to process the task when necessary.

[0020] Furthermore, the training of the large model includes: large model activation rule training, training the large model through data sets, so that the large model can learn what kind of combat tasks require the activation of professional models, and continuously optimize the training of the large model by introducing new specific task data and feedback.

[0021] Furthermore, a standardized interface protocol is used for data transmission between the large model and the professional model, and the transmitted data is packaged according to a predetermined data structure.

[0022] An intelligent adaptive scheduling system for unmanned aerial vehicles based on a large model, comprising:

[0023] The large model training unit trains the large model through data sets, so that the large model can learn which combat missions require the activation of specialized models, and continuously optimizes the training of the large model by introducing new specific mission data and feedback;

[0024] The large model activation unit, in the face of different task scenarios, analyzes and determines whether a specific professional model needs to be activated, and selects a professional model to process the task when necessary;

[0025] The professional model unit receives the parameter information transmitted by the large model, executes the corresponding subtasks, and feeds back the execution results to the large model training unit, so that the large model can perform continuous learning optimization;

[0026] The manual selection unit is used to modify the judgment of the large model activation unit or the selected professional model when the large model activation unit makes an error in judgment or selects an incorrect professional model, and feeds back the modification result to the large model training unit, so that the large model can perform continuous learning and optimization.

[0027] A computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the unmanned aerial vehicle intelligent adaptive scheduling method.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention integrates the big model technology into the UAV system, so that the big model can adaptively call the professional algorithm model built into the UAV system according to the external real-time environmental situation and the specific task type, including target recognition, route planning, flight control algorithm, etc.; by integrating the big model technology, the present invention enables the UAV system to respond to various complex environments more intelligently, and flexibly call the corresponding algorithm model according to the task requirements to optimize the task execution process; the present invention adopts the big model technology to efficiently process multimodal data, make decisions quickly, and improve the response speed and execution ability of the UAV system; in practical applications, the method of the present invention enables the UAV to complete the task faster and improve the accuracy and reliability of task completion. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0030] This patent proposes an innovative technical logic, which is to build a UAV intelligent algorithm adaptive scheduling system based on a large model by calling and integrating professional military models. Specifically, the algorithms such as target recognition, route planning, and game confrontation are encapsulated into small models and the calling interface is exposed to the large model, which is uniformly managed and scheduled. When receiving a specific task, the large model acts as the "brain" of the entire system. According to the task requirements, it automatically calls the appropriate intelligent algorithm model to solve the corresponding problem, thereby realizing the fully autonomous task execution of the UAV system. At the same time, the large model can also improve its scheduling and management capabilities for small models through continuous learning and optimization, thereby further improving the level of intelligence. Combined with Figure 1 , an intelligent adaptive scheduling method for UAVs based on a large model, specifically comprising the following steps:

[0031] S1) According to the specific mission requirements and computing resources of the UAV system, the large model is carefully selected, and the selected large model is fine-tuned for specific mission areas;

[0032] This step firstly selects the large model in detail according to the specific combat mission requirements and computing resources of the UAV system to ensure that the selected model is compatible with the system and can perform reasoning under the corresponding computing power. Then, by introducing proprietary data in specific mission areas, the selected large model is fine-tuned in specific mission areas to enhance its application effectiveness in vertical fields. In addition, fine-tuning is required to improve the ability of large models to schedule and manage professional models.

[0033] S2) Integrate multiple professional models into the UAV system to meet different mission requirements;

[0034] In order to successfully complete the mission, drones need to integrate a variety of professional models to meet different mission requirements, including algorithms such as target recognition, route planning, autonomous obstacle avoidance, and mission planning. During the integration process, special emphasis is placed on the reliability, real-time, and security of the algorithm to ensure that the system can stably perform combat missions in complex and dynamic environments.

[0035] S3) In the face of different task scenarios, the big model analyzes and determines whether a specific professional model needs to be activated, and selects the appropriate professional model to handle the task when necessary;

[0036] In the face of different mission scenarios, the large model will first analyze and determine whether a specific professional model needs to be activated, and select the appropriate professional model to handle the task when necessary. The large model itself has a strong understanding ability, for example, it can convert the operator's verbal instructions into operating instructions for the drone system. Only when performing a specific task will the large model decide to call the corresponding professional model. For example, the large model itself does not have the ability to directly solve the target recognition task. At this time, the large model will call the target recognition model to complete the task.

[0037] S4) The large model confirms that a professional model needs to be called, and based on the collected environmental information and task instructions, the required parameters are automatically extracted and passed to the professional model;

[0038] Given that different professional models have different requirements for parameter information when performing tasks, such as target detection algorithms require image information, and route planning algorithms require information such as the starting point, end point, and obstacles. After confirming that a professional model needs to be called, the large model automatically extracts and passes the required parameters to the professional model based on the collected intelligence and combat instructions.

[0039] S5) The specialized model receives the parameter information transmitted by the large model, executes the corresponding subtask, and feeds back the execution results to the large model, which then makes a decision;

[0040] After receiving the parameter information transmitted by the large model, the professional model will perform the corresponding subtasks and feed back the execution results to the large model. The large model will make subsequent decisions based on these results. For example, after receiving information such as the starting point, end point, and obstacle location, the drone route planning model will generate the optimal route, and the large model will decide whether to follow the route to execute the flight.

[0041] Step S1) includes selecting a large model and fine-tuning the large model in a lightweight manner, including:

[0042] S11) Select large models based on the computing power of the drone system. Based on the relevant information of the open source large models, the deployment and inference of large models with 6 billion and 13 billion parameters requires about 1 A800 computing card, and the deployment and inference of large models with 65 billion and 72 billion parameters requires more than 2 A800 computing cards.

[0043] S12) This patent intends to use an efficient parameter fine-tuning method based on low-rank matrix decomposition to achieve vertical field fine-tuning of the base model. Low-rank matrix decomposition is a base-to-low-rank adaptive fine-tuning technology. It uses a low-rank matrix decomposition method on the weight matrix of a large model to significantly reduce the number of model parameters, thereby reducing computational complexity and reducing memory requirements. Low-rank matrix decomposition freezes the pre-trained model weights and injects the trainable rank decomposition matrix into each weight of the Transformer layer. The weight update formula of the pre-trained model is as follows:

[0044] W h =W h-1 +ΔW

[0045] W h is the weight of the hth layer, W h-1 is the weight of the hth layer of the large model, ΔW is the update of the large model at each layer, the original parameters plus the updated parameters are the updated model weights. For example, W0 is (1.1, 1.2, 1.3), ΔW is (0.1, 0.2, 0), the new weights are (1.2, 1.4, 1.3)

[0046] Step S2) integrating professional models into the UAV system to cope with different combat tasks, the professional models include target recognition, route planning, game confrontation, etc., including the following steps:

[0047] S21) Integrate professional models into the UAV system and clarify the input and output of different algorithms. The target detection algorithm requires image information and returns the target information in the image in text form. The route planning algorithm requires information such as the starting point, end point, obstacles, etc. and returns the optimal path.

[0048] S22) The professional model needs to expose an interface to the outside for the large model to call, and also needs to expose a parameter interface to receive parameter information transmitted by the large model.

[0049] Step S3) The large model determines whether to activate the professional model according to the environment information, including:

[0050] S31) Environmental information and mission assessment: By analyzing environmental information, clarify the tasks that the UAV system needs to perform. Integrate data from multiple sensors and intelligence sources, and apply advanced analysis techniques such as multi-source data fusion, feature extraction, and pattern recognition to generate comprehensive environmental perception. Based on the environmental perception results, clarify the specific combat tasks that the UAV system needs to perform, such as flight control, path planning, or target identification.

[0051] S32) Training of the big model activation rules. Through fine-tuning training, the big model learns what kind of combat mission needs to activate the professional model. By introducing new specific mission data and feedback, the big model is continuously fine-tuned to improve the accuracy and efficiency of its activation decision. The big model trained by the present invention can dynamically adjust its decision to activate the professional model according to the changing environment and mission requirements.

[0052] Step S4) automatically extracts and transfers required parameters to the professional model based on the collected information and task instructions, specifically including:

[0053] S41) The large model analyzes the collected information and task instructions, and extracts the parameters required for the professional model to perform the task.

[0054] S42) After completing the parameter extraction, the large model needs to pass these parameters to the corresponding professional models. Define a set of standardized interface protocols to ensure that the parameter transfer between the large model and the professional models is correct and efficient. Encapsulate the extracted parameters according to a predetermined data structure, such as JSON, XML or Protocol Buffers, to facilitate network transmission or storage. Establish a reliable communication mechanism, such as using TCP / IP protocol or message queue (MQ), to ensure that the parameters can be accurately passed to the professional models.

[0055] Step S5) The professional model receives the parameter information transmitted by the large model, executes the corresponding subtask, and feeds back the execution results to the large model, and the large model makes a decision, which specifically includes:

[0056] S51) The specialized models receive parameter information from the large model through a well-defined interface, which is responsible for parsing the parameters and ensuring that they are in the correct format.

[0057] S52) The professional model executes relevant algorithms to complete the corresponding tasks.

[0058] S53) After the professional model completes the subtask, it feeds back the execution result to the big model so that the big model can make further decisions. The professional model encapsulates the execution result into a unified data format, such as a JSON object, and then sends it to the big model through a predefined API or message queue. After receiving the result, the big model uses the corresponding parser to understand the output of the professional model and convert it into information that can be used for decision-making.

[0059] An intelligent adaptive scheduling system for unmanned aerial vehicles based on a large model, comprising:

[0060] The large model training unit trains the large model through data sets, so that the large model can learn which combat missions require the activation of specialized models, and continuously optimizes the training of the large model by introducing new specific mission data and feedback;

[0061] The large model activation unit, in the face of different task scenarios, analyzes and determines whether a specific professional model needs to be activated, and selects a professional model to process the task when necessary;

[0062] The professional model unit receives the parameter information transmitted by the large model, executes the corresponding subtasks, and feeds back the execution results to the large model training unit, so that the large model can perform continuous learning optimization;

[0063] The manual selection unit is used to modify the judgment of the large model activation unit or the selected professional model when the large model activation unit makes an error in judgment or selects an incorrect professional model, and feeds back the modification result to the large model training unit, so that the large model can perform continuous learning and optimization.

[0064] A computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the unmanned aerial vehicle intelligent adaptive scheduling method.

[0065] In the technical logic of this patent, the big model calls the professional model in an efficient way. First, the professional model is integrated into the drone system, and the parameter interface is exposed to the outside so that the big model can be called smoothly. Secondly, when facing different scenarios and tasks, the big model will first analyze to determine whether the professional model needs to be called and which specific professional model needs to be called. Next, the big model will further analyze the scenario and task to extract the professional model parameters required for the call. Finally, through the back-end request, the big model successfully calls the professional model and makes the next task decision based on the solution result of the professional model. This technical logic realizes the close integration of the big model and the professional model, and improves the efficiency and accuracy of problem solving.

[0066] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0067] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model, characterized in that: Includes steps: S1), select a large model based on the specific mission requirements and computing resources of the UAV system, and fine-tune the selected large model in specific mission areas; S2), integrating multiple professional models to perform different mission requirements in the UAV system; S3), in the face of different task scenarios, the large model analyzes and determines whether a specific professional model needs to be activated, and selects a professional model to process the task when necessary; S4), the big model confirms the professional model to be called, and automatically extracts and transfers the required parameters to the professional model based on the collected environmental information and task instructions; S5), the professional model receives the parameter information transmitted by the large model, executes the corresponding subtasks, and feeds back the execution results to the large model, so that the large model can perform continuous learning and optimization.

2. The method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model according to claim 1 is characterized in that: The selection of large models based on the specific mission requirements and computing resources of the UAV system includes: selecting large models with 6 billion and 13 billion parameters for 1 A800 computing card, and selecting large models with 65 billion and 72 billion parameters for more than 2 A800 computing cards.

3. The method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model according to claim 1 is characterized in that: In step S1), fine-tuning the selected large model in a specific task field specifically includes: using an efficient parameter fine-tuning method based on low-rank matrix decomposition to perform vertical field fine-tuning of the large model, low-rank matrix decomposition to freeze the pre-trained large model weights, and injecting the trainable rank decomposition matrix into each weight of the Transformer layer of the large model.

4. The method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model according to claim 3 is characterized in that: The weight update is: IN h =In h-1 +ΔW Among them, W h is the weight of the hth layer of the large model, W h-1 is the weight of the hth layer of the large model, and ΔW is the update of the large model at each layer.

5. The method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model according to claim 1 is characterized in that: The professional model includes target recognition, route planning and game confrontation algorithm.

6. The method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model according to claim 1 is characterized in that: The step S3) specifically includes: Step S31), environmental information and task evaluation, by analyzing environmental information, clarifying the tasks that the UAV system needs to perform; Step S32), use the trained large model to analyze and determine whether a specific professional model needs to be activated, and select the professional model to process the task when necessary.

7. The method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model according to claim 6 is characterized in that: The training of the large model includes: large model activation rule training, training the large model through data sets, so that the large model can learn what kind of combat tasks require the activation of professional models, and continuously optimizing the training of the large model by introducing new specific task data and feedback.

8. The method for intelligent adaptive scheduling of unmanned aerial vehicles based on a large model according to claim 1, characterized in that: A standardized interface protocol is used for data transmission between the large model and the professional model, and the transmitted data is packaged according to a predetermined data structure.

9. A large model-based UAV intelligent adaptive scheduling system for implementing any of the methods described in claims 1-8, characterized in that: include: The large model training unit trains the large model through data sets, so that the large model can learn which combat missions require the activation of specialized models, and continuously optimizes the training of the large model by introducing new specific mission data and feedback; The large model activation unit, in the face of different task scenarios, analyzes and determines whether a specific professional model needs to be activated, and selects a professional model to process the task when necessary; The professional model unit receives the parameter information transmitted by the large model, executes the corresponding subtasks, and feeds back the execution results to the large model training unit, so that the large model can perform continuous learning optimization; The manual selection unit is used to modify the judgment of the large model activation unit or the selected professional model when the large model activation unit makes an error in judgment or selects an incorrect professional model, and feeds back the modification result to the large model training unit, so that the large model can perform continuous learning and optimization.

10. A computer storage medium, characterized in that: The computer storage medium stores an executable program, and the executable program is executed by a processor to implement the steps of the drone intelligent adaptive scheduling method according to any one of claims 1 to 8.

Images