A UAV control method and system with accuracy maintenance and enhanced generalization capability

Abstract

The present invention discloses a method and system for controlling a drone with maintained accuracy and enhanced generalization capability. The method includes obtaining multimodal data collected by multiple sensors carried by the drone; constructing a reinforcement learning model based on the multimodal data to generate an action strategy in a simulation environment; updating the multimodal data in real time according to environmental changes, and updating the action strategy; migrating the action strategy from the simulation environment to the real environment, and adjusting the migrated action strategy using evaluation data, and generating a drone action sequence in the real environment using the adjusted action strategy, which can improve the generalization of the drone control system while maintaining the accuracy of the drone control system.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control technology, and in particular to a method and system for controlling an unmanned aerial vehicle with maintained accuracy and enhanced generalization capability. Background Art

[0002] At present, with the advancement of technology and the improvement of regulations, drones have been applied in many fields, such as emergency rescue, disaster monitoring, logistics and transportation. For specific tasks in specific fields, in order to ensure the efficiency and performance of drones in specific scenarios, the control method of drones is often optimized for specific scenarios to ensure that drones have high accuracy in specific scenarios. However, this control method performs generally in other scenarios and is not generalizable.

[0003] Researchers have carried out a lot of work on improving the generalization and scene adaptability of drone control systems, involving model design, training methods, data strategies, optimization algorithms and other aspects. For example: developing multimodal models, such as CLIP and DALL-E, combining modal information such as vision and text, and improving model generalization capabilities through cross-modal learning; transfer learning reduces dependence on large amounts of new data by transferring knowledge of existing models in similar tasks or fields to new tasks. Existing studies have proposed training general models based on large-scale drone data sets, and reducing the dependence of drone systems on mission environments by fine-tuning them to adapt to different tasks. However, although this method improves the generalization of drones, it will lead to reduced accuracy of drone control systems.

[0004] The real progress in the deployment of UAV operations lies in achieving "precise versatility", that is, solving a variety of tasks without sacrificing accuracy. Therefore, there is currently a lack of general methods that can effectively improve the generalization and scenario adaptability of UAV systems. Summary of the invention

[0005] In order to solve the technical problem that the current UAV control system cannot maintain the accuracy of the UAV system while improving the generalization, the embodiments of the present invention provide a UAV control method and system with accuracy maintenance and enhanced generalization capability.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] The embodiment of the present invention provides a method for controlling a drone with maintained accuracy and enhanced generalization capability, the method comprising: obtaining multimodal data collected by a plurality of sensors carried by the drone, using the multimodal data to build a scene graph, and using the scene graph to extract environmental features; determining the mission objectives, constraints and reward functions of the drone based on the mission requirements of the drone according to the scene graph and the environmental features; constructing a reinforcement learning model based on the scene graph, the environmental features, the mission objectives, the constraints and the reward function to generate an action strategy in a simulation environment; obtaining an operation action set of the drone, iteratively updating the operation action set based on the action strategy in the simulation environment, and generating an action sequence after performing a value assessment on the operation actions in the operation action set; updating the multimodal data in real time according to environmental changes, and regenerating the scene graph, environmental features and the scene sequence using the updated multimodal data. , the mission objectives, constraints and reward functions of the UAV; based on the regenerated scene graph, the environmental features, the mission objectives, constraints and reward functions of the UAV, update the action strategy in the simulation environment; redefine the action sequence of the UAV using the updated action strategy in the simulation environment; perform the UAV task according to the redetermined action sequence of the UAV, and generate evaluation data after the task is completed; use a generative adversarial network to align the simulation environment features with the real environment features to generate a state alignment model; based on the state alignment model, migrate the action strategy in the simulation environment to the real environment to obtain the action strategy after migrating to the real environment; update the real environment features in real time according to environmental changes, and adjust the action strategy after migrating to the real environment according to the evaluation data, so as to generate a new UAV action sequence in the real environment with the updated real environment features and the adjusted action strategy.

[0008] In one embodiment, multimodal data collected by multiple sensors carried by a drone are obtained, a scene graph is constructed using the multimodal data, and environmental features are extracted using the scene graph, including: obtaining multimodal data collected by multiple sensors carried by a drone, extracting features from each modal data in the multimodal data, and fusing features of all modal data to obtain fused features; extracting temporal features according to temporal changes of the fused features; constructing a scene graph based on the fused features and the temporal features; the nodes in the scene graph are multiple features after feature mapping of the fused features; the edges of the scene graph are calculated by distances between nodes and the temporal features; updating nodes in the scene graph, and generating environmental features based on the updated scene graph.

[0009] In one embodiment, the constraint condition of the drone is expressed as:

[0010]

[0011] in, Constraints, For collision avoidance constraints, is the energy limit constraint, For other constraints;

[0012] in,

[0013]

[0014] in, is a set of waypoints; Indicates the obstacle area; Indicates that the waypoint does not overlap with the obstacle area;

[0015]

[0016] in, ; Indicates from arrive energy consumption; The remaining battery power of the drone.

[0017] In one embodiment, the reward function of the drone is expressed as:

[0018]

[0019] Among them, the state s is composed of the environmental characteristics and mission objectives ,Right now: ; Indicates taking action in state s The reward value of , is the weight coefficient, which is used to balance the importance of different task objectives in the reward; Bonus items related to task efficiency; For rewards related to mission safety; Reward items related to task accuracy; refers to the degree of violation of the constraint. If the constraint is satisfied, the penalty is 0. is a penalty factor used to control the penalty for constraint violation.

[0020] In one embodiment, the expression of the action strategy in the simulation environment is:

[0021]

[0022] in, is the action strategy in the simulation environment, is a discount factor used to measure the importance of long-term rewards and short-term rewards; T is the time step of the task, represents the state of the drone at time t, , For environmental characteristics, For the mission objectives, This means to find the expected value: Indicates status Take action The reward value.

[0023] In one embodiment, the operation action set is iteratively updated based on the action strategy in the simulation environment, and the operation actions in the operation action set are value evaluated to generate an action sequence, including:

[0024] The following calculation formula is used to evaluate the value of the operation actions in the operation action set:

[0025]

[0026] in, (s, ) indicates that the state s is in the action strategy Take action The cumulative reward that can be obtained, which represents the value of the current strategy and is used to select the optimal action; Indicates status Take action The reward value of state s is determined by the environmental characteristics and mission objectives Composition, that is ; is a discount factor used to weigh the importance of long-term rewards and short-term rewards; Indicates that from the current state Take Action Then, transfer to the next state , and choose the best action Expected return after

[0027] The action sequence is generated using the following calculation formula:

[0028]

[0029] in, Indicates the state at time t Strategy in Action Take action The cumulative rewards that can be obtained.

[0030] In one embodiment, re-determining the action sequence of the drone using the updated action strategy in the simulation environment includes:

[0031] Re-determining the optimized action of the drone according to the current state of the drone and the updated action strategy in the simulation environment; and regenerating the action sequence of the drone according to the current state of the drone and the updated action strategy in the simulation environment;

[0032] The optimization action of the drone is re-determined using the following calculation formula:

[0033]

[0034] in, is the optimization action at the current moment, indicating the state at time step t Next, choose the best action , is the state-action value function, which means that in the state Strategy in Action Take action The expected return that can be obtained when

[0035] The action sequence of the drone is re-determined using the following calculation formula:

[0036]

[0037] in, To re-determine the drone action sequence, is the state-action value function, which means that in the state Strategy in Action Take action The expected return that can be obtained.

[0038] In one embodiment, the evaluation data includes task completion rate, safety and resource utilization;

[0039] The calculation formula of the task completion rate is:

[0040]

[0041] in, is the task completion rate, is the number of completed tasks, is the total number of tasks;

[0042] The safety calculation formula is:

[0043]

[0044] in, For safety, is the number of collision-free tasks, is the total number of tasks;

[0045] The calculation formula of the resource utilization is:

[0046]

[0047] in, is the resource utilization rate, is the actual resource consumption, The total amount of available resources.

[0048] In one embodiment, adjusting the action strategy after migrating the real environment according to the evaluation data includes:

[0049] The action strategy after migrating to the real environment is adjusted according to the evaluation data through the following calculation formula:

[0050]

[0051] in, is the updated action strategy, which is used to select the optimal action in state s; To transfer the action strategy after the real environment; Represents the increment of the strategy optimized based on the evaluation data.

[0052] An embodiment of the present invention also provides a drone control system with maintained accuracy and enhanced generalization capability, comprising: a processor and a memory for storing a computer program that can be run on the processor; wherein the processor executes the steps of the above-described method when running the computer program.

[0053] This embodiment has the following beneficial effects:

[0054] This embodiment performs feature extraction and fusion on multimodal data, and can accurately perceive detailed information in the environment (such as object positions, target markers, obstacles), thereby improving the accuracy of task execution.

[0055] This embodiment constructs a mission scenario in a simulation environment based on multimodal data, establishes a mathematical model for task execution, clarifies mission objectives and constraints, optimizes action strategies through reinforcement learning, and controls the UAV to perform tasks according to the action strategies. This ensures that the UAV can accurately perform tasks according to predetermined goals (such as the shortest path, shortest time, etc.) and reduce execution errors.

[0056] This embodiment updates the action strategy according to dynamic changes in the environment, so that the drone can adapt to changes in various environments (such as sudden obstacles and different weather conditions), thereby enhancing the performance of the drone in diverse scenarios.

[0057] This embodiment reduces the performance loss from simulation to reality by aligning simulation data with real data and migrating strategies, and ensures the accurate execution of the migrated strategies in the real environment.

[0058] The migrated strategy in this embodiment can adapt to multiple unknown scenarios without the need for additional labeled data. Through rapid migration and adaptive adjustment, the cross-scenario application capability of the system is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a flow chart of a method for controlling a drone with accuracy preservation and generalization enhancement according to an embodiment of the present invention;

[0060] Figure 2 This is a schematic diagram of the process of the drone system according to an embodiment of the present invention;

[0061] Figure 3 1 is a diagram showing the internal structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0063] The embodiment of the present invention provides a method for controlling a drone with high accuracy and high generalization capability. Figure 1 As shown, the method includes:

[0064] Step 101: obtaining multimodal data collected by multiple sensors carried by the drone, using the multimodal data to build a scene graph, and using the scene graph to extract environmental features; determining the mission objectives, constraints and reward functions of the drone based on the mission requirements of the drone according to the scene graph and the environmental features;

[0065] Step 102: constructing a reinforcement learning model based on the scene graph, the environmental features, the task objectives, the constraints and the reward function, and generating an action strategy in a simulation environment; obtaining an operation action set of the drone, iteratively updating the operation action set based on the action strategy in the simulation environment, and generating an action sequence after performing a value evaluation on the operation actions in the operation action set;

[0066] Step 103: updating the multimodal data in real time according to environmental changes, and regenerating the scene graph, environmental features, mission objectives of the drone, constraints and reward functions using the updated multimodal data; updating the action strategy in the simulation environment based on the regenerated scene graph, environmental features, mission objectives of the drone, constraints and reward functions; redetermining the action sequence of the drone using the updated action strategy in the simulation environment; executing the drone mission according to the redetermined action sequence of the drone, and generating evaluation data after the mission is completed;

[0067] Step 104: using a generative adversarial network to align the features of the simulation environment with the features of the real environment to generate a state alignment model; based on the state alignment model, migrating the action strategy in the simulation environment to the real environment to obtain the action strategy after migrating to the real environment;

[0068] Step 105: Update the real environment characteristics in real time according to the environmental changes, and adjust the action strategy after migrating the real environment according to the evaluation data, so as to generate a drone action sequence in a new real environment with the updated real environment characteristics and the adjusted action strategy.

[0069] This embodiment proposes a method for maintaining the accuracy and enhancing the generalization capability of a UAV system based on simulation learning, so that the UAV system can handle multi-scenario tasks without direct experience, and helps develop a UAV system with high accuracy and strong generalization capability.

[0070] The UAV flight simulation tool can support the simulation of various weather, terrain and environment. By simulating flight in a virtual environment, the performance, flight control algorithm and mission planning of the UAV can be verified, avoiding high-cost and risky actual experiments. This embodiment first optimizes the action strategy of the UAV based on task modeling and uses simulation learning to achieve efficient execution of the UAV; then, on the premise of reducing the difference in data distribution between the simulation environment and the real environment, the action strategy in the simulation environment is migrated to the real environment, thereby realizing the application of the model in different scenarios, improving the generalization of the model, and enhancing the adaptability of the model in multiple real scenarios.

[0071] Specifically, after acquiring multimodal data collected by multiple sensors carried by the drone, this embodiment first extracts features from each modal data in the multimodal data, and fuses the features of all modal data to obtain fused features; extracts time series features according to the time series changes of the fused features; then constructs a scene graph based on the fused features and the time series features; the nodes in the scene graph are multiple features after feature mapping of the fused features; the edges of the scene graph are calculated by the distance between the nodes and the time series features; the nodes of the scene graph are updated, and environmental features are generated based on the updated scene graph.

[0072] Here, the multimodal data collected by various sensors include but are not limited to:

[0073] RGB images : Collected by high-definition camera and represented as a three-dimensional tensor ,in are the height, width and number of channels of the image respectively;

[0074] Depth Map : Obtained through a depth camera or stereo vision module, represented as a two-dimensional matrix ;

[0075] Point cloud data : Generated by LiDAR, represented as a collection of points: , where N is the number of points, is the three-dimensional coordinate of the point;

[0076] Other modal data S: including infrared thermal imaging matrix, environmental acoustic signals, etc., expressed in matrix or vector form.

[0077] Before feature extraction, the above multimodal data can be synchronized to the same time base by aligning timestamps: , where t represents time, It is the synchronized multimodal input data set.

[0078] After alignment is completed, relevant features can be extracted from the above multimodal data. Here, the specific extraction methods include:

[0079] RGB image feature extraction: Convolutional neural network (CNN) is used to extract local and global visual features: ,in Output feature tensor for pre-trained visual feature extraction networks, such as ResNet ;

[0080] Deep image feature extraction: Extract depth information through a specific three-dimensional convolutional network (3D-CNN): ,in is a deep feature extractor that outputs feature tensors .

[0081] Point cloud feature extraction: Use PointNet++ to extract features from point cloud data: ,in For point cloud processing network, output global feature vector .

[0082] Other modality feature extraction: Process other modality data through convolutional or recurrent neural networks: ,in for modality-specific feature extraction networks.

[0083] After the above features are extracted, the attention mechanism can be used to weight the features of different modalities to obtain fusion features. Here, the weighting method can be:

[0084] ,

[0085] in, For modal The feature importance score of is the scoring function, ∈( , , ).

[0086] In addition, for the weighted fusion features, GRU can be used to capture the temporal changes of the features. The calculation formula is:

[0087]

[0088] This embodiment can more accurately perceive detailed information in the environment (such as object locations, target markers, obstacles) through the fusion of multimodal data, thereby improving the accuracy of task execution.

[0089] After obtaining the fusion features and the temporal features, a scene graph may be created, and nodes of the scene graph may be updated, and environmental features may be generated based on the updated scene graph. Here, the specific process may be:

[0090] Step 1: Node construction: Each key point in the environment (obstacle, path node, target point) is used as a node V of the graph, and the node features are composed of fusion features supply, ,

[0091] in, is the feature mapping function, Indicates The characteristics of a node.

[0092] Step 2: Edge construction: The edges E in the graph are determined by the geometric relationship between nodes (such as Euclidean distance, connectivity), and the edge features are determined by the time series features. And the node distance is calculated:

[0093] , in, and For Node and spatial location, Generates functions for edge features.

[0094] Step 3: Scene graph expression: The constructed scene graph is represented as: ,in, , .

[0095] Step 4: Node update: Use graph neural network (GNN) to update the node features in the environment graph, combining the features of neighbor nodes and edge features:

[0096]

[0097] in:

[0098] Representation Node In the Characteristics of the layer;

[0099] Representation Node The set of neighbors of ;

[0100] It is the mapping function between edge features and neighbor node features;

[0101] It is the feature aggregation and update function.

[0102] Step 5: Global feature generation: after updating multiple layers of nodes, the environmental features of the entire graph are generated. , representing the entire scene:

[0103]

[0104] Among them, L is the number of layers of the GNN network, and the Pooling operation can be average pooling or maximum pooling.

[0105] Through the above steps, the environmental characteristics can be obtained and scene graph . In environmental characteristics Based on this, we can further obtain the task objectives Here, the environmental characteristics Extract task-related areas (such as target points, dangerous areas) and express the task objectives in quantitative form: , where each It is a task execution indicator, such as shortest path, maximum security, etc.

[0106] In addition, based on the scene graph and global features You can also model constraints. Here, you can specifically establish the following constraints:

[0107]

[0108] in, As constraints, For collision avoidance constraints, is the energy limit constraint, For other constraints;

[0109] in,

[0110]

[0111] in, is a set of waypoints; Indicates the obstacle area; Indicates that the waypoint does not overlap with the obstacle area;

[0112]

[0113] in, is a set of path points,

[0114] Indicates from arrive energy consumption; The remaining battery power of the drone.

[0115] In addition, the following reward function can be established:

[0116]

[0117] Among them, the state s is composed of the environmental characteristics and mission objectives Composition, namely: ; Indicates taking action in state s The reward value of , is the weight coefficient, which is used to balance the importance of different task objectives in the reward; Bonus items related to task efficiency; For rewards related to mission safety; Reward items related to task accuracy; refers to the degree of violation of the constraint. If the constraint is satisfied, the penalty is 0. is a penalty factor used to control the penalty for constraint violation.

[0118] Establishing mission objectives , Constraints And the reward function After that, based on the mission objectives , Constraints And the reward function Construct a reinforcement learning model to generate an action strategy in a simulation environment; and iteratively update the action strategy in the simulation environment, perform value evaluation, and generate an action sequence. Here, the specific process can be:

[0119] Step 1: Action space definition, define the action set according to the drone's operating capabilities ,For example:

[0120] {change speed, adjust direction, ..., hover}

[0121] Furthermore, considering the constraints , for example: Collision Avoidance: Action Set Each action in the set must ensure that the drone does not enter the obstacle area; Energy limit: All action sequences must satisfy the energy consumption does not exceed the maximum limit. Legal action set As shown in the following formula:

[0122]

[0123] Step 2: Reinforcement learning model construction, using deep reinforcement learning (DRL) method to build a policy network and optimize action selection through iterative updates:

[0124]

[0125] in: is a discount factor that measures the importance of long-term rewards and short-term rewards; T is the time step of the task. represents the state of the drone at time t, This means to find the expected value: Indicates status Take action The reward value.

[0126] Step 3: Action evaluation, evaluate the value of the state-action pair through Q-learning ( , ):

[0127]

[0128] in: To perform actions The next state after is the optimal action in the next state; Indicates that from the current state Take Action Then, transfer to the next state , and choose the best action Expected return after (s, ) indicates that the state s is in the strategy Take action The cumulative reward that can be obtained represents the value of the current strategy and is used to select the optimal action.

[0129] Step 4: Action sequence generation, based on policy network And Q-learning algorithm, generate a complete action sequence:

[0130]

[0131] This sequence is guaranteed to satisfy the constraints To achieve the mission objectives .

[0132] In this embodiment, the strategy optimized by reinforcement learning can ensure that the drone can accurately perform tasks according to predetermined goals (such as the shortest path, shortest time, etc.), reduce execution errors, and enhance the accuracy of drone execution.

[0133] In addition, as the environment changes, the multimodal data will be updated in real time. To ensure the accuracy of the action strategy, the action strategy can be dynamically updated in real time according to environmental changes to enhance the adaptability of the drone to environmental changes and ensure the accuracy of the drone's mission execution.

[0134] Here, the update process can be specifically:

[0135] Step 1: Dynamic environment perception: Update the task objectives and constraints according to the changes in the environment and real-time feedback during task execution. The updated state is expressed as :

[0136]

[0137] in, is the new state, which represents the updated state representation according to the multimodal data and task objectives in the new environment; To update the environmental features, the environmental change data (such as the movement of obstacles, changes in target points, etc.) is perceived in real time.

[0138] Combined state transition data , analyze the impact of dynamic environment on task objectives, such as: changes in obstacle positions, changes in path passability, and adjustments to task priorities.

[0139] Step 2: Dynamic strategy update, based on the new state representation , adjust the action strategy to ensure that the strategy can still achieve the task goal in the new environment:

[0140]

[0141] in, is the updated dynamic strategy, which indicates the probability of taking an action in the updated state; Refers to the expected return, which means that after executing an action, from the current state To begin with, the sum of rewards at each moment in the future; is the discount factor.

[0142] In a dynamic environment, the following two optimization mechanisms are used:

[0143] Local adjustment: The purpose of the local adjustment mechanism is to optimize each step in real time based on the current state and feedback during execution. This mechanism enables the strategy to respond to subtle changes and ensure accurate execution of the task. The following formula is to reselect the optimal action based on environmental changes The formula is:

[0144]

[0145] in, is the optimization action at the current moment, indicating the state at time step t Next, choose the best action ; State-action value function, which means that in state Next, take action The expected return that can be obtained.

[0146] Global adjustment: The global optimization mechanism is a comprehensive optimization of the entire task execution process. During the overall execution of the task, the strategy is re-evaluated and optimized to maximize the long-term return. When the environment changes significantly (such as the path is completely unavailable), the action sequence is regenerated:

[0147]

[0148] in, is the globally optimized action sequence, a new action sequence generated by evaluating the action selections at all time steps in the entire task .

[0149] This embodiment can optimize the execution effect of the strategy and ensure the accuracy of task execution (such as collision avoidance, path accuracy, etc.) through real-time monitoring of environmental changes and feedback evaluation. In addition, through task modeling and dynamic adjustment, the strategy can be flexibly adjusted under different task objectives and environmental changes, reducing excessive dependence on specific scenarios and ensuring the applicability of the strategy in various scenarios.

[0150] The drone performs the task according to the above action sequence. After the task is completed, a method for evaluating the task completion is required to further evaluate the accuracy of the action sequence determined by the above action strategy. Here, this embodiment can generate evaluation data Specifically, the evaluation data It may include task completion rate, safety, resource utilization, etc. The specific formula is as follows:

[0151] Task completion rate:

[0152]

[0153] in, is the number of completed tasks, is the total number of tasks;

[0154] Safety, counting the number of collisions or potential dangers during task execution:

[0155]

[0156] in, is the number of collision-free tasks;

[0157] Resource utilization, the resources consumed to complete the calculation task:

[0158]

[0159] in, is the actual resource consumption, The total amount of available resources.

[0160] Through the above process, the action strategy and evaluation data in the simulation environment can be obtained. In order to apply the action strategy in the simulation environment to the real environment, it is also necessary to migrate the action strategy in the simulation environment and improve the accuracy and adaptability of the action strategy in the simulation environment to the real environment through alignment and optimization.

[0161] Here, the migration, alignment, and optimization processes may specifically be:

[0162] Use the Generative Adversarial Network (GAN) to align the simulation and real environment features. The formula is as follows:

[0163]

[0164] Among them: X represents the real feature, Z represents the simulated feature; X~P Real Indicates that the true feature obeys P Real Distribution; Z~P simIndicates that the simulation characteristics obey P sim distributed;

[0165] G is the generator, which simulates the features Transformed into data similar to the real features;

[0166] D is the discriminator, which distinguishes between simulated and real features;

[0167] The distribution of real environment features is obtained in real time through the task perception execution module;

[0168] To simulate the distribution of environmental characteristics, from the simulation data Environmental characteristics obtained from .

[0169] Through adversarial training, the generator G learns to map features in the simulation environment to corresponding features in the real environment, thereby achieving migration from simulation to reality.

[0170] Generate state alignment model , the implementation process is shown in the following formula:

[0171]

[0172] in and Distribute state representations of real and simulated environments.

[0173] The strategy migration method will simulate the strategy By mapping the state alignment to the real environment, we can ensure that the strategy remains efficient in the real environment. The implementation process is shown in the following formula:

[0174]

[0175] in, The action selection strategy is obtained through reinforcement learning for the strategy in the simulation environment;

[0176] To align the model across modalities, represent the state in the simulation environment Mapping to the state of the real environment ;

[0177] To transfer the strategy to the real environment, it indicates the probability of selecting the optimal action under the state of the real environment.

[0178] The migrated strategy is tested in a real environment A comprehensive evaluation is conducted to identify deficiencies and optimize strategies through a feedback mechanism. The output optimization strategy directly reflects the improvement in the accuracy and generalization of the system, enabling the UAV system to maintain high accuracy and strong generalization in different scenarios.

[0179] Step 1: Perform adaptability assessment to evaluate task completion rate, safety, and resource utilization:

[0180]

[0181] in, is the task completion rate; It is a mission safety assessment; It is a resource utilization assessment.

[0182] Step 2: The environment status is updated online in real time. The latest status is generated through the environmental features collected by multimodal sensors and used as the input for policy optimization to ensure that the strategy adapts to environmental changes:

[0183]

[0184] in, For dynamic environment characteristics, As the mission objective. As the core input for strategy optimization, it ensures that the strategy adapts to environmental changes.

[0185] Step 3: Strategy update and optimization based on evaluation results Adjustment strategy:

[0186] Local adjustment: to the current state Select new action , to achieve instant optimization:

[0187]

[0188] in, is the optimal action at the current moment, that is, in state The best action to choose next;

[0189] is the state-action value function, which means that in the state The expected reward when taking action a.

[0190] Global optimization: Based on the feedback data accumulated during the entire task execution process, adjust the global direction of the strategy to balance task objectives, resource consumption and security:

[0191]

[0192] in, is the updated strategy, which represents the optimized strategy used to select the optimal action in state s; is the strategy after migration, the initial strategy migrated from the simulation environment to the real environment; represents the strategy increment optimized according to the evaluation results, Driven, the size of the increment depends on the feedback data during task execution; the update rule ensures that the strategy is adaptable in all environments.

[0193] Step 4: Action sequence generation and verification, use the updated strategy to generate a new action sequence:

[0194]

[0195] in, is the optimized action sequence; is the state-action value function, evaluating each time step t and executing the action Expected return when.

[0196] After generating a new action sequence, the system will combine the task feedback evaluation indicators Verify its effect to ensure that the optimized action sequence can achieve the expected goal.

[0197] By executing the four steps of adaptability assessment, real-time online update of environmental status, strategy update and optimization, and action sequence generation and verification, the UAV system can continuously adjust its strategy according to real-time feedback and environmental changes during mission execution, optimize mission execution results, and ensure the achievement of mission objectives.

[0198] The method for maintaining the accuracy and enhancing the generalization of the drone system proposed in this embodiment can make full use of simulation data and avoid the expensive problem of data collection in complex scenarios. Researchers use simulation environments to train models, learn knowledge in simulation environments, and then apply models to the real world through migration technology. This "simulation to reality" method helps the model generalize in different scenarios and improves the model's adaptability to multiple real-world scenarios.

[0199] The present solution will be described in detail below in conjunction with a specific embodiment.

[0200] Specifically, see Figure 2, the method of this embodiment is applied to an unmanned aerial vehicle system, and the unmanned aerial vehicle system includes a multimodal data characterization module, a task perception execution module and a zero-sample migration module: wherein the multimodal data characterization module includes a multimodal perception data modeling method, a multimodal fusion method, and a multimodal scene feature representation method. The task perception execution module includes a task modeling method, an action planning and evaluation method, and a task perception optimization method. The zero-sample migration module includes a modality alignment and mapping method, a strategy migration method, and an online adaptive method. Among them, the multimodal perception data modeling method aims to establish a unified data structure to integrate different types of sensor data, laying the foundation for multimodal fusion. The multimodal fusion method first extracts meaningful feature representations from the original data of different modalities, converts the complex original data into a high-dimensional representation suitable for fusion, and then integrates the single-modal features into a whole to ensure that the feature information is comprehensive and the redundancy is minimized. The multimodal scene feature representation method converts the fused features into a structured global representation to provide input for task decision-making and path planning. The task modeling method aims to establish a mathematical model for task execution, clarify task objectives, constraints and indicators, and provide input for subsequent planning and decision-making. The action planning and evaluation method is based on task modeling and uses simulation learning to optimize the UAV's action selection strategy to achieve efficient execution. The task perception optimization method ensures the rationality of action planning and the efficiency of task execution through dynamic environment perception and strategy update. At the same time, its output provides efficient strategy templates and evaluation benchmarks for the zero-sample migration module. The modal alignment and mapping method aims to narrow the difference between the data distribution of the simulation environment and the real environment to ensure the generalization ability of the migration strategy. The strategy migration method uses the aligned real environment state to migrate and optimize the dynamic strategy. The online adaptive method further improves the generalization ability of the UAV system by performing adaptive evaluation and online update of the migrated strategy in the real environment.

[0201] The following will introduce the above contents respectively.

[0202] 1. Multimodal Perception Data Modeling Method

[0203] The multimodal perception data modeling method synchronizes the various sensor data carried by the drone in time series and converts them into a unified input data structure. The data structure construction process is as follows:

[0204] Step 1: Integrate multiple sensor data, including but not limited to:

[0205] RGB images : Collected by high-definition camera, represented as a three-dimensional tensor ,in are the height, width and number of channels of the image respectively;

[0206] Depth Map : Obtained through a depth camera or stereo vision module, represented as a two-dimensional matrix ;

[0207] Point cloud data : Generated by LiDAR, represented as a collection of points: , where N is the number of points, is the three-dimensional coordinate of the point;

[0208] Other modal data S: including infrared thermal imaging matrix, environmental acoustic signals, etc., expressed in matrix or vector form.

[0209] Step 2: Data synchronization and alignment: Synchronize multimodal data to the same time base through timestamp alignment: , where t represents time, It is the synchronized multimodal input data set.

[0210] 2. Multimodal Fusion Method

[0211] The multimodal fusion method extracts key information from each modality data. The process is as follows:

[0212] Step 1: Extract features from each modal data.

[0213] RGB image feature extraction: Convolutional neural network (CNN) is used to extract local and global visual features: ,in Output feature tensor for pre-trained visual feature extraction networks, such as ResNet ;

[0214] Deep image feature extraction: Extract depth information through a specific three-dimensional convolutional network (3D-CNN): ,in is a deep feature extractor that outputs feature tensors .

[0215] Point cloud feature extraction:

[0216] Use PointNet++ to extract features from point cloud data: ,in For point cloud processing network, output global feature vector .

[0217] Other modality feature extraction: Process other modality data through convolutional or recurrent neural networks: ,in for modality-specific feature extraction networks.

[0218] Step 2: Multimodal weighted fusion, using the attention mechanism to weight different modal features:

[0219]

[0220]

[0221] in, For modal The feature importance score of is the scoring function, ∈( , , ).

[0222] Step 3: Time series modeling, for dynamic environments, use GRU to capture the time series changes of features:

[0223]

[0224] 3. Multimodal Scene Feature Representation Method

[0225] Multimodal scene feature representation method uses the comprehensive features output by multimodal fusion method and timing characteristics , extract key information in the scene (such as obstacle locations, target points, etc.) and generate a graph structure. The process is as follows:

[0226] Step 1: Node construction: Each key point in the environment (obstacle, path node, target point) is used as a node V of the graph, and the node features are given by supply.

[0227]

[0228] in, is the feature mapping function, Indicates The characteristics of a node.

[0229] Step 2: Edge construction: The edges E in the graph are determined by the geometric relationship between nodes (such as Euclidean distance, connectivity), and the edge features are determined by the time series features. And the node distance is calculated:

[0230]

[0231] in, and For Node and spatial location, Generates functions for edge features.

[0232] Step 3: Scene graph expression: The constructed scene graph is represented as: ,in, , .

[0233] Step 4: Node update: Use graph neural network (GNN) to update the node features in the environment graph, combining the features of neighbor nodes and edge features:

[0234]

[0235] in:

[0236] Representation Node In the Characteristics of the layer;

[0237] Representation Node The set of neighbors of ;

[0238] It is the mapping function between edge features and neighbor node features;

[0239] It is the feature aggregation and update function.

[0240] Step 5: Global feature generation: after updating multiple layers of nodes, the environmental features of the entire graph are generated. , representing the entire scene:

[0241]

[0242] Among them, L is the number of layers of the GNN network, and the Pooling operation can be average pooling or maximum pooling.

[0243] 4. Task Modeling Method

[0244] The task modeling method builds task objectives and constraints based on the environmental features and scene graphs generated by the multimodal data representation module, combined with task requirements, to provide structured input for action planning. The specific modeling process is as follows:

[0245] Step 1: Determine the mission objectives: by environmental characteristics Extract task-related areas (such as target points, dangerous areas) and express the task objectives in quantitative form: , where each It is a task execution indicator, such as shortest path, maximum security, etc.

[0246] Step 2: Constraint modeling to ensure safe and efficient operation of drones in multimodal environments. Constraints are based on the scene graph. and global features Modeling, taking collision avoidance constraints and energy limit constraints as examples, the formula is as follows:

[0247] ① Collision avoidance constraints: the drone needs to avoid obstacles during movement, and its path should always be within the obstacle-free area:

[0248]

[0249] in: is a set of waypoints; Indicates the obstacle area; Indicates that the waypoint does not overlap with the obstacle area.

[0250] ②Energy limit constraint: the path planning of the drone must meet the energy budget, and the energy consumption should be less than the remaining power:

[0251]

[0252] in: ; Indicates from arrive energy consumption; The remaining battery power of the drone.

[0253] ③Other constraints are

[0254] Based on the above constraints, the constraint set for path planning is:

[0255]

[0256] Final path planning The solution must satisfy:

[0257]

[0258] Step 3: Reward function design. The reward function directly combines multimodal features to ensure the optimization of task indicators to guide the drone system to take correct actions. The following is a detailed description of the formula and parameters for reward function design:

[0259]

[0260] Among them: state s is the global feature output by the multimodal representation module and the mission objectives set Composition, namely:

[0261] ; Indicates taking action in state s The reward value of , The corresponding weight coefficient is used to balance the importance of different task objectives in the reward, which is usually determined through task scenario experience or experimental optimization; Bonus items related to task efficiency; For rewards related to mission safety; Reward items related to task accuracy; refers to the degree of violation of the constraint. If the constraint is satisfied, the penalty is 0. is a penalty factor used to control the penalty for constraint violation.

[0262] 5. Action Planning and Evaluation Methods

[0263] Action planning and evaluation methods based on task objectives generated by task modeling and constraints , using reinforcement learning to plan the optimal action sequence , and evaluate its execution effect to provide input for the task-aware optimization module.

[0264] Step 1: Action space definition, define the action set according to the drone's operating capabilities ,For example:

[0265] {change speed, adjust direction, ..., hover}

[0266] Furthermore, considering the constraints , for example: Collision Avoidance: Action Set Each action in the set must ensure that the drone does not enter the obstacle area; Energy limit: All action sequences must satisfy the energy consumption does not exceed the maximum limit. Legal action set As shown in the following formula:

[0267]

[0268] Step 2: Reinforcement learning model construction, using deep reinforcement learning (DRL) method to build a policy network and optimize action selection through iterative updates:

[0269]

[0270] in: is a discount factor that measures the importance of long-term rewards and short-term rewards; T is the time step of the task. represents the state of the drone at time t, This means to find the expected value: Indicates status Take action The reward value.

[0271] Step 3: Action evaluation, evaluate the value of the state-action pair through Q-learning ( , ):

[0272]

[0273] in: To perform actions The next state after is the optimal action in the next state; Indicates that from the current state Take Action Then, transfer to the next state , and choose the best action Expected return after (s, ) indicates that the state s is in the strategy Take action The cumulative reward that can be obtained represents the value of the current strategy and is used to select the optimal action.

[0274] In reinforcement learning, the strategy represents the probability of selecting action a in state s. The strategy guides the behavior of the agent by optimizing the reward function. Strategy optimization is based on the state-action value function To complete, Evaluate the expected reward of taking a specific action in a specific state.

[0275] During the strategy evaluation and optimization process, the system calculates To continuously optimize strategies , so that in each state, the selected action can maximize the long-term reward.

[0276] Step 4: Action sequence generation, based on policy network And Q-learning algorithm, generate a complete action sequence:

[0277]

[0278] This sequence is guaranteed to satisfy the constraints To achieve the mission objectives .

[0279] Step 5: Dynamically adjust pre-assessment and document strategies to cope with dynamic environments The output results and their adaptability provide a basis for the next step of task perception optimization. Record the state transition of each step , evaluate the global rewards and local execution effects of the sequence, laying the foundation for subsequent dynamic adjustment of strategies.

[0280] 6. Task-aware optimization method

[0281] The task-aware optimization method is based on the action sequences, strategies, and state transition data generated in the above steps, and dynamically adjusts according to real-time changes in the environment to optimize task execution efficiency, safety, and accuracy, providing a stable strategy and feedback mechanism for the "zero-sample migration module".

[0282] Step 1: Dynamic environment perception: Update the task objectives and constraints according to the changes in the environment and real-time feedback during task execution. The updated state is expressed as :

[0283]

[0284] in, is the new state, which represents the updated state representation according to the multimodal data and task objectives in the new environment;

[0285] To update the environmental features, the environmental change data (such as the movement of obstacles, changes in target points, etc.) is perceived in real time.

[0286] Combined state transition data , analyze the impact of dynamic environment on task objectives, such as: changes in obstacle positions, changes in path passability, and adjustments to task priorities.

[0287] Step 2: Dynamic strategy update, based on the new state representation , adjust the action strategy to ensure that the strategy can still achieve the task goal in the new environment:

[0288]

[0289] in, is the updated dynamic strategy, which indicates the probability of taking an action in the updated state; Refers to the expected return, which means that after executing an action, from the current state To begin with, the sum of rewards at each moment in the future; is the discount factor.

[0290] In a dynamic environment, the following two optimization mechanisms are used:

[0291] Local adjustment: The purpose of the local adjustment mechanism is to optimize each step in real time based on the current state and feedback during execution. This mechanism enables the strategy to respond to subtle changes and ensure accurate execution of the task. The following formula is to reselect the optimal action based on environmental changes The formula is:

[0292]

[0293] in, is the optimization action at the current moment, indicating the state at time step t Next, choose the best action ; State-action value function, which means that in state Next, take action The expected return that can be obtained.

[0294] Global adjustment: The global optimization mechanism is a comprehensive optimization of the entire task execution process. During the overall execution of the task, the strategy is re-evaluated and optimized to maximize the long-term return. When the environment changes significantly (such as the path is completely unavailable), the action sequence is regenerated:

[0295]

[0296] in, is the globally optimized action sequence, a new action sequence generated by evaluating the action selections at all time steps in the entire task .

[0297] Step 3: Resource optimization scheduling, combining the mission objectives and the simulated real-time environment, dynamically allocate the drone's resources (such as battery power, computing power) to optimize the overall mission efficiency:

[0298] The energy distribution is shown in the following formula:

[0299]

[0300] in To perform actions energy consumption; The remaining available battery energy.

[0301] It involves dynamically evaluating and adjusting the priorities of different tasks to ensure that limited resources are optimally allocated.

[0302] Task priority adjustment involves dynamically evaluating and adjusting the priorities of different tasks to ensure that limited resources are optimally allocated.

[0303] Task Priority It is determined by multiple factors, including the importance of the task, the remaining time, the degree of completion of the task, etc., as shown in the following formula:

[0304]

[0305] in, Indicates the urgency of the task; Indicates the importance of the task; Indicates the remaining time ratio of the task, reflecting the time urgency for completing the task.

[0306] When multiple tasks are executed simultaneously, the execution order of the tasks is dynamically adjusted according to the dynamic priority of each task:

[0307]

[0308] It is a collection of tasks sorted by priority, ensuring that the most urgent or important tasks are executed first.

[0309] Step 4: After the task is executed, generate evaluation data , including task completion rate, safety and resource utilization, etc. The specific formula is as follows:

[0310] Task completion rate:

[0311]

[0312] in, is the number of completed tasks, is the total number of tasks;

[0313] Safety, counting the number of collisions or potential dangers during task execution:

[0314]

[0315] in, is the number of collision-free tasks;

[0316] Resource utilization, the resources consumed to complete the calculation task:

[0317]

[0318] in, is the actual resource consumption, The total amount of available resources.

[0319] 7. Modal Alignment and Mapping Methods

[0320] The modality alignment and mapping method uses a generative adversarial network (GAN) to align the simulated and real environment features. The formula is as follows:

[0321]

[0322] Among them: X represents the real feature, Z represents the simulated feature; X~P Real Indicates that the true feature obeys PReal Distribution; Z~P sim Indicates that the simulation characteristics obey P sim distributed;

[0323] G is the generator, which simulates the features Transformed into data similar to the real features;

[0324] D is the discriminator, which distinguishes between simulated and real features;

[0325] The distribution of real environment features is obtained in real time through the task perception execution module;

[0326] To simulate the distribution of environmental characteristics, from the simulation data Environmental characteristics obtained from .

[0327] Through adversarial training, the generator G learns to map features in the simulation environment to corresponding features in the real environment, thereby achieving migration from simulation to reality.

[0328] Generate state alignment model , the implementation process is shown in the following formula:

[0329]

[0330] in and Distribute state representations of real and simulated environments.

[0331] 8. Strategy Migration Method

[0332] The strategy migration method will simulate the strategy By mapping the state alignment to the real environment, we can ensure that the strategy remains efficient in the real environment. The implementation process is shown in the following formula:

[0333]

[0334] in, The action selection strategy is obtained through reinforcement learning for the strategy in the simulation environment; To align the model across modalities, represent the state in the simulation environment Mapping to the state of the real environment ; To transfer the strategy to the real environment, it indicates the probability of selecting the optimal action under the state of the real environment.

[0335] IX. Online Adaptive Method

[0336] Online adaptive methods are used to adapt the transferred policies in real environments A comprehensive evaluation is conducted to identify deficiencies and optimize strategies through a feedback mechanism. The output optimization strategy directly reflects the improvement in the accuracy and generalization of the system, enabling the UAV system to maintain high accuracy and strong generalization in different scenarios.

[0337] Step 1: Perform adaptability assessment to evaluate task completion rate, safety, and resource utilization:

[0338]

[0339] in, is the task completion rate; It is a mission safety assessment; It is a resource utilization assessment.

[0340] Step 2: The environment status is updated online in real time. The latest status is generated through the environmental features collected by multimodal sensors and used as the input for policy optimization to ensure that the strategy adapts to environmental changes:

[0341]

[0342] in, For dynamic environment characteristics, As the mission objective. As the core input for strategy optimization, it ensures that the strategy adapts to environmental changes.

[0343] Step 3: Strategy update and optimization based on evaluation results Adjustment strategy:

[0344] Local adjustment: to the current state Select new action , to achieve instant optimization:

[0345]

[0346] in, is the optimal action at the current moment, that is, in state The best action to choose next;

[0347] is the state-action value function, which means that in the state The expected reward when taking action a.

[0348] Global optimization: Based on the feedback data accumulated during the entire task execution process, adjust the global direction of the strategy to balance task objectives, resource consumption and security:

[0349]

[0350] in, is the updated strategy, which represents the optimized strategy used to select the optimal action in state s; is the strategy after migration, the initial strategy migrated from the simulation environment to the real environment; represents the strategy increment optimized according to the evaluation results, Driven, the size of the increment depends on the feedback data during task execution; the update rule ensures that the strategy is adaptable in all environments.

[0351] Step 4: Action sequence generation and verification, use the updated strategy to generate a new action sequence:

[0352]

[0353] in, is the optimized action sequence; is the state-action value function, evaluating each time step t and executing the action Expected return when.

[0354] After generating a new action sequence, the system will combine the task feedback evaluation indicators Verify its effect to ensure that the optimized action sequence can achieve the expected goal.

[0355] The system of this embodiment has the following advantages:

[0356] 1. The multimodal data representation module aims to integrate multimodal perception data, extract features and complete fusion, and finally generate a structured environmental representation, which provides a unified input for the UAV's subsequent mission perception and execution, ensuring the accuracy and robustness of the system in multiple scenarios.

[0357] ① Through the fusion of multimodal data, the system can more accurately perceive detailed information in the environment (such as object location, target markers, obstacles), thereby improving the accuracy of task execution.

[0358] ② Multimodal perception ensures that the system can maintain high recognition accuracy and reliability in different environments (such as lighting changes, weather changes, etc.), enhancing the system's environmental adaptability.

[0359] 2. The task perception and execution module constructs the task scenario in the simulation environment based on the environmental features generated by the multimodal representation module, establishes a mathematical model of task execution, clarifies the task objectives, constraints and indicators, and finally optimizes the action sequence by combining reinforcement learning and dynamic programming.

[0360] ① Through reinforcement learning optimization strategies, ensure that the UAV can accurately perform tasks according to the predetermined goals (such as the shortest path, shortest time, etc.) and reduce execution errors.

[0361] ② Through real-time monitoring of environmental changes and feedback evaluation, the execution effect of the strategy is optimized to ensure the accuracy of task execution (such as collision avoidance, path accuracy, etc.).

[0362] ③Through task modeling and dynamic adjustment, the system can flexibly adjust strategies under different task objectives and environmental changes, reduce excessive reliance on specific scenarios, and ensure the applicability of strategies in various scenarios.

[0363] ④This module can dynamically update strategies based on the environment, allowing the drone to adapt to changes in various environments (such as sudden obstacles, different weather conditions), enhancing the system's performance in diverse scenarios.

[0364] 3. The zero-sample transfer module combines the dynamic strategies and feedback data output by the task perception execution module, and uses adversarial learning and adaptive technology to achieve high-fidelity cross-modal transfer. In the absence of real environment data, it achieves efficient transfer from simulation to reality and improves the generalization ability of the UAV system in multi-scenario tasks.

[0365] ① By aligning simulation data with real data and migrating strategies, the performance loss from simulation to reality is reduced, ensuring the accurate execution of the migrated strategy in the real environment.

[0366] ② The migrated strategy can adapt to multiple unknown scenarios without the need for additional labeled data. Through rapid migration and adaptive adjustment, the system's cross-scenario application capabilities are improved.

[0367] In order to implement the method of an embodiment of the present invention, an embodiment of the present invention also provides a drone control system with accuracy preservation and enhanced generalization capability, including: a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, the steps of the above-mentioned method are executed.

[0368] The above-mentioned system provided in this embodiment belongs to the same concept as the above-mentioned method embodiment. The specific implementation process thereof is detailed in the method embodiment and will not be repeated here.

[0369] In order to implement the method of the embodiment of the present invention, the embodiment of the present invention also provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the above method.

[0370] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention further provides an electronic device (computer device). Specifically, in one embodiment, the computer device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05 and a memory (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A06. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor A01, the method of any one of the above embodiments is implemented. The display screen A04 of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device A05 of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0371] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0372] The device provided by an embodiment of the present invention includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the method of any one of the above embodiments is implemented.

[0373] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0374] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0375] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0376] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0377] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0378] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0379] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0380] It can be understood that the memory of the embodiment of the present invention can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0381] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0382] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A drone control method with accuracy preservation and enhanced generalization capability, characterized in that: The method comprises: Acquire multimodal data collected by multiple sensors carried by the drone, use the multimodal data to build a scene graph, and use the scene graph to extract environmental features; determine the mission objectives, constraints and reward functions of the drone based on the mission requirements of the drone according to the scene graph and the environmental features; Based on the scene graph, the environmental features, the task objectives, the constraints and the reward function, a reinforcement learning model is constructed to generate an action strategy in a simulation environment; an operation action set of the drone is obtained, the operation action set is iteratively updated based on the action strategy in the simulation environment, and an action sequence is generated after the operation actions in the operation action set are evaluated in value; The multimodal data is updated in real time according to environmental changes, and the scene graph, environmental features, mission objectives of the drone, constraints and reward functions are regenerated using the updated multimodal data; the action strategy in the simulation environment is updated based on the regenerated scene graph, environmental features, the mission objectives of the drone, the constraints and the reward function; the action sequence of the drone is re-determined using the updated action strategy in the simulation environment; the drone mission is executed according to the re-determined action sequence of the drone, and evaluation data is generated after the mission is completed; Using a generative adversarial network to align the characteristics of the simulation environment with the characteristics of the real environment to generate a state alignment model; based on the state alignment model, migrating the action strategy in the simulation environment to the real environment to obtain the action strategy after migrating to the real environment; updating the real environment features in real time according to the environmental changes, and adjusting the action strategy after migrating the real environment according to the evaluation data, so as to generate the drone action sequence in the new real environment with the updated real environment features and the adjusted action strategy; The expression of the reward function of the drone is: Among them, the state s is composed of the environmental characteristics and mission objectives Composition, namely: ; Indicates taking action in state s The reward value of , is the weight coefficient, which is used to balance the importance of different task objectives in the reward; Bonus items related to task efficiency; For rewards related to mission safety; Reward items related to task accuracy; refers to the degree of violation of the constraint. If the constraint is satisfied, the penalty is 0. is a penalty factor used to control the penalty for constraint violation.

2. The method for controlling a drone with accuracy preservation and generalization enhancement according to claim 1, characterized in that: Acquire multimodal data collected by multiple sensors carried by the drone, use the multimodal data to build a scene graph, and use the scene graph to extract environmental features, including: Acquire multimodal data collected by multiple sensors carried by the drone, extract features from each modal data in the multimodal data, and fuse the features of all modal data to obtain fused features; extract time series features according to the time series changes of the fused features; Based on the fused features and the timing features, a scene graph is constructed; the nodes in the scene graph are multiple features after feature mapping of the fused features; the edges of the scene graph are obtained by calculating the distances between the nodes and the timing features; the nodes of the scene graph are updated, and environmental features are generated based on the updated scene graph.

3. The method for controlling a drone with accuracy preservation and enhanced generalization capability according to claim 1, characterized in that: The expression of the constraint condition of the UAV is: in, As constraints, For collision avoidance constraints, is the energy limit constraint, For other constraints; in, in, is a set of waypoints; Indicates the obstacle area; Indicates that the waypoint does not overlap with the obstacle area; in, ; Indicates from arrive energy consumption; The remaining battery power of the drone.

4. The method for controlling a drone with accuracy preservation and generalization enhancement according to claim 1, characterized in that: The expression of the action strategy in the simulation environment is: in, is the action strategy in the simulation environment, is a discount factor used to measure the importance of long-term rewards and short-term rewards; T is the time step of the task, represents the state of the drone at time t, , For environmental characteristics, For the mission objectives, This means to find the expected value: Indicates status Take action The reward value.

5. The method for controlling a drone with accuracy preservation and generalization enhancement according to claim 4, characterized in that: The operation action set is iteratively updated based on the action strategy in the simulation environment, and the operation actions in the operation action set are value evaluated to generate an action sequence, including: The following calculation formula is used to evaluate the value of the operation actions in the operation action set: in, (s, ) indicates that the state s is in the action strategy Take action The cumulative reward that can be obtained, which represents the value of the current strategy and is used to select the optimal action; Indicates status Take action The reward value of state s is determined by the environmental characteristics and mission objectives Composition, that is ; is a discount factor used to weigh the importance of long-term rewards and short-term rewards; Indicates that from the current state Take Action Then, transfer to the next state , and choose the best action Expected return after The action sequence is generated using the following calculation formula: in, Indicates the state at time t Strategy in Action Take action The cumulative rewards that can be obtained.

6. The method for controlling a drone with accuracy preservation and generalization enhancement according to claim 1, characterized in that: Re-determining the action sequence of the drone using the updated action strategy in the simulation environment includes: Re-determining the optimized action of the drone according to the current state of the drone and the updated action strategy in the simulation environment; and regenerating the action sequence of the drone according to the current state of the drone and the updated action strategy in the simulation environment; The optimization action of the drone is re-determined using the following calculation formula: in, is the optimization action at the current moment, indicating the state at time step t Next, choose the best action , is the state-action value function, which means that in the state Strategy in Action Take action The expected return that can be obtained when The action sequence of the drone is re-determined using the following calculation formula: in, To re-determine the drone action sequence, is the state-action value function, which means that in the state Strategy in Action Take action The expected return that can be obtained.

7. The method for controlling a drone with accuracy preservation and generalization enhancement according to claim 1, characterized in that: The evaluation data includes task completion rate, safety and resource utilization; The calculation formula of the task completion rate is: in, is the task completion rate, is the number of completed tasks, is the total number of tasks; The calculation formula of the safety is: in, For safety, is the number of collision-free tasks, is the total number of tasks; The calculation formula of the resource utilization is: in, is the resource utilization rate, is the actual resource consumption, The total amount of available resources.

8. The method for controlling a drone with accuracy preservation and generalization enhancement according to claim 1, characterized in that: The action strategy after the migration to the real environment is adjusted according to the evaluation data, including: The action strategy after migrating to the real environment is adjusted according to the evaluation data through the following calculation formula: in, is the updated action strategy, which is used to select the optimal action in state s; To transfer the action strategy after the real environment; Represents the increment of the strategy optimized based on the evaluation data.

9. A UAV control system with accuracy preservation and enhanced generalization capability, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, the steps of the method according to any one of claims 1 to 8 are executed.

Images